mindspore_lite.LLMReq

class mindspore_lite.LLMReq(prompt_cluster_id: int, req_id: int, prompt_length: int)[source]

LLMEngine request, used to represent a multi round inference task.

property decoder_cluster_id: Get decoder cluster id of this inference task in LLMEngine

static next_req_id()[source]: Get next request id of this inference task

property prefix_id: Get decoder prefix id of this inference task in LLMEngine

property prompt_cluster_id: Get prompt cluster id of this inference task in LLMEngine

property prompt_length: Set prompt length of this inference task

property req_id: Get request id of this inference task

property sequence_length: Get decoder sequence length of this inference task in LLMEngine