MindSpore Transformers
介绍
整体架构
模型库
安装
安装指南
大模型全流程指南
预训练
SFT微调
推理
服务化部署
功能特性
启动任务
Ckpt权重
Safetensors权重
配置文件说明
加载 Hugging Face 模型配置
日志
使用Tokenizer
训练功能
推理功能
高阶开发
大模型精度调优指南
大模型性能调优指南
开发迁移
推理配置模板使用指南
与 Megatron-LM 比对训练精度
API
优秀实践
制作 MindSpore Transformers 的 Docker 镜像的实践案例
使用DeepSeek-R1进行模型蒸馏的实践案例
转换模型权重为Megatron模型权重的实践案例
环境变量
环境变量说明
贡献指南
MindSpore Transformers贡献指南
魔乐社区贡献指南
FAQ
模型相关 FAQ
功能相关 FAQ
MindSpore Transformers
»
索引
索引
A
|
B
|
C
|
E
|
F
|
G
|
I
|
M
|
N
|
P
|
R
|
S
|
T
|
U
|
內
A
added_tokens_decoder() (mindformers.models.PreTrainedTokenizer 类方法)
(mindformers.models.PreTrainedTokenizerFast 类方法)
added_tokens_encoder() (mindformers.models.PreTrainedTokenizer 类方法)
(mindformers.models.PreTrainedTokenizerFast 类方法)
B
batch() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
build_conversation_input_text() (mindformers.models.multi_modal.ModalContentTransformTemplate 类方法)
build_inputs_with_special_tokens() (mindformers.models.LlamaTokenizer 方法)
(mindformers.models.LlamaTokenizerFast 方法)
build_labels() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
build_modal_context() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
C
can_generate() (mindformers.models.PreTrainedModel 类方法)
chat() (mindformers.generation.GenerationMixin 方法)
clear() (mindformers.core.EmF1Metric 方法)
(mindformers.core.EntityScore 方法)
(mindformers.core.PerplexityMetric 方法)
(mindformers.core.PromptAccMetric 方法)
convert_args_to_mindformers_config() (mindformers.TrainingArguments 方法)
convert_ids_to_tokens() (mindformers.models.PreTrainedTokenizer 方法)
(mindformers.models.PreTrainedTokenizerFast 方法)
convert_tokens_to_ids() (mindformers.models.PreTrainedTokenizer 方法)
(mindformers.models.PreTrainedTokenizerFast 方法)
create_token_type_ids_from_sequences() (mindformers.models.LlamaTokenizer 方法)
E
eval() (mindformers.core.EmF1Metric 方法)
(mindformers.core.EntityScore 方法)
(mindformers.core.PerplexityMetric 方法)
(mindformers.core.PromptAccMetric 方法)
evaluate() (mindformers.Trainer 方法)
F
finetune() (mindformers.Trainer 方法)
forward() (mindformers.generation.GenerationMixin 方法)
from_config() (mindformers.AutoModel 类方法)
(mindformers.AutoModelForCausalLM 类方法)
from_dict() (mindformers.models.PretrainedConfig 类方法)
from_json_file() (mindformers.models.PretrainedConfig 类方法)
from_pretrained() (mindformers.AutoConfig 类方法)
(mindformers.AutoModel 类方法)
(mindformers.AutoModelForCausalLM 类方法)
(mindformers.AutoProcessor 类方法)
(mindformers.AutoTokenizer 类方法)
(mindformers.models.PretrainedConfig 类方法)
(mindformers.models.PreTrainedModel 类方法)
G
generate() (mindformers.generation.GenerationMixin 方法)
get_added_vocab() (mindformers.models.PreTrainedTokenizer 方法)
(mindformers.models.PreTrainedTokenizerFast 方法)
get_cls() (mindformers.tools.register.MindFormerRegister 类方法)
get_config_dict() (mindformers.models.PretrainedConfig 类方法)
get_instance() (mindformers.tools.register.MindFormerRegister 类方法)
get_instance_from_cfg() (mindformers.tools.register.MindFormerRegister 类方法)
get_moe_config() (mindformers.TrainingArguments 方法)
get_need_update_output_items() (mindformers.models.multi_modal.ModalContentTransformTemplate 静态方法)
get_parallel_config() (mindformers.TrainingArguments 方法)
get_recompute_config() (mindformers.TrainingArguments 方法)
get_special_tokens_mask() (mindformers.models.LlamaTokenizer 方法)
get_warmup_steps() (mindformers.TrainingArguments 方法)
I
infer() (mindformers.generation.GenerationMixin 方法)
is_exist() (mindformers.tools.register.MindFormerRegister 类方法)
M
merge_from_dict() (mindformers.tools.MindFormerConfig 方法)
mindformers.AutoConfig (內置类)
mindformers.AutoModel (內置类)
mindformers.AutoModelForCausalLM (內置类)
mindformers.AutoProcessor (內置类)
mindformers.AutoTokenizer (內置类)
mindformers.core.AdamW (內置类)
mindformers.core.build_context()
內置函数
mindformers.core.CheckpointMonitor (內置类)
mindformers.core.ConstantWarmUpLR (內置类)
mindformers.core.ConstantWithCoolDownLR (內置类)
mindformers.core.CosineAnnealingLR (內置类)
mindformers.core.CosineAnnealingWarmRestarts (內置类)
mindformers.core.CosineWithRestartsAndWarmUpLR (內置类)
mindformers.core.CosineWithWarmUpLR (內置类)
mindformers.core.CrossEntropyLoss (內置类)
mindformers.core.EmF1Metric (內置类)
mindformers.core.EntityScore (內置类)
mindformers.core.EvalCallBack (內置类)
mindformers.core.get_context()
內置函数
mindformers.core.init_context()
內置函数
mindformers.core.LearningRateWiseLayer (內置类)
mindformers.core.LinearWithWarmUpLR (內置类)
mindformers.core.MFLossMonitor (內置类)
mindformers.core.PerplexityMetric (內置类)
mindformers.core.PolynomialWithWarmUpLR (內置类)
mindformers.core.ProfileMonitor (內置类)
mindformers.core.PromptAccMetric (內置类)
mindformers.core.set_context()
內置函数
mindformers.core.SummaryMonitor (內置类)
mindformers.core.TrainingStateMonitor (內置类)
mindformers.core.WarmUpStableDecayLR (內置类)
mindformers.dataset.CausalLanguageModelDataset (內置类)
mindformers.dataset.KeyWordGenDataset (內置类)
mindformers.dataset.MultiTurnDataset (內置类)
mindformers.generation.GenerationConfig (內置类)
mindformers.generation.GenerationMixin (內置类)
mindformers.ModelRunner (內置类)
mindformers.models.ChatGLM2Config (內置类)
mindformers.models.ChatGLM2ForConditionalGeneration (內置类)
mindformers.models.ChatGLM3Tokenizer (內置类)
mindformers.models.ChatGLM4Tokenizer (內置类)
mindformers.models.LlamaConfig (內置类)
mindformers.models.LlamaForCausalLM (內置类)
mindformers.models.LlamaTokenizer (內置类)
mindformers.models.LlamaTokenizerFast (內置类)
mindformers.models.multi_modal.ModalContentTransformTemplate (內置类)
mindformers.models.PretrainedConfig (內置类)
mindformers.models.PreTrainedModel (內置类)
mindformers.models.PreTrainedTokenizer (內置类)
mindformers.models.PreTrainedTokenizerFast (內置类)
mindformers.modules.OpParallelConfig (內置类)
mindformers.pet.models.LoraModel (內置类)
mindformers.pet.pet_config.LoraConfig (內置类)
mindformers.pet.pet_config.PetConfig (內置类)
mindformers.pipeline()
內置函数
mindformers.pipeline.MultiModalToTextPipeline (內置类)
mindformers.run_check()
內置函数
mindformers.tools.MindFormerConfig (內置类)
mindformers.tools.register.MindFormerModuleType (內置类)
mindformers.tools.register.MindFormerRegister (內置类)
mindformers.Trainer (內置类)
mindformers.TrainingArguments (內置类)
mindformers.wrapper.MFPipelineWithLossScaleCell (內置类)
mindformers.wrapper.MFTrainOneStepCell (內置类)
N
num_special_tokens_to_add() (mindformers.models.PreTrainedTokenizer 方法)
(mindformers.models.PreTrainedTokenizerFast 方法)
P
post_init() (mindformers.models.PreTrainedModel 方法)
post_process() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
postprocess() (mindformers.generation.GenerationMixin 方法)
predict() (mindformers.Trainer 方法)
prepare_for_tokenization() (mindformers.models.PreTrainedTokenizer 方法)
process_predict_query() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
process_train_item() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
R
register() (mindformers.AutoConfig 类方法)
(mindformers.AutoModel 类方法)
(mindformers.AutoModelForCausalLM 类方法)
(mindformers.AutoProcessor 类方法)
(mindformers.AutoTokenizer 类方法)
(mindformers.tools.register.MindFormerRegister 类方法)
register_cls() (mindformers.tools.register.MindFormerRegister 类方法)
register_for_auto_class() (mindformers.models.PreTrainedModel 类方法)
S
save_pretrained() (mindformers.models.PretrainedConfig 方法)
(mindformers.models.PreTrainedModel 方法)
save_vocabulary() (mindformers.models.LlamaTokenizerFast 方法)
set_dataloader() (mindformers.TrainingArguments 方法)
set_logging() (mindformers.TrainingArguments 方法)
set_lr_scheduler() (mindformers.TrainingArguments 方法)
set_optimizer() (mindformers.TrainingArguments 方法)
set_save() (mindformers.TrainingArguments 方法)
set_training() (mindformers.TrainingArguments 方法)
set_truncation_and_padding() (mindformers.models.PreTrainedTokenizerFast 方法)
show_support_list() (mindformers.AutoConfig 类方法)
supported_modal() (mindformers.models.multi_modal.ModalContentTransformTemplate 类方法)
T
to_dict() (mindformers.models.PretrainedConfig 方法)
to_diff_dict() (mindformers.models.PretrainedConfig 方法)
to_json_file() (mindformers.models.PretrainedConfig 方法)
to_json_string() (mindformers.models.PretrainedConfig 方法)
tokenize() (mindformers.models.PreTrainedTokenizer 方法)
train() (mindformers.Trainer 方法)
train_new_from_iterator() (mindformers.models.PreTrainedTokenizerFast 方法)
U
update() (mindformers.core.EmF1Metric 方法)
(mindformers.core.EntityScore 方法)
(mindformers.core.PerplexityMetric 方法)
(mindformers.core.PromptAccMetric 方法)
update_post_processor() (mindformers.models.LlamaTokenizerFast 方法)
內
內置函数
mindformers.core.build_context()
mindformers.core.get_context()
mindformers.core.init_context()
mindformers.core.set_context()
mindformers.pipeline()
mindformers.run_check()