MindSpore Transformers
介绍
整体架构
模型库
安装
安装指南
大模型全流程指南
训练指南
预训练实践
监督微调实践
推理指南
服务化部署指南
评测指南
功能特性
启动任务
Ckpt权重
Safetensors权重
配置文件说明
加载 Hugging Face 模型配置
日志
使用Tokenizer
训练功能
推理功能
高阶开发
大模型精度调优指南
大模型性能调优指南
开发迁移
推理配置模板使用指南
推理精度比对
与 Megatron-LM 比对训练精度
训练配置模板使用说明
权重转换开发适配
API
优秀实践
制作 MindSpore Transformers 的 Docker 镜像的实践案例
使用DeepSeek-R1进行模型蒸馏的实践案例
转换模型权重为Megatron模型权重的实践案例
环境变量
环境变量说明
贡献指南
MindSpore Transformers贡献指南
魔乐社区贡献指南
FAQ
模型相关 FAQ
功能相关 FAQ
RELEASE NOTES
Release Notes
MindSpore Transformers
»
索引
索引
A
|
B
|
C
|
E
|
F
|
G
|
I
|
M
|
N
|
P
|
R
|
S
|
T
|
U
|
內
A
added_tokens_decoder() (mindformers.models.PreTrainedTokenizer 类方法)
(mindformers.models.PreTrainedTokenizerFast 类方法)
added_tokens_encoder() (mindformers.models.PreTrainedTokenizer 类方法)
(mindformers.models.PreTrainedTokenizerFast 类方法)
B
batch() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
build_conversation_input_text() (mindformers.models.multi_modal.ModalContentTransformTemplate 类方法)
build_inputs_with_special_tokens() (mindformers.models.LlamaTokenizer 方法)
(mindformers.models.LlamaTokenizerFast 方法)
build_labels() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
build_modal_context() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
C
can_generate() (mindformers.models.PreTrainedModel 类方法)
chat() (mindformers.generation.GenerationMixin 方法)
clear() (mindformers.core.EmF1Metric 方法)
(mindformers.core.EntityScore 方法)
(mindformers.core.PerplexityMetric 方法)
(mindformers.core.PromptAccMetric 方法)
convert_args_to_mindformers_config() (mindformers.TrainingArguments 方法)
convert_ids_to_tokens() (mindformers.models.PreTrainedTokenizer 方法)
(mindformers.models.PreTrainedTokenizerFast 方法)
convert_tokens_to_ids() (mindformers.models.PreTrainedTokenizer 方法)
(mindformers.models.PreTrainedTokenizerFast 方法)
create_token_type_ids_from_sequences() (mindformers.models.LlamaTokenizer 方法)
E
eval() (mindformers.core.EmF1Metric 方法)
(mindformers.core.EntityScore 方法)
(mindformers.core.PerplexityMetric 方法)
(mindformers.core.PromptAccMetric 方法)
evaluate() (mindformers.Trainer 方法)
F
finetune() (mindformers.Trainer 方法)
forward() (mindformers.generation.GenerationMixin 方法)
from_config() (mindformers.AutoModel 类方法)
(mindformers.AutoModelForCausalLM 类方法)
from_dict() (mindformers.models.PretrainedConfig 类方法)
from_json_file() (mindformers.models.PretrainedConfig 类方法)
from_pretrained() (mindformers.AutoConfig 类方法)
(mindformers.AutoModel 类方法)
(mindformers.AutoModelForCausalLM 类方法)
(mindformers.AutoProcessor 类方法)
(mindformers.AutoTokenizer 类方法)
(mindformers.models.PretrainedConfig 类方法)
(mindformers.models.PreTrainedModel 类方法)
G
generate() (mindformers.generation.GenerationMixin 方法)
get_added_vocab() (mindformers.models.PreTrainedTokenizer 方法)
(mindformers.models.PreTrainedTokenizerFast 方法)
get_cls() (mindformers.tools.register.MindFormerRegister 类方法)
get_config_dict() (mindformers.models.PretrainedConfig 类方法)
get_instance() (mindformers.tools.register.MindFormerRegister 类方法)
get_instance_from_cfg() (mindformers.tools.register.MindFormerRegister 类方法)
get_moe_config() (mindformers.TrainingArguments 方法)
get_need_update_output_items() (mindformers.models.multi_modal.ModalContentTransformTemplate 静态方法)
get_parallel_config() (mindformers.TrainingArguments 方法)
get_recompute_config() (mindformers.TrainingArguments 方法)
get_special_tokens_mask() (mindformers.models.LlamaTokenizer 方法)
get_warmup_steps() (mindformers.TrainingArguments 方法)
I
infer() (mindformers.generation.GenerationMixin 方法)
is_exist() (mindformers.tools.register.MindFormerRegister 类方法)
M
merge_from_dict() (mindformers.tools.MindFormerConfig 方法)
mindformers.AutoConfig (內置类)
mindformers.AutoModel (內置类)
mindformers.AutoModelForCausalLM (內置类)
mindformers.AutoProcessor (內置类)
mindformers.AutoTokenizer (內置类)
mindformers.core.AdamW (內置类)
mindformers.core.build_context()
內置函数
mindformers.core.CheckpointMonitor (內置类)
mindformers.core.ConstantWarmUpLR (內置类)
mindformers.core.ConstantWithCoolDownLR (內置类)
mindformers.core.CosineAnnealingLR (內置类)
mindformers.core.CosineAnnealingWarmRestarts (內置类)
mindformers.core.CosineWithRestartsAndWarmUpLR (內置类)
mindformers.core.CosineWithWarmUpLR (內置类)
mindformers.core.CrossEntropyLoss (內置类)
mindformers.core.EmF1Metric (內置类)
mindformers.core.EntityScore (內置类)
mindformers.core.EvalCallBack (內置类)
mindformers.core.get_context()
內置函数
mindformers.core.init_context()
內置函数
mindformers.core.LearningRateWiseLayer (內置类)
mindformers.core.LinearWithWarmUpLR (內置类)
mindformers.core.MFLossMonitor (內置类)
mindformers.core.PerplexityMetric (內置类)
mindformers.core.PolynomialWithWarmUpLR (內置类)
mindformers.core.ProfileMonitor (內置类)
mindformers.core.PromptAccMetric (內置类)
mindformers.core.set_context()
內置函数
mindformers.core.SummaryMonitor (內置类)
mindformers.core.TrainingStateMonitor (內置类)
mindformers.core.WarmUpStableDecayLR (內置类)
mindformers.dataset.CausalLanguageModelDataset (內置类)
mindformers.dataset.KeyWordGenDataset (內置类)
mindformers.dataset.MultiTurnDataset (內置类)
mindformers.generation.GenerationConfig (內置类)
mindformers.generation.GenerationMixin (內置类)
mindformers.ModelRunner (內置类)
mindformers.models.ChatGLM2Config (內置类)
mindformers.models.ChatGLM2ForConditionalGeneration (內置类)
mindformers.models.ChatGLM3Tokenizer (內置类)
mindformers.models.ChatGLM4Tokenizer (內置类)
mindformers.models.LlamaConfig (內置类)
mindformers.models.LlamaForCausalLM (內置类)
mindformers.models.LlamaTokenizer (內置类)
mindformers.models.LlamaTokenizerFast (內置类)
mindformers.models.multi_modal.ModalContentTransformTemplate (內置类)
mindformers.models.PretrainedConfig (內置类)
mindformers.models.PreTrainedModel (內置类)
mindformers.models.PreTrainedTokenizer (內置类)
mindformers.models.PreTrainedTokenizerFast (內置类)
mindformers.modules.OpParallelConfig (內置类)
mindformers.pet.models.LoraModel (內置类)
mindformers.pet.pet_config.LoraConfig (內置类)
mindformers.pet.pet_config.PetConfig (內置类)
mindformers.pipeline()
內置函数
mindformers.pipeline.MultiModalToTextPipeline (內置类)
mindformers.run_check()
內置函数
mindformers.tools.MindFormerConfig (內置类)
mindformers.tools.register.MindFormerModuleType (內置类)
mindformers.tools.register.MindFormerRegister (內置类)
mindformers.Trainer (內置类)
mindformers.TrainingArguments (內置类)
mindformers.wrapper.MFPipelineWithLossScaleCell (內置类)
mindformers.wrapper.MFTrainOneStepCell (內置类)
N
num_special_tokens_to_add() (mindformers.models.PreTrainedTokenizer 方法)
(mindformers.models.PreTrainedTokenizerFast 方法)
P
post_init() (mindformers.models.PreTrainedModel 方法)
post_process() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
postprocess() (mindformers.generation.GenerationMixin 方法)
predict() (mindformers.Trainer 方法)
prepare_for_tokenization() (mindformers.models.PreTrainedTokenizer 方法)
process_predict_query() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
process_train_item() (mindformers.models.multi_modal.ModalContentTransformTemplate 方法)
R
register() (mindformers.AutoConfig 类方法)
(mindformers.AutoModel 类方法)
(mindformers.AutoModelForCausalLM 类方法)
(mindformers.AutoProcessor 类方法)
(mindformers.AutoTokenizer 类方法)
(mindformers.tools.register.MindFormerRegister 类方法)
register_cls() (mindformers.tools.register.MindFormerRegister 类方法)
register_for_auto_class() (mindformers.models.PreTrainedModel 类方法)
S
save_pretrained() (mindformers.models.PretrainedConfig 方法)
(mindformers.models.PreTrainedModel 方法)
save_vocabulary() (mindformers.models.LlamaTokenizerFast 方法)
set_dataloader() (mindformers.TrainingArguments 方法)
set_logging() (mindformers.TrainingArguments 方法)
set_lr_scheduler() (mindformers.TrainingArguments 方法)
set_optimizer() (mindformers.TrainingArguments 方法)
set_save() (mindformers.TrainingArguments 方法)
set_training() (mindformers.TrainingArguments 方法)
set_truncation_and_padding() (mindformers.models.PreTrainedTokenizerFast 方法)
show_support_list() (mindformers.AutoConfig 类方法)
supported_modal() (mindformers.models.multi_modal.ModalContentTransformTemplate 类方法)
T
to_dict() (mindformers.models.PretrainedConfig 方法)
to_diff_dict() (mindformers.models.PretrainedConfig 方法)
to_json_file() (mindformers.models.PretrainedConfig 方法)
to_json_string() (mindformers.models.PretrainedConfig 方法)
tokenize() (mindformers.models.PreTrainedTokenizer 方法)
train() (mindformers.Trainer 方法)
train_new_from_iterator() (mindformers.models.PreTrainedTokenizerFast 方法)
U
update() (mindformers.core.EmF1Metric 方法)
(mindformers.core.EntityScore 方法)
(mindformers.core.PerplexityMetric 方法)
(mindformers.core.PromptAccMetric 方法)
update_post_processor() (mindformers.models.LlamaTokenizerFast 方法)
內
內置函数
mindformers.core.build_context()
mindformers.core.get_context()
mindformers.core.init_context()
mindformers.core.set_context()
mindformers.pipeline()
mindformers.run_check()