mindspore_gs.ptq.AutoQuantForCausalLM

class mindspore_gs.ptq.AutoQuantForCausalLM

Auto Model Quantization Class

This class provides automatic model detection and selection for quantizing causal language models. It uses a registry mechanism to automatically identify and instantiate the appropriate quantized model implementation based on the pretrained model configuration.

The class implements a factory pattern that scans through all registered model hubs and attempts to create a model instance from each one until a successful match is found.

Examples

>>> from mindspore_gs.ptq.models import AutoQuantForCausalLM
>>>
>>> # Automatically select the appropriate model implementation
>>> model = AutoQuantForCausalLM.from_pretrained("/path/to/model.yaml")

static from_pretrained(pretained)

Create a quantized model instance from a pretrained model path.

This method automatically detects the model type from the provided pretrained model path and selects the appropriate quantized model implementation. It iterates through all registered model hubs and attempts to create a model instance from each one.

Parameters

pretained (str) – Path or identifier of the pretrained model. This can be a local file path to a model configuration file or a model identifier recognized by the system.

Returns

BaseQuantForCausalLM. A quantized model instance that inherits from BaseQuantForCausalLM. The specific type depends on the detected model framework and configuration.

Examples

>>> from mindspore_gs.ptq.models import AutoQuantForCausalLM
>>>
>>> # Automatically select Qwen3 model implementation
>>> model = AutoQuantForCausalLM.from_pretrained("/path/to/qwen3_model.yaml")
>>>
>>> # Automatically select DeepSeekV3 model implementation
>>> model = AutoQuantForCausalLM.from_pretrained("/path/to/deepseek_config.yaml")