# Environment Variable Descriptions
[](https://gitee.com/mindspore/docs/blob/r2.7.1/docs/mindformers/docs/source_en/env_variables.md)
The following environment variables are supported by MindSpore Transformers.
## Debugging Variables
| Variables Names | Default | Interpretations | Descriptions | Application Scenarios |
|--------------------------------------------|---------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **HCCL_DETERMINISTIC** | false | Whether to enable deterministic computation of reductive communication operators, where reductive communication operators include AllReduce, ReduceScatter, Reduce. | `true`: turns on the HCCL deterministic switch;
`false`: turns off the HCCL deterministic switch. | Turning on deterministic computation eliminates the randomness introduced by inconsistent ordering of multi-card computations, but it results in a performance degradation compared to turning it off. It is recommended to turn it on in scenarios where consistency is required. |
| **LCCL_DETERMINISTIC** | 0 | whether to turn the LCCL deterministic operator AllReduce (order-preserving addition) on. | `1`: turns on the LCCL deterministic switch;
`0`: turns off the LCCL deterministic switch. | Turning on deterministic computation eliminates the randomness introduced by inconsistent ordering of multi-card computations, but it results in a performance degradation compared to turning it off. It is recommended to turn it on in scenarios where consistency is required.
Takes effect when rankSize<=8. |
| **CUSTOM_MATMUL_SHUFFLE** | on | Whether to enable shuffle operations for custom matrix multiplication. | `on`: turns on matrix shuffle;
`off`: turns off matrix shuffle. | The shuffle operation is optimized for specific matrix sizes and memory access patterns. If the matrix size does not match the shuffle-optimized size, turning off shuffling may result in better performance. Please set it according to the actual scenario. |
| **ASCEND_LAUNCH_BLOCKING** | 0 | In training or online inference scenarios, this environment variable can be used to control whether synchronization mode is activated during operator execution. | `1`: synchronized mode is mandatory;
`0`: synchronized mode is optional. | Since the default operator executes asynchronously during NPU model training, when an error is reported during operator execution, the error stack information printed is not the actual call stack information. When set to `1`, synchronized mode is mandatory, which prints the correct call stack information and makes it easier to debug and locate problems in the code. Setting it to `1` provides more efficient arithmetic. |
| **TE_PARALLEL_COMPILER** | 8 | The number of threads on which the operator is compiled in parallel. Enables parallel compilation when greater than 1. | Takes a positive integer;Maximum number of cpu cores\*80%/number of Ascend AI processors, value range 1 to 32, default value is 8. | When the network model is large, parallel compilation of the operator can be turned on by configuring this environment variable;
setting it to `1` for single-threaded compilation simplifies the difficulty when debugging. |
| **CPU_AFFINITY** | 0 | Turn on the CPU affinity switch, thus ensuring that each process or thread is bound to a single CPU core to improve performance. | `1`: turn on the CPU affinity switch;
`0`: turn off the CPU affinity switch. | CPU affinity is turned off by default for **optimized resource utilization** and **energy saving**. |
| **MS_MEMORY_STATISTIC** | 0 | Memory Statistics. | `1`: turn on memory statistics;
`0`: turn off memory statistics. | During memory analysis, basic memory usage can be counted. You can refer to [Optimization Guide](https://www.mindspore.cn/mindformers/docs/en/r1.7.0/advanced_development/performance_optimization.html) for details. |
| **MINDSPORE_DUMP_CONFIG** | NA | Specify the path to the configuration file that the [cloud-side Dump function](https://www.mindspore.cn/tutorials/en/r2.7.1/debug/dump.html) or [end-side Dump function](https://www.mindspore.cn/lite/docs/en/r2.7.1/tools/benchmark_tool.html#dump) depends on. | File path, support relative path and absolute path. |
| **GLOG_v** | 3 | Controls the level of MindSpore logs. | `0`: DEBUG
`1`: INFO
`2`: WARNING
`3`: ERROR: indicates that an error has been reported in the execution of the program, an error log is output, and the program may not be terminated;
`4`: CRITICAL, indicates that an exception has occurred in the execution of the program, and the execution of the program will be terminated. |
| **ASCEND_GLOBAL_LOG_LEVEL** | 3 | Controls the logging level of CANN. | `0`: DEBUG
`1`: INFO
`2`: WARNING
`3`: ERROR
`4`: NULL, no log is output. |
| **ASCEND_SLOG_PRINT_TO_STDOUT** | 0 | Whether to display on the screen. When turned on, the logs will not be saved in the log file, but the generated logs will be displayed directly on the screen. | `1`: Display on the screen
`0`: Do not display on the screen |
| **ASCEND_GLOBAL_EVENT_ENABLE** | 0 | Whether to enable event logging. | `1`: turn on Event logging;
`0`: turn off Event logging. |
| **HCCL_EXEC_TIMEOUT** | 1836 | This environment variable allows you to control the amount of time to wait for synchronization when executing between devices, where each device process waits for the other device to perform communication synchronization for the configured amount of time. | The range is: (0, 17340], and the default value is 1836 in seconds. |
| **HCCL_CONNECT_TIMEOUT** | 120 | Used in distributed training or inference scenarios to limit the timeout wait time of the socket building process between different devices. | The environment variable needs to be configured as an integer in the range [120,7200], with the default value of 120s. |
| **MS_NODE_ID** | NA | Specifies process rank id in dynamic cluster scenarios. | The rank_id of the process, unique within the cluster. |
| **MS_ALLOC_CONF** | NA | Sets memory allocation policies. | Configuration items, formatted as key:value, with multiple items separated by commas. For example: export MS_ALLOC_CONF=enable_vmm:true,memory_tracker:true.
enable_vmm: Whether to enable virtual memory; default value is true.
vmm_align_size: Sets virtual memory alignment size in MB; default value is 2.
memory_tracker: Whether to enable memory tracker; default value is false.
memory_tracker_path: Enables memory tracker and saves to specified path. Default is disabled with empty save path.
simple_tracker: Whether to enable simplified tracker mode, omitting tracker_graph.ir and retaining only the last user task. Takes effect when memory_tracker is enabled. Default is false.
acl_allocator: Whether to use the ACL memory allocator. Default value is true.
somas_whole_block: Whether to use SOMAS whole-block memory allocation. Default value is false. | |
| **MS_INTERNAL_DISABLE_CUSTOM_KERNEL_LIST** | PagedAttention | Enables a list of custom operators. An experimental configuration item, generally not required. Will be removed in future. | Configured as a string, with operator names separated by commas. | |
| **TRANSFORMERS_OFFLINE** | 0 | Forces the Auto interface to read only offline local files. | `1`, `ON`, `TRUE`, `YES`: Forces reading only offline local files;
Other values: Allows downloading files from the network. | |
| **MDS_ENDPOINT** | https://modelers.cn | Sets the endpoint for openMind Hub. | Configured as a URL address in string format. | |
| **OM_MODULES_CACHE** | ~/.cache/openmind/modules | Cache path for openMind modules. | Configured as a directory path in string format. | |
| **OPENMIND_CACHE** | ~/.cache/openmind/hub | Cache path for openMind Hub. | Configured as a directory path in string format. | |
| **openmind_IS_CI** | | Indicates whether openMind is operating within a CI access control environment. | `1`, `ON`, `TRUE`, `YES`: Within CI environment;
All other values: Not within CI environment. | |
## Other Variables
| Variables Names | Default | Interpretations | Descriptions | Application Scenarios |
|--------------------------------------|--------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **RUN_MODE** | predict | Set the running mode. | `predict`: inference
`finetune`: Fine-tuning
`train`: Training
`eval`: Evaluation |
| **USE_ROPE_SELF_DEFINE** | true | Whether to enable ROPE fusion operator. | `true`: enable ROPE fusion operator;
`false`: disable ROPE fusion operator. | Enabling the ROPE fusion operator by default can improve the computation efficiency. Except for debugging scenarios, turn it off as needed, and generally do not make special settings. |
| **MS_ENABLE_INTERNAL_BOOST** | on | Whether to turn on the internal acceleration of the MindSpore framework. | `on`: turn on MindSpore internal acceleration;
`off`: turn off MindSpore internal acceleration. | In order to achieve high-performance inference, this parameter is turned on by default. In cases where debugging or comparing different acceleration strategies is performed, this parameter needs to be turned off to observe the impact on performance. |
| **MS_GE_ATOMIC_CLEAN_POLICY** | 1 | Whether to clean up the memory occupied by atomic operators in the network. | `0`: centralized cleanup of memory occupied by all atomic operators in the network;
`1`: no centralized memory cleanup, individual zeroing of each atomic operator in the network. | The switch is set to `1` by default, which makes it easy for the user to process each operator individually, allowing operations such as operator memory reuse. Setting it to `0` centrally cleans up the memory occupied by the operators. |
| **MS_ASCEND_CHECK_OVERFLOW_MODE** | INFNAN_MODE | Sets the overflow detection mode. | `SATURATION_MODE`: saturation mode, saturates to floating-point extremes (+-MAX) when the calculation overflows;
`INFNAN_MODE`: INF/NAN mode, follows the IEEE 754 standard, and outputs INF/NAN calculations as defined. | In large model tuning, the overflow state is aligned with PyTorch and it is recommended to use INFNAN_MODE, i.e. export MS_ASCEND_CHECK_OVERFLOW_MODE=INFNAN_MODE.
Try setting this variable to INFNAN_MODE when encountering persistent overflow problems. |
| **MF_LOG_SUFFIX** | NA | Set custom suffixes for all log folders. | Suffix for the log folder. Default: no suffix | Adding a consistent suffix isolates logs across tasks from being overwritten. |
| **PLOG_REDIRECT_TO_OUTPUT** | False | Controls whether plog logs change storage paths. | `True`: store the logs in the ./output directory;
`False`: Store to the default storage location. | This setting makes it easier to query the plog log. |
| **MS_ENABLE_FA_FLATTEN** | on | Controls whether support FlashAttention flatten optimization. | `on`: Enable FlashAttention flatten optimization;
`off`: Disable FlashAttention flatten optimization. | Provide a fallback mechanism for models that have not yet been adapted to FlashAttention flatten optimization. |
| **EXPERIMENTAL_KERNEL_LAUNCH_GROUP** | NA | Control whether to support the batch parallel submission of operators. If supported, enable the parallel submission and configure the number of parallel submissions. | `thread_num`: The number of concurrent threads is not recommended to be increased. The default value is 2;
`kernel_group_num`: Total number of operator groups, 'kernel_group_num/thread_num' groups per thread, default is `8`. | This feature will continue to evolve in the future, and the subsequent behavior may change. Currently, only the `deepseek` reasoning scenario is supported, with certain performance optimization, but other models using this feature may deteriorate, and users need to use it with caution, as follows:`export EXPERIMENTAL_KERNEL_LAUNCH_GROUP="thread_num:2,kernel_group_num:8"`. |
| **ENFORCE_EAGER** | False | Control whether to disable jit mode. | `False`: Enable jit mode;
`True`: Do not enable jit mode. | Jit compiles functions into a callable MindSpore graph, sets ENFORCE_EAGER to False to enable jit mode, which can generate performance benefits. Currently, only inference mode is supported. |
| **MS_ENABLE_TFT** | NA | Enable Training Fault Tolerance function. Most functions rely on [MindIO TFT](https://www.hiascend.com/document/detail/zh/mindx-dl/600/clusterscheduling/ref/mindiottp/mindiotft001.html). | The value of the environment variable can be: "{TTP:1,UCE:1,HCCE:1,ARF:1,TRE:1,TSP:1,RSC:1}". When a certain feature is used, the corresponding field can be configured as "1". | Usage can refer to [High Availability](https://www.mindspore.cn/mindformers/docs/en/r1.7.0/feature/high_availability.html). |
| **MS_WORKER_NUM** | NA | Number of processes assigned the role MS_WORKER. | Integer greater than 0. | Distributed scenarios. |
| **RANK_ID** | NA | Specifies the logical ID for invoking the NPU. | 0–7. When multiple machines are parallelised, DEVICE_ID may duplicate across different servers. Using RANK_ID avoids this issue (in multi-machine parallelisation, RANK_ID = SERVER_ID * DEVICE_NUM + DEVICE_ID, where DEVICE_ID denotes the Ascend AI processor number on the current machine). | |
| **RANK_SIZE** | NA | Specifies the number of NPU units to invoke. | An integer greater than 1. | |
| **LD_PRELOAD** | NA | Specifies the shared library to preload. | Specifies the path to the shared library. | |
| **DEVICE_ID** | 0 | Specifies the device ID for invoking the NPU. | 0 to the number of NPUs on the server. | |
| **MS_SCHED_PORT** | NA | Specifies the port number for Scheduler binding. | Port number within the range 1024–65535. | |
| **NPU_ASD_ENABLE** | 0 | Whether to enable feature value detection. | `0`: Disable feature value detection
`1`: Logs detected anomalies without throwing exceptions
`2`: Logs anomalies and throws exceptions
`3`: Logs in both normal and anomalous scenarios (Note: Logging occurs only when CANN is set to INFO or DEBUG levels in normal scenarios). When anomalies are detected, the detection operator throws an exception. | |
| **MS_SDC_DETECT_ENABLE** | 0 | Enable/disable CheckSum detection for silent failures. | `0`: Disable CheckSum detection for silent failures.
`1`: Enable CheckSum detection for silent failures. | |
| **ASCEND_HOME_PATH** | NA | Installation path for the Ascend software package. | Set to the specified path. | |
| **ENABLE_LAZY_INLINE** | 1 | Whether to enable Lazy Inline mode. This environment variable will be deprecated and removed in the next version. | `0`: Disable Lazy Inline.
`1`: Enable Lazy Inline. | |
| **LOCAL_DEFAULT_PATH** | ./output | Sets the default path for logs. | Set to the specified path. | |
| **STDOUT_DEVICES** | NA | Sets the list of device IDs for standard output. | Set as a numeric list, with multiple IDs separated by commas. | |
| **REGISTER_PATH** | | Directory path containing the plug-in code to be registered. | Set to the specified path. | |
| **LOG_MF_PATH** | ./output/log | Log path for MindSpore Transformers. | Set to the specified path. | |
| **DEVICE_NUM_PER_NODE** | 8 | Number of NPUs on the server. | An integer greater than 0. | |
| **SHARED_PATHS** | | Paths for shared storage. | Set to the specified path. | |
| **ASCEND_PROCESS_LOG_PATH** | NA | Log path for the Ascend process. | Set to the specified path. | |
| **ENABLE_LAZY_INLINE_NO_PIPELINE** | 0 | Whether to enable Lazy Inline mode during non-pipelined parallelism. This environment variable will be deprecated and removed in the next version. | `0`: Lazy Inline disabled.
`1`: Lazy Inline enabled. | |
| **REMOTE_SAVE_URL** | None | URL used when saving training results on ModelArts. Currently deprecated and will be removed in future. | Enter the URL for saving results. | |