MindSpore Insight Commands

Viewing the Command Help Information

mindinsight --help

Viewing the Version Information

mindinsight --version

Starting the Service

MindSpore Insight service only supports local access by default. For remote access, please modify the configuration file mindinsight\conf\constants.py. Modify the HOST in the file to the server IP, and make sure that the startup port has been opened or the firewall has been closed.

mindinsight start [-h] [--workspace <WORKSPACE>] [--port <PORT>]
                  [--url-path-prefix <URL_PATH_PREFIX>]
                  [--reload-interval <RELOAD_INTERVAL>]
                  [--summary-base-dir <SUMMARY_BASE_DIR>]
                  [--enable-debugger <ENABLE_DEBUGGER>]
                  [--debugger-port <DEBUGGER_PORT>]
                  [--offline-debugger-mem-limit <OFFLINE_DEBUGGER_MEMORY_LIMIT>]
                  [--max-offline-debugger-session-num <MAX_OFFLINE_DEBUGGER_SESSION_NUMBER>]

Optional parameters are as follows:

Name	Argument	Description	Type	Default	Scope	Specifications
`-h, --help`	Optional	Displays the help information about the start command.	-	-	-	-
`--workspace <WORKSPACE>`	Optional	Specifies the path for storing MindSpore Insight logs.	String	$HOME/mindinsight	-	-
`--port <PORT>`	Optional	Specifies the port number of the web visualization service.	Integer	8080	1~65535	-
`--url-path-prefix <URL_PATH_PREFIX>`	Optional	Specifies the URL path prefix of the web visualization service.	String	Empty string	-	URL path prefix consists of segments separated by slashes. Each segment supports alphabets / digits / underscores / dashes / dots, but not single dot or double dots.
`--reload-interval <RELOAD_INTERVAL>`	Optional	Specifies the interval (unit: second) for loading data.	Integer	3	0~300	The value 0 indicates that data is loaded only once.
`--summary-base-dir <SUMMARY_BASE_DIR>`	Optional	Specifies the root directory for loading training log data.	String	./	-	MindSpore Insight traverses the direct subdirectories in this directory and searches for log files. If a direct subdirectory contains log files, it is identified as the log file directory. If a root directory contains log files, it is identified as the log file directory. In the ModelArts development environment, this parameter can be specified as an OBS path. Please refer to ModelArts documentation for more information.
`--enable-debugger <ENABLE_DEBUGGER>`	Optional	Whether to launch the MindSpore Insight Debugger.	Boolean	False	True/False/1/0	The debugger entry can be shown on MindSpore Insight UI only when MindSpore Insight Debugger is launched.
`--debugger-port <DEBUGGER_PORT>`	Optional	Specifies the port number of the debugger server.	Integer	50051	1~65535	-
`--offline-debugger-mem-limit <OFFLINE_DEBUGGER_MEMORY_LIMIT>`	Optional	Specifies the maximum memory limit of a single offline debugger session. When the offline debugger cannot be executed due to insufficient memory, set it according to the device memory.	Integer	16*1024	6*1024~The upper limit of int32	-
`--max-offline-debugger-session-num <MAX_OFFLINE_DEBUGGER_SESSION_NUMBER>`	Optional	Specifies the maximum session number of the offline debugger. The session number refers to the amount of training jobs that can be debugged at the same time.	Integer	2	1~2	-
`--max-graph-node-size <MAX_GRAPH_NODE_SIZE >`	Optional	Set the maximum number of graph nodes loaded by the debugger.	Integer	100000	1~2000000	-

--workspace log directory description：

Module name	Log directory description	Log format
datavisual	Training kanban module, it records all the logs of training Kanban module.	`datavisual.<PORT>.log`
debugger	Debugger module, it records all logs of the debugger module.	`debugger.<PORT>.log`
explainer	Explain the AI module, it records all logs that explain the data parsed by the AI module.	`explainer.<PORT>.log`
gunicorn	Web service module, it records all logs of the Web service module.	`access.<PORT>.log` `error.<PORT>.log`
lineage	Traceability module, it records all logs of the traceability module.	`lineage.<PORT>.log`
notebook	Record all logs using MindSpore Insight in the ModelArts notebook.	`notebook.<PORT>.log`
optimizer	Optimizer module, it records all optimizer module logs.	`optimizer.<PORT>.log`
parse_summary	Summary file parsing module, it records all logs when using the summary file parsing module.	`parse_summary.<PORT>.log`
profiler	Performance analysis module, it records all logs of the performance analysis module.	`profiler.<PORT>.log`
restful_api	RESTFul API module, it records all RESTFul API interaction logs.	`restful_api.<PORT>.log`
scripts	Start and stop the MindSpore Insight module, it records all MindSpore Insight starts and stops.	`start.<PORT>.log` `stop.<PORT>.log`
utils	Public module, it records all logs of the public module.	`utils.<PORT>.log`

Note: There is one log file for each module, but when a log file exceeds 50M, it will be renamed and archived in the format of <module name>_<PORT>.log.<id> ,module name indicates the module name, PORT indicates the PORT number, and ID indicates the number of file renaming and archiving times.

When the service is started, the parameter values of the command line are saved as the environment variables of the process and start with MINDINSIGHT_, for example, MINDINSIGHT_PORT, MINDINSIGHT_WORKSPACE, etc.

Execute command:

mindinsight start --port 8000 --workspace /path/to/workspace/dir --summary-base-dir /path/to/summary/base/dir

The startup is successful if it prompts as follows:

Web address: http://127.0.0.1:8000
service start state: success

Viewing the Service Process Information

MindSpore Insight provides user with web services. Run the following command to view the running web service process:

ps -ef | grep mindinsight

Run the following command to access the working directory WORKSPACE corresponding to the service process based on the service process ID:

lsof -p <PID> | grep access

Output the working directory WORKSPACE as follows:

gunicorn  <PID>  <USER>  <FD>  <TYPE>  <DEVICE>  <SIZE/OFF>  <NODE>  <WORKSPACE>/log/gunicorn/access.log

Stopping the Service

mindinsight stop [-h] [--port PORT]

Optional parameters are as follows:

Name	Argument	Description	Type	Default	Scope	Specifications
`-h, --help`	Optional	Displays the help information about the stop command.	-	-	-	-
`--port <PORT>`	Optional	Specifies the port number of the web visualization service.	Integer	8080	1~65535	-

Execute command:

mindinsight stop --port 8000

The shutdown is successful if it prompts as follows:

Stop mindinsight service successfully

Parsing Summary

MindSpore Insight provides tools for parsing summary log files. Users can save the scalars in the summary log file into a csv file and the images into a png file through the commands, which is convenient for viewing and further processing.

mindinsight parse_summary [--summary-dir] [--output]

Optional parameters are as follows:

Name	Argument	Description	Type	Default	Scope	Specifications
`--summary-dir`	Optional	Specifies the root directory of summary files. If the directory contains multiple summary files, only the latest summary file is parsed.	String	./	-	The summary file directory needs to be readable and executable, and the summary file needs to be readable.
`--output`	Optional	Specifies the root directory for saving output files.	String	./	-	-

Execute command:

mindinsight parse_summary --summary-dir ./ --output ./

The output directory structure is as follows:

└─output_{datetime}
    ├─image
    │   └─{tag}_{step}.png
    │
    └─scalar.csv

In which,

output_{datetime} is the output directory. The rule is ‘output_yyyyMMdd_HHmmss_SSSSSS’ including year, month, day, hour, minute, second and microseconds.
{tag}_{step}.png is the image in training process. ‘tag’ and ‘step’ are the tag and step in the training (special characters in tag are deleted and ‘/’ is replaced by ‘_’).
scalar.csv is the file which save scalars (encoding: ‘utf-8’).

Using Mindoptimizer to Tune Hyperparameters

MindSpore Insight provides parameters tuning command. The command-line interface (CLI) provides the following commands:

usage: mindoptimizer [-h] [--version] [--config <CONFIG>]
                     [--iter <ITER>]

Optional parameters are as follows:

Name	Argument	Description	Type	Default	Scope	Specifications
`-h, --help`	Optional	Displays the help information about the start command.	-	-	-	-
`--config <CONFIG>`	Required	Specifies the configuration file.	String	-	-	Physical file path (file:/path/to/config.yaml), and the file format is yaml.
`--iter <ITER>`	Optional	Specifies the run times for tuning parameters	Integer	1	Positive integer	-