lite_boost

LiteBoost is an inference acceleration toolkit for Ascend hardware, built on top of MindSpore Lite. It provides high-performance custom operators, multi-card parallel inference, quantization and sparsity, and other inference acceleration capabilities.

Parallel

`lite_boost.parallel.initialize_usp`	Initialize the HCCL distributed environment for parallel inference.
`lite_boost.BoostManager`	Modify a supported model in-place for distributed parallel inference.

Layers

`lite_boost.layers.rms_norm`	Applies per-row RMS normalization over the last dim of x.
`lite_boost.layers.nearest_exact_upsample`	Upsamples x using nearest-exact interpolation.
`lite_boost.layers.rope_apply`	Applies rotary position embeddings (RoPE) to x.
`lite_boost.layers.flash_attention`	Computes flash attention with automatic backend fallback.

Operators

`lite_boost.ops.rain_fusion_attention`	Block-sparse fusion attention forward computation.
`lite_boost.ops.sparse_attention`	High-level sparse attention entry point.
`lite_boost.ops.recurrent_gated_delta_rule`	Recurrent GatedDeltaRule operator — CANN aclnn-backed recurrent linear attention decode.
`lite_boost.ops.chunk_gated_delta_rule`	Chunked (prefill) Gated Delta Rule operator on NPU (ascend_a2 op).