# Introduction to AXera NPU (Neutron)

(soc_introduction)=

AXera NPU (Neutron) is the neural-network compute engine used in AXera chip platforms. Neutron versions differ in system architecture, the number of compute cores, virtual NPU (vNPU) partitioning, the offline compilation `npu_mode`, and AXEngine runtime modes.

A vNPU is a logical partition of the underlying NPU compute resources. During model compilation, use the `--npu_mode` option of `pulsar2 build` to specify the amount of compute resources required by the model. When the model is loaded on the chip, AXEngine schedules it to matching NPU Core or vNPU resources according to the current NPU runtime mode.

This section first introduces the basic Neutron terminology and the responsibilities of each EU type. The subsections then describe how NPU compute resources are organized in each Neutron version. Finally, the chip platform table links each platform and its available `npu_mode` values to the corresponding hardware resources and AXEngine runtime modes.

## Hardware terminology

`NPU Core`

: The basic NPU execution unit. After compilation, a model is divided into executable tasks that are mapped to one or more NPU Cores. Each NPU Core generally contains EUs, Local OCM, and synchronization management capabilities.

`EU`

: Execution Unit, a hardware unit that performs computation or data movement. Different EUs handle different types of work, such as convolution, tensor or vector computation, data movement, and data rearrangement.

`Local OCM`

: On-Chip Memory within an NPU Core. Compared with DDR, OCM has lower access latency and higher bandwidth. It is generally used for input tiles, weight tiles, intermediate results, and temporary data that the current task accesses frequently.

`Shared OCM`

: On-chip memory shared by multiple NPU Cores or vNPUs. It is used to exchange intermediate data across tasks and reduce the additional transfers and memory accesses caused by staging data through DDR.

`SyncManager`

: A synchronization management module that describes and manages execution dependencies among NPU Cores, EUs, and subtasks, ensuring the correct order of data reads and writes during parallel execution.

`npu_mode`

: An offline compilation option of `pulsar2 build` that specifies the target amount of NPU resources. Values such as `NPU1`, `NPU2`, `NPU3`, and `NPU4` represent compilation targets of different sizes; they do not identify a particular NPU Core or runtime vNPU.

`AXEngine runtime mode`

: The hardware resource partitioning mode configured through `AX_ENGINE_NPU_ATTR_T.eHardMode` when `AX_ENGINE_Init` initializes AXEngine. It determines the vNPU or NPU resource layout exposed to models at runtime and must satisfy the `npu_mode` resource requirements of the model being loaded.

## EU types

EUs are the hardware resources that perform work in Neutron. They can be divided into compute EUs and data-movement EUs. Based on operator types, tensor shapes, data layouts, OCM pressure, and target hardware capabilities, the compiler automatically maps model tasks to suitable EUs. Users normally do not need to select EU types manually.

`CONV`

: A convolution compute EU for highly reusable, compute-intensive deep-learning workloads. Typical tasks include convolution, deconvolution, convolution post-processing, and some matrix computations that can be mapped to a convolution dataflow.

`TENG`

: Tensor Engine, a tensor or vector compute EU suitable for multi-channel parallel data processing and general tensor computation. Typical tasks include element-wise operations, activations, lookups, reductions, coordinate or address calculation, and some data replication and format-assistance operations.

`SDMA`

: A data-movement EU responsible for transferring data among DDR, OCM, and on-chip memory regions. Typical tasks include loading model inputs and weights, reading and writing intermediate results, writing outputs, channel cropping or padding, and data-format encoding or decoding.

`DAU`

: A compute EU used for some data rearrangement, filtering, and sorting tasks. Typical tasks include Padding, Transpose, TopK/Sort, and other data organization operations.

:::{note}
The number of EUs, supported sub-functions, and available runtime modes may differ among Neutron versions and chip platforms. Refer to the chip specification and toolchain support for the actual capabilities.
:::

## Offline compilation and runtime modes

`npu_mode` and the AXEngine runtime mode are settings at two different stages and must be used together:

- `npu_mode` takes effect during offline compilation. It determines the resource scale, task partitioning, and scheduling constraints generated by the compiler. The compiled `axmodel` carries these resource requirements.
- The AXEngine runtime mode takes effect when the application initializes the NPU. It determines whether the underlying hardware is exposed as one complete NPU resource, multiple physical or logical vNPUs, or scheduling resources across multiple NPU Cores.
- A runtime mode cannot automatically turn a model compiled for fewer resources into a larger-resource model, nor can a model compiled for more resources run in a runtime mode that provides insufficient resources. During deployment, determine the model's `npu_mode` first, and then select an AXEngine mode that provides matching resources.

## Neutron architectures

See the following subsections for the resource organization of each Neutron version:

```{toctree}
:maxdepth: 1

v3/introduction
v4/introduction
v6/introduction
v7/introduction
```

(neutron_platform_npu_mode)=

### Chip platforms and available npu_mode values

The following table lists the Neutron version and available `npu_mode` options for each chip platform. `npu_mode` describes the amount of NPU resources used to compile a model; it does not identify a particular NPU Core or vNPU.

For the mapping among AXEngine runtime modes, offline `npu_mode` values, and hardware compute resources, see the corresponding Neutron architecture document.

```{eval-rst}
.. list-table::
    :header-rows: 1
    :align: center
    :widths: 18 18 24 40

    * - Chip platform
      - Neutron version
      - Available ``npu_mode``
      - Details
    * - ``AX650``
      - :ref:`v3 <neutron_v3_architecture>`
      - ``NPU1`` / ``NPU2`` / ``NPU3``
      - :ref:`Neutron v3 runtime modes and npu_mode <neutron_v3_platform_npu_mode>`
    * - ``M76H``
      - :ref:`v3 <neutron_v3_architecture>`
      - ``NPU1`` / ``NPU2`` / ``NPU3``
      - :ref:`Neutron v3 runtime modes and npu_mode <neutron_v3_platform_npu_mode>`
    * - ``AX620E``
      - :ref:`v4 <neutron_v4_architecture>`
      - ``NPU1`` / ``NPU2``
      - :ref:`Neutron v4 runtime modes and npu_mode <neutron_v4_platform_npu_mode>`
    * - ``AX630C``
      - :ref:`v4 <neutron_v4_architecture>`
      - ``NPU1`` / ``NPU2``
      - :ref:`Neutron v4 runtime modes and npu_mode <neutron_v4_platform_npu_mode>`
    * - ``AX620Q``
      - :ref:`v4 <neutron_v4_architecture>`
      - ``NPU1`` / ``NPU2``
      - :ref:`Neutron v4 runtime modes and npu_mode <neutron_v4_platform_npu_mode>`
    * - ``AX615``
      - :ref:`v6 <neutron_v6_architecture>`
      - ``NPU1`` / ``NPU2``
      - :ref:`Neutron v6 runtime modes and npu_mode <neutron_v6_platform_npu_mode>`
    * - ``M57``
      - :ref:`v6 <neutron_v6_architecture>`
      - ``NPU1``
      - :ref:`Neutron v6 runtime modes and npu_mode <neutron_v6_platform_npu_mode>`
    * - ``AX637``
      - :ref:`v6 <neutron_v6_architecture>`
      - ``NPU1``
      - :ref:`Neutron v6 runtime modes and npu_mode <neutron_v6_platform_npu_mode>`
    * - ``AX8860``
      - :ref:`v7 <neutron_v7_architecture>`
      - ``NPU1`` / ``NPU2`` / ``NPU4``
      - :ref:`Neutron v7 runtime modes and npu_mode <neutron_v7_platform_npu_mode>`
```