2.4.3. Neutron v6 Architecture#
Neutron v6 iteratively optimizes the power consumption and area of the Neutron v4 architecture. It retains an overall organization of 1 NPU Core, 1 CONV, 1 TENG, 1 SDMA, and optional dual-vNPU time-division multiplexing.
Like Neutron v4, Neutron v6 divides EUs into compute and data-movement types. With time-division multiplexing enabled, two logical vNPUs can overlap the computation and communication stages of different inference tasks. The two logical vNPUs share on-chip resources, so each vNPU has less OCM available than the complete NPU resource with multiplexing disabled.
Some Neutron v6 platforms expose only the single-NPU resource mode. See the runtime modes and
npu_modedescriptions below for the available options.
2.4.3.1. Runtime modes and npu_mode#
2.4.3.1.1. AXEngine runtime modes#
Chip platform |
AXEngine runtime mode |
Compute resources |
|---|---|---|
|
|
Enables logical vNPU partitioning and schedules two vNPUs on the same NPU Core through hardware time-division multiplexing. The two logical vNPUs share on-chip resources, reducing the OCM available to each vNPU. |
|
|
Disables logical vNPU partitioning and hardware time-division multiplexing. One complete NPU resource is exposed at runtime, and the model uses all NPU resources for inference. |
|
|
Only the single-NPU resource mode is supported. |
|
|
Only the single-NPU resource mode is supported. |
2.4.3.1.2. Offline npu_mode and hardware resources#
Chip platform |
|
Hardware compute resources |
|---|---|---|
|
|
Compilation resources for one logical vNPU in time-division multiplexing mode, with less OCM available than in complete NPU resource mode. |
|
|
Complete NPU compilation resources with time-division multiplexing disabled. All NPU resources are used for inference, with more OCM available than to one logical vNPU in multiplexing mode. |
|
|
One NPU Core containing 1 |
|
|
One NPU Core containing 1 |