​https://github.com/PSAL-POSTECH/ONNXim

​

ONNXim is a fast cycle-level simulator that can model multi-core NPUs for DNN inference. Its features include the following:

• Faster simulation speed in comparison to other detailed NPU simulation frameworks (see the figure below).

• Support for modeling multi-core NPUs.

• Support for cycle-level simulation of memory (through Ramulator) and network-on-chip (through Booksim2), which is important for properly modeling memory-bound operations in deep learning.

• Use of ONNX graphs as DNN model specifications, enabling simulation of DNNs implemented in different deep learning frameworks (e.g., PyTorch and TensorFlow).

• Support language models that do not use ONNX graphs. Additionally, enable auto-regressive generation phases and iteration-level batching.

For more details, please refer to our paper below:

Hyungkyu Ham, Wonhyuk Yang, Yunseon Shin, Okkyun Woo, Guseul Heo, Sangyeop Lee, Jongse Park, Gwangsun Kim, "ONNXim: A Fast, Cycle-level Multi-core NPU Simulator," arXiv:2406.08051