3 <img src="Arm_NN_horizontal_blue.png" class="center" alt="Arm NN Logo" width="300"/>
6 * [Quick Start Guides](#quick-start-guides)
7 * [Pre-Built Binaries](#pre-built-binaries)
8 * [Software Overview](#software-overview)
9 * [Get Involved](#get-involved)
10 * [Contributions](#contributions)
11 * [Disclaimer](#disclaimer)
13 * [Third-Party](#third-party)
14 * [Build Flags](#build-flags)
16 ## Announcement: As part of Arm's commitment to the use of inclusive language we will be moving away from 'master' branch to 'main' soon.
17 ## From 15 August 2022 our 'master' branch will be frozen and we will be using 'main' branch instead.
21 **_Arm NN_** is the **most performant** machine learning (ML) inference engine for Android and Linux, accelerating ML
22 on **Arm Cortex-A CPUs and Arm Mali GPUs**. This ML inference engine is an open source SDK which bridges the gap
23 between existing neural network frameworks and power-efficient Arm IP.
25 Arm NN outperforms generic ML libraries due to **Arm architecture-specific optimizations** (e.g. SVE2) by utilizing
26 **[Arm Compute Library (ACL)](https://github.com/ARM-software/ComputeLibrary/)**. To target Arm Ethos-N NPUs, Arm NN
27 utilizes the [Ethos-N NPU Driver](https://github.com/ARM-software/ethos-n-driver-stack). For Arm Cortex-M acceleration,
28 please see [CMSIS-NN](https://github.com/ARM-software/CMSIS_5).
30 Arm NN is written using portable **C++14** and built using [CMake](https://cmake.org/) - enabling builds for a wide
31 variety of target platforms, from a wide variety of host environments. **Python** developers can interface with Arm NN
32 through the use of our **Arm NN TF Lite Delegate**.
36 **The Arm NN TF Lite Delegate provides the widest ML operator support in Arm NN** and is an easy way to accelerate
37 your ML model. To start using the TF Lite Delegate, first download the **[Pre-Built Binaries](#pre-built-binaries)** for
38 the latest release of Arm NN. Using a Python interpreter, you can load your TF Lite model into the Arm NN TF Lite
39 Delegate and run accelerated inference. Please see this
40 **[Quick Start Guide](delegate/DelegateQuickStartGuide.md)** on GitHub or this more comprehensive
41 **[Arm Developer Guide](https://developer.arm.com/documentation/102561/latest/)** for information on how to accelerate
42 your TF Lite model using the Arm NN TF Lite Delegate.
44 The fastest way to integrate Arm NN into an **Android app** is by using our **Arm NN AAR (Android Archive) file with
45 Android Studio**. The AAR file nicely packages up the Arm NN TF Lite Delegate, Arm NN itself and ACL; ready to be
46 integrated into your Android ML application. Using the AAR allows you to benefit from the **vast operator support** of
47 the Arm NN TF Lite Delegate. We held an **[Arm AI Tech Talk](https://www.youtube.com/watch?v=Zu4v0nqq2FA)** on how to
48 accelerate an ML Image Segmentation app in 5 minutes using this AAR file, with the supporting guide
49 **[here](https://developer.arm.com/documentation/102744/latest)**. To download the Arm NN AAR file, please see the
50 **[Pre-Built Binaries](#pre-built-binaries)** section below.
52 We also provide Debian packages for Arm NN, which are a quick way to start using Arm NN and the TF Lite Parser
53 (albeit with less ML operator support than the TF Lite Delegate). There is an installation guide available
54 [here](InstallationViaAptRepository.md) which provides instructions on how to install the Arm NN Core and the TF Lite
55 Parser for Ubuntu 20.04.
57 To build Arm NN from scratch, we provide the **[Arm NN Build Tool](build-tool/README.md)**. This tool consists of
58 **parameterized bash scripts** accompanied by a **Dockerfile** for building Arm NN and its dependencies, including
59 **[Arm Compute Library (ACL)](https://github.com/ARM-software/ComputeLibrary/)**. This tool replaces/supersedes the
60 majority of the existing Arm NN build guides as a user-friendly way to build Arm NN. The main benefit of building
61 Arm NN from scratch is the ability to **exactly choose which components to build, targeted for your ML project**.<br>
66 | Operating System | Architecture-specific Release Archive (Download) |
67 |-----------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
68 | Android (AAR) | [](https://github.com/ARM-software/armnn/releases/download/v22.08/ArmnnDelegate-release.aar) |
69 | Android 8.1 "O/Oreo" (API level 27) | [](https://github.com/ARM-software/armnn/releases/download/v22.08/ArmNN-android-27-arm64-v8.2-a.tar.gz) [](https://github.com/ARM-software/armnn/releases/download/v22.08/ArmNN-android-27-arm64-v8a.tar.gz) |
70 | Android 9 "P/Pie" (API level 28) | [](https://github.com/ARM-software/armnn/releases/download/v22.08/ArmNN-android-28-arm64-v8.2-a.tar.gz) [](https://github.com/ARM-software/armnn/releases/download/v22.08/ArmNN-android-28-arm64-v8a.tar.gz) |
71 | Android 10 "Q/Quince Tart" (API level 29) | [](https://github.com/ARM-software/armnn/releases/download/v22.08/ArmNN-android-29-arm64-v8.2-a.tar.gz) [](https://github.com/ARM-software/armnn/releases/download/v22.08/ArmNN-android-29-arm64-v8a.tar.gz) |
72 | Linux | [](https://github.com/ARM-software/armnn/releases/download/v22.08/ArmNN-linux-aarch64.tar.gz) [](https://github.com/ARM-software/armnn/releases/download/v22.08/ArmNN-linux-x86_64.tar.gz) |
76 The Arm NN SDK supports ML models in **TensorFlow Lite** (TF Lite) and **ONNX** formats.
78 **Arm NN's TF Lite Delegate** accelerates TF Lite models through **Python or C++ APIs**. Supported TF Lite operators
79 are accelerated by Arm NN and any unsupported operators are delegated (fallback) to the reference TF Lite runtime -
80 ensuring extensive ML operator support. **The recommended way to use Arm NN is to
81 [convert your model to TF Lite format](https://www.tensorflow.org/lite/convert) and use the TF Lite Delegate.** Please
82 refer to the [Quick Start Guides](#quick-start-guides) for more information on how to use the TF Lite Delegate.
84 Arm NN also provides **TF Lite and ONNX parsers** which are C++ libraries for integrating TF Lite or ONNX models
85 into your ML application. Please note that these parsers do not provide extensive ML operator coverage as compared
86 to the Arm NN TF Lite Delegate.
88 **Android** ML application developers have a number of options for using Arm NN:
89 * Use our Arm NN AAR (Android Archive) file with **Android Studio** as described in the
90 [Quick Start Guides](#quick-start-guides) section
91 * Download and use our [Pre-Built Binaries](#pre-built-binaries) for the Android platform
92 * Build Arm NN from scratch with the Android NDK using this [GitHub guide](BuildGuideAndroidNDK.md)
94 Arm also provides an [Android-NN-Driver](https://github.com/ARM-software/android-nn-driver) which implements a
95 hardware abstraction layer (HAL) for the Android NNAPI. When the Android NN Driver is integrated on an Android device,
96 ML models used in Android applications will automatically be accelerated by Arm NN.
98 For more information about the Arm NN components, please refer to our
99 [documentation](https://github.com/ARM-software/armnn/wiki/Documentation).
101 Arm NN is a key component of the [machine learning platform](https://mlplatform.org/), which is part of the
102 [Linaro Machine Intelligence Initiative](https://www.linaro.org/news/linaro-announces-launch-of-machine-intelligence-initiative/).
104 For FAQs and troubleshooting advice, see the [FAQ](docs/FAQ.md) or take a look at previous
105 [GitHub Issues](https://github.com/ARM-software/armnn/issues).
109 The best way to get involved is by using our software. If you need help or encounter an issue, please raise it as a
110 [GitHub Issue](https://github.com/ARM-software/armnn/issues). Feel free to have a look at any of our open issues too.
111 We also welcome feedback on our documentation.
113 Feature requests without a volunteer to implement them are closed, but have the 'Help wanted' label, these can be
114 found [here](https://github.com/ARM-software/armnn/issues?q=is%3Aissue+label%3A%22Help+wanted%22+).
115 Once you find a suitable Issue, feel free to re-open it and add a comment, so that Arm NN engineers know you are
116 working on it and can help.
118 When the feature is implemented the 'Help wanted' label will be removed.
122 The Arm NN project welcomes contributions. For more details on contributing to Arm NN please see the
123 [Contributing page](https://mlplatform.org/contributing/) on the [MLPlatform.org](https://mlplatform.org/) website,
124 or see the [Contributor Guide](CONTRIBUTING.md).
126 Particularly if you'd like to implement your own backend next to our CPU, GPU and NPU backends there are guides for
127 backend development: [Backend development guide](src/backends/README.md),
128 [Dynamic backend development guide](src/dynamic/README.md).
132 The armnn/tests directory contains tests used during Arm NN development. Many of them depend on third-party IP, model
133 protobufs and image files not distributed with Arm NN. The dependencies for some tests are available freely on
134 the Internet, for those who wish to experiment, but they won't run out of the box.
138 Arm NN is provided under the [MIT](https://spdx.org/licenses/MIT.html) license.
139 See [LICENSE](LICENSE) for more information. Contributions to this project are accepted under the same license.
141 Individual files contain the following tag instead of the full license text.
143 SPDX-License-Identifier: MIT
145 This enables machine processing of license information based on the SPDX License Identifiers that are available
146 here: http://spdx.org/licenses/
150 Third party tools used by Arm NN:
152 | Tool | License (SPDX ID) | Description | Version | Provenience
153 |----------------|-------------------|------------------------------------------------------------------|-------------|-------------------
154 | cxxopts | MIT | A lightweight C++ option parser library | SHA 12e496da3d486b87fa9df43edea65232ed852510 | https://github.com/jarro2783/cxxopts
155 | doctest | MIT | Header-only C++ testing framework | 2.4.6 | https://github.com/onqtam/doctest
156 | fmt | MIT | {fmt} is an open-source formatting library providing a fast and safe alternative to C stdio and C++ iostreams. | 7.0.1 | https://github.com/fmtlib/fmt
157 | ghc | MIT | A header-only single-file std::filesystem compatible helper library | 1.3.2 | https://github.com/gulrak/filesystem
158 | half | MIT | IEEE 754 conformant 16-bit half-precision floating point library | 1.12.0 | http://half.sourceforge.net
159 | mapbox/variant | BSD | A header-only alternative to 'boost::variant' | 1.1.3 | https://github.com/mapbox/variant
160 | stb | MIT | Image loader, resize and writer | 2.16 | https://github.com/nothings/stb
164 Arm NN uses the following security related build flags in their code:
167 |---------------------|
171 | -Wno-missing-braces |
173 | -Wsign-conversion |
1.8.13 