# Arm NN Build Tool The **_Arm NN Build Tool_** provides a collection of **parameterized bash scripts** accompanied by a **Dockerfile** for building Arm NN and its dependencies, including **[Arm Compute Library (ACL)](https://github.com/ARM-software/ComputeLibrary/)**. This tool replaces/supersedes the majority of the existing Arm NN build guides as a user-friendly way to build Arm NN from scratch. The main benefit of building Arm NN from scratch is the ability to **exactly choose which components to build, targeted for your ML project**.
The Arm NN Build Tool is tested on **x86_64 (Intel) and aarch64 (Arm) build hosts** for the **Ubuntu** platform and on **arm64 (Apple Silicon)**. At present, the tool supports **targeting Linux devices (Ubuntu 20.04) and macOS (Mac with Apple silicon). In Linux it supports building on x86_64, aarch64 and android64** architectures.
We recommend using the Arm NN Build Tool through the use of Docker. However, the scripts may be [executed directly on your machine](#build-arm-nn-without-docker) if desired. **If you already have Docker installed** and want to quickly build the Arm NN Dockerfile with some default build arguments, please follow the [Docker Build Steps](#docker-build-steps). * [Overview](#overview) * [System Requirements](#system-requirements) * [Install Docker](#install-docker) * [Ubuntu / Linux](#ubuntu--linux) * [Mac](#mac) * [Windows](#windows) * [Docker Build Arguments Reference](#docker-build-arguments-reference) * [SETUP_ARGS](#setup_args) * [BUILD_ARGS](#build_args) * [Docker Build Steps](#docker-build-steps) * [Build Arm NN Without Docker](#build-arm-nn-without-docker) * [Advanced Usage](#advanced-usage) * [Use custom Arm NN and ACL repository versions during build](#use-custom-arm-nn-and-acl-repository-versions-during-build) * [Additional Docker Build Arguments](#additional-docker-build-arguments) * [Inspect the contents of the Docker Image](#inspect-the-contents-of-the-docker-image) * [Bind Mounts](#bind-mounts) ## Overview The Arm NN Build Tool consists of an official Arm NN **Dockerfile** with associated **scripts** for building Arm NN: 1. **```install-packages.sh```:** builds CMake from source and installs system-wide **apt packages** 2. **```setup-armnn.sh```:** downloads and builds Arm NN **dependencies** (e.g. Flatbuffers, TF Lite, Protobuf) 3. **```build-armnn.sh```:** builds Arm NN and ACL 4. **```common.sh```:** common variables and functions used by ```setup-armnn.sh``` and ```build-armnn.sh``` 5. **```validation.sh```:** validation of common command-line arguments to ```setup-armnn.sh``` and ```build-armnn.sh``` 6. **```docker-copy-to-host```:** helpful script to copy contents of a Docker Image to the host machine A **Dockerfile** is a text document which outlines the instructions for building a **Docker Image**. Running ```docker build``` on the Arm NN Dockerfile will result in a Docker Image that contains: 1. The Ubuntu operating system 2. Installed system-wide packages required by Arm NN 3. Source code for Arm NN and its dependencies 4. The build for Arm NN and its dependencies All scripts except ```docker-copy-to-host.sh``` are used during the ```docker build``` process. The ```docker-copy-to-host.sh``` script creates a temporary [**Docker Container**](https://www.docker.com/resources/what-container/) which is required to retrieve files from the Docker Image. A Docker Container is a **runtime** version of a Docker Image. It can be interacted with through a command-line terminal, like a lightweight virtual machine (VM). The ```docker-copy-to-host.sh``` script is designed to run on the host machine after ```docker build``` to copy the Arm NN build to the host. The **diagram below** outlines the operation of the Arm NN **Dockerfile** and the contents of the resulting **Docker Image** after running ```docker build```. During ```docker build```, the provided **```SETUP_ARGS```** determine which of Arm NN's dependencies to download and build. The **```BUILD_ARGS```** Docker argument determines which components of Arm NN to build. The resulting Docker Image contains the Arm NN build (along with everything required to build it) in the home directory ```/home/arm-user```. There are two directories: **```source```** which contains the source code of Arm NN and its dependencies, and **```build```** which contains the Arm NN build and the builds of each dependency of Arm NN. The **```armnn_aarch64_build.tar.gz``` tarball** in the home directory is a compressed archive of the Arm NN build which is located at ```/home/arm-user/build/armnn```. The home directory also contains the **scripts** mentioned above (not pictured in the diagram), which are copied into the Docker Image during the ```docker build``` process.
Arm NN Dockerfile

**Benefits of using Docker:** 1. Run one ```docker build``` command instead of executing multiple scripts, saving developer time 2. Can easily manage multiple different build configurations at once, each in their own Docker Image 3. System packages are installed in the Docker Image only, isolated from the host machine 4. Build in the exact environment the Arm NN Team used during testing, ensuring consistency in builds 5. Allows Arm NN to be built on a wide range of hosts, as long as they support Docker **Disadvantages of using Docker:** 1. Requires the developer to install Docker on their host machine 2. Requires the developer to get familiar with Docker (this guide should help) 3. Possible incompatibilities/limitations if the host machine's Linux Kernel version is too different than that of the base OS in the Docker Image. This is due to the fact that Docker Containers share the Linux Kernel of the host machine (unlike VM's which each have their own Kernel).
## System Requirements This guide will not provide the system requirements for installing Docker - please refer to the Docker documentation during installation. The built Docker Image has disk requirements of **less than 6GB**, depending on the Arm NN components chosen and whether the build included a cross compile (i.e. building aarch64 on x86_64). Repeated builds may cause the total disk usage to increase beyond 6GB. The ```docker images``` command shows the disk usage of each Docker Image. To view total disk space used by all Docker Images and Docker Containers, use the command ```docker system df```. This Dockerfile was built on a **Ubuntu 20.04 host machine with Docker version 20.10.14**. Other Linux distros such as **Debian** should also work as long as the Docker version is similar to **20.10.14**. This should also work on Mac with Apple silicon but needs some manual setup to be done. Please find the steps in ```install-packages.sh```. The build outputs from this Dockerfile have been tested on an **Odroid N2+ target device with Ubuntu 20.04** installed. The build outputs should also work on newer versions of Ubuntu (and Debian) but this has not been exhaustively tested. To view the system packages that are installed during ```docker build```, please refer to the ```install-packages.sh``` script.
## Install Docker Builds of the official Arm NN Dockerfile have been tested on the **Ubuntu 20.04** and Mac with Apple silicon host platforms. Whilst other platforms have not been tested, Docker should be able to build the Arm NN Dockerfile on other platforms such as Windows. On **Linux**, we recommend using **Docker Engine** which is used through the command-line interface (CLI). For **Windows and Mac** users, **Docker Desktop** is available which allows the use of the CLI plus a graphical interface.
This guide will **only** provide steps on using Docker through the use of the CLI and not Docker Desktop. ### Ubuntu / Linux For Ubuntu users, we recommend installing **Docker Engine** with the following steps from the Docker documentation:
1. [Uninstall any old versions of Docker](https://docs.docker.com/engine/install/ubuntu/#uninstall-old-versions) 2. [Install using the repository](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) 3. [Add user to Docker Unix Group](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user) (**recommended**: allows running of Docker commands **without ```sudo```**) Similar steps for installing Docker Engine on other Linux distributions can be found by clicking on your distro under the Server table on [this page](https://docs.docker.com/engine/install/#server). ### Mac The Dockerfile will also work on Mac with Apple silicon. But we haven't tested this using Docker Desktop. For **Mac** users, please use the Docker Desktop download link on [this page](https://docs.docker.com/desktop/install/mac-install/). ### Windows We do not officially test the building of the Arm NN Dockerfile on Windows. However, it's likely that it will work if your system supports Docker. For **Windows**, use the Docker Desktop download link on [this page](https://docs.docker.com/desktop/install/windows-install/). **Note:** we unfortunately cannot provide support for the installation of Docker and/or the use of the Arm NN Dockerfile on Windows.
## Docker Build Arguments Reference The following describes the ```docker build``` arguments that can be used to customize the Arm NN build. Docker build arguments such as ```SETUP_ARGS``` and ```BUILD_ARGS``` are provided in the ```docker build``` as a ```--build-arg```. ### SETUP_ARGS These arguments are eventually passed to ```setup-armnn.sh``` which downloads and builds Arm NN dependencies. For ease of use (but longer initial docker build), use ```--all``` to have all Arm NN dependencies available for use during Arm NN builds. Repeated docker builds with the same ```SETUP_ARGS``` will skip the setup process (using [caching](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache) of previous Docker build steps). The ```SETUP_ARGS``` string should start and end with **double quotes** ```"```. | SETUP_ARGS | Description | |----------------------------|:-------------------------------------------------------------------------------------| | --tflite-classic-delegate | **flag:** setup dependencies for the existing Arm NN TF Lite Delegate | | --tflite-opaque-delegate | **flag:** setup dependencies for the new Arm NN Opaque Delegate | | --tflite-parser | **flag:** setup dependencies for the Arm NN TF Lite Parser | | --onnx-parser | **flag:** setup dependencies for the Arm NN ONNX parser | | --all | **flag:** setup dependencies for all Arm NN components listed above | | --target-arch= | **mandatory option:** specify a target architecture ```aarch64, x86_64, android64``` | **At least one component** (e.g. ```--tflite-classic-delegate```) must be provided **or** else provide ```--all``` to setup dependencies for all components. **Examples:**
Setup for aarch64 with all Arm NN dependencies:
```SETUP_ARGS="--target-arch=aarch64 --all"``` Setup for aarch64 with the existing TF Lite Delegate and TF Lite Parser dependencies only:
```SETUP_ARGS="--target-arch=aarch64 --tflite-classic-delegate --tflite-parser"```
### BUILD_ARGS The following arguments are given to ```build-armnn.sh``` and define which components of Arm NN to include in the build. The ```BUILD_ARGS``` string should start and end with **double quotes** ```"```. | BUILD_ARGS | Description | |---------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | --tflite-classic-delegate | **flag:** build the existing Arm NN TF Lite Delegate component | | --tflite-opaque-delegate | **flag:** build the new Arm NN Opaque Delegate | | --tflite-parser | **flag:** build the Arm NN TF Lite Parser component | | --onnx-parser | **flag:** build the Arm NN ONNX parser component | | --all | **flag:** build all Arm NN components listed above | | --target-arch= | **mandatory option:** specify a target architecture ```aarch64, x86_64, android64``` | | --neon-backend | **flag:** build Arm NN with the NEON backend (CPU acceleration from ACL) | | --cl-backend | **flag:** build Arm NN with the OpenCL backend (GPU acceleration from ACL) | | --ref-backend | **flag:** build Arm NN with the reference backend
**Should be used for verification purposes only.
Does not provide any performance acceleration.** | | --debug | **flag:** build Arm NN (and ACL) with debug turned on (optional: defaults to off) | | --clean | **flag:** remove previous Arm NN and ACL build prior to script execution (optional: defaults to off) | | --symlink-armnn | **flag:** instead of cloning, make a symbolic link from the armnn directory containing the build-tool to the source directory | | --armnn-cmake-args= | **option:** provide additional comma-separated CMake arguments string for building Arm NN (optional)
String should start and end with **single quotes** ```'```
Please refer to **armnn/cmake/GlobalConfig.cmake** | | --acl-scons-params= | **option**: provide additional comma-separated scons parameters string for building ACL (optional)
String should start and end with **single quotes** ```'```
ACL provide [documentation](https://arm-software.github.io/ComputeLibrary/latest/how_to_build.xhtml#S1_1_build_options) for their build options | **At least one component** (i.e. ```--tflite-classic-delegate```, ```--tflite-opaque-delegate```, ```--tflite-parser```, ```--onnx-parser```) must be provided or else provide ```--all``` to build all Arm NN components.
**A component given in ```BUILD_ARGS``` must also have been given in ```SETUP_ARGS``` previously**, or else Arm NN will fail to build.
**At least one backend** (i.e. ```--neon-backend```, ```--cl-backend```, ```--ref-backend```) must be chosen.
**Examples:**
Build for aarch64 with all Arm NN components, NEON enabled and OpenCL enabled:
```BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend"``` Build for aarch64 with the existing Arm NN TF Lite Delegate, OpenCL enabled and additional ACL scons params:
```BUILD_ARGS="--target-arch=aarch64 --tflite-classic-delegate --cl-backend --acl-scons-params='compress_kernels=1,benchmark_examples=1'"``` Setup for aarch64 with all Arm NN dependencies, OpenCL enabled and additional Arm NN cmake args:
```BUILD_ARGS="--target-arch=aarch64 --all --cl-backend --armnn-cmake-args='-DBUILD_SAMPLE_APP=1,-DBUILD_UNIT_TESTS=0'"``` **Example _valid_ combination of SETUP_ARGS and BUILD_ARGS:**
``` SETUP_ARGS="--target-arch=aarch64 --all" BUILD_ARGS="--target-arch=aarch64 --tflite-classic-delegate --neon-backend --cl-backend" ``` **Example _invalid_ combination of SETUP_ARGS and BUILD_ARGS:**
``` SETUP_ARGS="--target-arch=aarch64 --tflite-classic-delegate" BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend" ``` The example above is invalid since it tries to build all Arm NN components after **only** building the dependencies required by the TF Lite Delegate.
## Docker Build Steps First clone the Arm NN repo to obtain the build-tool. This automatically downloads the latest release version of Arm NN. ``` git clone https://github.com/ARM-software/armnn.git armnn ```
Enter into the Arm NN build-tool directory where the Dockerfile and associated scripts are located. ``` cd armnn/build-tool ```
Run the ```docker build```, which downloads and builds Arm NN and its dependencies. This process is isolated from the host machine's filesystem and the result is a Docker Image being created.
Dockerfile arguments are provided using ```--build-arg```, the relative path to the Dockerfile is given with ```--file``` and the current directory is given with ```.```
Provide a descriptive name for the Image with ```--tag``` in the form of ```image_name:tag``` (example given below). The backslashes ```\``` tell Bash to expect a continuation of the command on the next line. This example's choice of **[SETUP_ARGS](#setup_args)** and **[BUILD_ARGS](#build_args)** builds all Arm NN components with NEON and OpenCL accelerated backends for aarch64 target. This process should take **less than an hour** with a modern machine but will vary depending on the arguments chosen and host machine specification. ``` docker build \ --build-arg SETUP_ARGS="--target-arch=aarch64 --all" \ --build-arg BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend" \ --tag armnn:aarch64 \ --file docker/Dockerfile . ```
Arm NN and all of its dependencies are now built within a Docker Image. Built Docker images can be listed with ```docker images```. ``` docker images REPOSITORY TAG IMAGE ID CREATED SIZE armnn aarch64 cc623174fd98 7 seconds ago 5.02GB ```
A tarball archive of the Arm NN build is located inside the Docker home directory (```/home/arm-user/```), named ```armnn__build.tar.gz```. If the ```--debug``` build flag is provided in the ```BUILD_ARGS``` above, the tarball will be named ```armnn__build_debug.tar.gz```.
The ```docker-copy-to-host.sh``` script will copy a file from the Docker Image (in arm-user's home directory) to the host machine.
The script copies the tarball into a new directory on the host in ```build-tool/docker_output```. It takes two arguments: ```image_name:tag``` and ```filename```.
The filename is a relative path from the home directory created inside the Docker Image (```/home/arm-user/```). ``` ./scripts/docker-copy-to-host.sh armnn:aarch64 armnn_aarch64_build.tar.gz ```
This tarball can now be used for integration in an ML application. The method of extraction is given below.
If the ```--target-arch``` chosen in the arguments above matches the host machine, the build can be tested locally (else copy the tarball to a remote device). ``` cd docker_output # Extract the tarball into a directory called _build # If --debug is enabled, the extracted build directory will be called _build_debug tar -xzf armnn_aarch64_build.tar.gz cd aarch64_build # Set LD_LIBRARY_PATH to the current aarch64_build directory (.) export LD_LIBRARY_PATH=.; ./UnitTests # If the Arm NN TF Lite Delegate is built, we can also run DelegateUnitTests cd delegate # Set LD_LIBRARY_PATH to the current delegate directory (.) and the aarch64_build directory (..) export LD_LIBRARY_PATH=.:..; ./DelegateUnitTests ```
**Note:** Repeated Docker builds may result in an accumulation of unwanted, dangling images. **To remove dangling images** (good practice), run the following command: ```docker image prune```
## Build Arm NN without Docker For ease of use, we recommend that the official Arm NN Dockerfile is used to build Arm NN. If you'd like to build locally on a Ubuntu host machine without Docker, execute the build-tool scripts directly instead. This involves running ```install-packages.sh``` followed by ```setup-armnn.sh``` and ```build-armnn.sh```. The arguments for ```setup-armnn.sh``` and ```build-armnn.sh``` are very much the same as ```SETUP_ARGS``` and ```BUILD_ARGS```. Please use ```--help``` for more specific information. ``` cd build-tool/scripts sudo ./install-packages.sh ./setup-armnn.sh --target-arch=aarch64 --all ./build-armnn.sh --target-arch=aarch64 --all --neon-backend --cl-backend ./setup-armnn.sh --help ./build-armnn.sh --help ```
## Advanced Usage ### Use custom Arm NN and ACL repository versions during build By default, the docker build process (specifically, during ```build-armnn.sh```) will download the latest release versions of Arm NN and ACL. If you'd like to use different versions during the build, check them out in the ```build-tool``` directory on the host.
When providing custom repositories, the following ```docker build``` argument must be provided ```--build-arg BUILD_TYPE=dev```. This will trigger Docker to copy the custom repos into the Docker Image during build. The ACL repo is only required if supplying the ```--neon-backend``` or ```--cl-backend``` BUILD_ARGS options. **Note:** the Arm NN version used for build-tool (Dockerfile and scripts) is not the same version of Arm NN that is used during the build. This means that separate versions of Arm NN can be used for the build-tool and for building Arm NN itself.
If you repeat the ```docker build``` process multiple times (e.g. with different ```BUILD_ARGS```), the setup process will be skipped as long as ```SETUP_ARGS``` remains the same. ``` cd build-tool git clone https://github.com/ARM-software/armnn.git armnn cd armnn git checkout cd .. # custom ACL repo only required when supplying --neon-backend or --cl-backend BUILD_ARGS options git clone https://github.com/ARM-software/ComputeLibrary.git acl cd acl git checkout cd .. # Example docker build with BUILD_TYPE=dev, ran inside the build-tool directory docker build \ --build-arg BUILD_TYPE=dev \ --build-arg SETUP_ARGS="--target-arch=aarch64 --all" \ --build-arg BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend" \ --tag armnn:aarch64 \ --file docker/Dockerfile . ```
### Additional Docker Build Arguments #### UBUNTU_VERSION The default base Image used during ```docker build``` is ```ubuntu:20.04```. Building Arm NN with this default image should be sufficient for a wide range of target devices. To use a different Ubuntu base Image, provide ```UBUNTU_VERSION``` during ```docker build``` e.g. ```--build-arg UBUNTU_VERSION=20.04```. #### BUILD_TYPE By default, ```BUILD_TYPE``` is set to ```production```. This means that the latest release versions of Arm NN and ACL are used during builds. To use custom repositories as explained [above](#Use-custom-Arm-NN-and-ACL-repository-versions-during-build), provide ```--build-arg BUILD_TYPE=dev```. #### Additional SETUP_ARGS and BUILD_ARGS The following are additional ```SETUP_ARGS``` and ```BUILD_ARGS``` that can be provided to the ```docker build```. | SETUP_ARGS / BUILD_ARGS | Description | |-------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------| | --num-threads= | **option:** specify number of threads/cores to build Arm NN and its dependencies with (this defaults to number of online cores on the host)
e.g. --num-threads=4 | | -x | **flag:** enable shell tracing during setup-armnn.sh and build-armnn.sh script execution | #### USER_ID and GROUP_ID By default, the Docker Image creates a user called ```arm-user``` with user/group ID of 1000. This ensures the build outputs can be used by a wide range of users on target devices.
To provide a more specific user and group ID to be associated with the build, provide ```--build-arg USER_ID=XXXX``` and ```--build-arg GROUP_ID=YYYY```.
### Inspect the contents of the Docker Image To inspect the full contents of the Docker Image, the Image must be run as a **Docker Container**. A Container is a runtime version of the Image, like a **lightweight virtual machine** with its own filesystem. The following shows how to run a container with an interactive terminal based on a specified Docker Image ```image_name:tag```. The ID of the generated container in this example is ```4da7c575a95a```. An ```ls``` command shows the contents of the ```/home/arm-user``` directory. ``` docker run --interactive --tty armnn:aarch64 arm-user@4da7c575a95a:~$ ls armnn_aarch64_build.tar.gz build build-armnn.sh common.sh setup-armnn.sh source validation.sh ``` The ```source``` directory contains the downloaded dependencies for building Arm NN.
The ```build``` directory contains the built Arm NN dependencies and the Arm NN build itself.
The ```armnn_aarch64_build.tar.gz``` tarball is an archive of the build located at ```build/armnn/aarch64_build```.
The scripts (e.g. ```build-armnn.sh```) in this directory can be run as you wish, changes will persist within the container but not the Docker Image. To exit the docker container, run ```CTRL-D```. A list of Docker containers can be obtained with the following command: ``` # --all shows both running and non-running containers docker ps --all CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 4da7c575a95a armnn:aarch64 "bash" 7 minutes ago Exited (0) 7 seconds ago pedantic_joliot ``` To restart the same container (and any changes made to it) in a terminal again, use the following commands: ``` # 4da7c575a95a in this case is the CONTAINER ID from above docker start 4da7c575a95a # restart container in background docker attach 4da7c575a95a # re-attach to terminal # After exiting with CTRL-D docker stop 4da7c575a95a ``` To save space over time, stopped Docker containers can be deleted with the following command:
```docker container prune```.
### Bind Mounts **Bind Mounts** may be used to mount directories from the host machine to inside a Docker container. This could be useful when frequently changing the contents of the Arm NN or ACL source repositories (in ```/home/arm-user/source```) between runs of build-armnn.sh inside the container. Note that bind mounts are a feature that apply to runtime versions of Docker images i.e. Docker Containers (```docker run```, not ```docker build```). Please refer to the [Docker documentation for more information about bind mounts](https://docs.docker.com/storage/bind-mounts/).