Arm NN Build Tool

The Arm NN Build Tool provides a collection of parameterized bash scripts accompanied by a Dockerfile for building Arm NN and its dependencies, including Arm Compute Library (ACL). This tool replaces/supersedes the majority of the existing Arm NN build guides as a user-friendly way to build Arm NN from scratch. The main benefit of building Arm NN from scratch is the ability to exactly choose which components to build, targeted for your ML project.
The Arm NN Build Tool is tested on x86_64 (Intel) and aarch64 (Arm) build hosts for the Ubuntu platform and on arm64 (Apple Silicon). At present, the tool supports targeting Linux devices (Ubuntu 20.04) and macOS (Mac with Apple silicon). In Linux it supports building on x86_64, aarch64 and android64 architectures.
We recommend using the Arm NN Build Tool through the use of Docker. However, the scripts may be executed directly on your machine if desired.

If you already have Docker installed and want to quickly build the Arm NN Dockerfile with some default build arguments, please follow the Docker Build Steps.

• Overview
• System Requirements
• Install Docker
• Ubuntu / Linux
• Mac
• Windows
• Docker Build Arguments Reference
• SETUP_ARGS
• BUILD_ARGS
• Docker Build Steps
• Build Arm NN Without Docker
• Advanced Usage
• Use custom Arm NN and ACL repository versions during build
• Additional Docker Build Arguments
• Inspect the contents of the Docker Image
• Bind Mounts

Overview

The Arm NN Build Tool consists of an official Arm NN Dockerfile with associated scripts for building Arm NN: 1. install-packages.sh: builds CMake from source and installs system-wide apt packages 2. setup-armnn.sh: downloads and builds Arm NN dependencies (e.g. Flatbuffers, TF Lite, Protobuf) 3. build-armnn.sh: builds Arm NN and ACL 4. common.sh: common variables and functions used by setup-armnn.sh and build-armnn.sh 5. validation.sh: validation of common command-line arguments to setup-armnn.sh and build-armnn.sh 6. docker-copy-to-host: helpful script to copy contents of a Docker Image to the host machine

A Dockerfile is a text document which outlines the instructions for building a Docker Image. Running docker build on the Arm NN Dockerfile will result in a Docker Image that contains: 1. The Ubuntu operating system 2. Installed system-wide packages required by Arm NN 3. Source code for Arm NN and its dependencies 4. The build for Arm NN and its dependencies

All scripts except docker-copy-to-host.sh are used during the docker build process.

The docker-copy-to-host.sh script creates a temporary Docker Container which is required to retrieve files from the Docker Image. A Docker Container is a runtime version of a Docker Image. It can be interacted with through a command-line terminal, like a lightweight virtual machine (VM). The docker-copy-to-host.sh script is designed to run on the host machine after docker build to copy the Arm NN build to the host.

The diagram below outlines the operation of the Arm NN Dockerfile and the contents of the resulting Docker Image after running docker build. During docker build, the provided SETUP_ARGS determine which of Arm NN's dependencies to download and build. The BUILD_ARGS Docker argument determines which components of Arm NN to build.

The resulting Docker Image contains the Arm NN build (along with everything required to build it) in the home directory /home/arm-user. There are two directories: source which contains the source code of Arm NN and its dependencies, and build which contains the Arm NN build and the builds of each dependency of Arm NN.

The armnn_aarch64_build.tar.gz tarball in the home directory is a compressed archive of the Arm NN build which is located at /home/arm-user/build/armnn. The home directory also contains the scripts mentioned above (not pictured in the diagram), which are copied into the Docker Image during the docker build process.

Benefits of using Docker: 1. Run one docker build command instead of executing multiple scripts, saving developer time 2. Can easily manage multiple different build configurations at once, each in their own Docker Image 3. System packages are installed in the Docker Image only, isolated from the host machine 4. Build in the exact environment the Arm NN Team used during testing, ensuring consistency in builds 5. Allows Arm NN to be built on a wide range of hosts, as long as they support Docker

Disadvantages of using Docker: 1. Requires the developer to install Docker on their host machine 2. Requires the developer to get familiar with Docker (this guide should help) 3. Possible incompatibilities/limitations if the host machine's Linux Kernel version is too different than that of the base OS in the Docker Image. This is due to the fact that Docker Containers share the Linux Kernel of the host machine (unlike VM's which each have their own Kernel).

System Requirements

This guide will not provide the system requirements for installing Docker - please refer to the Docker documentation during installation.

The built Docker Image has disk requirements of less than 6GB, depending on the Arm NN components chosen and whether the build included a cross compile (i.e. building aarch64 on x86_64). Repeated builds may cause the total disk usage to increase beyond 6GB. The docker images command shows the disk usage of each Docker Image. To view total disk space used by all Docker Images and Docker Containers, use the command docker system df.

This Dockerfile was built on a Ubuntu 20.04 host machine with Docker version 20.10.14. Other Linux distros such as Debian should also work as long as the Docker version is similar to 20.10.14. This should also work on Mac with Apple silicon but needs some manual setup to be done. Please find the steps in install-packages.sh.

The build outputs from this Dockerfile have been tested on an Odroid N2+ target device with Ubuntu 20.04 installed. The build outputs should also work on newer versions of Ubuntu (and Debian) but this has not been exhaustively tested.

To view the system packages that are installed during docker build, please refer to the install-packages.sh script.

Install Docker

Builds of the official Arm NN Dockerfile have been tested on the Ubuntu 20.04 and Mac with Apple silicon host platforms. Whilst other platforms have not been tested, Docker should be able to build the Arm NN Dockerfile on other platforms such as Windows. On Linux, we recommend using Docker Engine which is used through the command-line interface (CLI). For Windows and Mac users, Docker Desktop is available which allows the use of the CLI plus a graphical interface.
This guide will only provide steps on using Docker through the use of the CLI and not Docker Desktop.

Ubuntu / Linux

For Ubuntu users, we recommend installing Docker Engine with the following steps from the Docker documentation:
1. Uninstall any old versions of Docker 2. Install using the repository 3. Add user to Docker Unix Group (recommended: allows running of Docker commands without sudo)

Similar steps for installing Docker Engine on other Linux distributions can be found by clicking on your distro under the Server table on this page.

Mac

The Dockerfile will also work on Mac with Apple silicon. But we haven't tested this using Docker Desktop. For Mac users, please use the Docker Desktop download link on this page.

Windows

We do not officially test the building of the Arm NN Dockerfile on Windows. However, it's likely that it will work if your system supports Docker. For Windows, use the Docker Desktop download link on this page. Note: we unfortunately cannot provide support for the installation of Docker and/or the use of the Arm NN Dockerfile on Windows.

Docker Build Arguments Reference

The following describes the docker build arguments that can be used to customize the Arm NN build. Docker build arguments such as SETUP_ARGS and BUILD_ARGS are provided in the docker build as a --build-arg.

SETUP_ARGS

These arguments are eventually passed to setup-armnn.sh which downloads and builds Arm NN dependencies. For ease of use (but longer initial docker build), use --all to have all Arm NN dependencies available for use during Arm NN builds. Repeated docker builds with the same SETUP_ARGS will skip the setup process (using caching of previous Docker build steps). The SETUP_ARGS string should start and end with double quotes ".

At least one component (e.g. --tflite-classic-delegate) must be provided or else provide --all to setup dependencies for all components.

Examples:
Setup for aarch64 with all Arm NN dependencies:
SETUP_ARGS="--target-arch=aarch64 --all"

Setup for aarch64 with the existing TF Lite Delegate and TF Lite Parser dependencies only:
SETUP_ARGS="--target-arch=aarch64 --tflite-classic-delegate --tflite-parser"

BUILD_ARGS

The following arguments are given to build-armnn.sh and define which components of Arm NN to include in the build. The BUILD_ARGS string should start and end with double quotes ".

At least one component (i.e. --tflite-classic-delegate, --tflite-opaque-delegate, --tflite-parser, --onnx-parser) must be provided or else provide --all to build all Arm NN components.
A component given in BUILD_ARGS must also have been given in SETUP_ARGS previously, or else Arm NN will fail to build.
At least one backend (i.e. --neon-backend, --cl-backend, --ref-backend) must be chosen.

Examples:
Build for aarch64 with all Arm NN components, NEON enabled and OpenCL enabled:
BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend"

Build for aarch64 with the existing Arm NN TF Lite Delegate, OpenCL enabled and additional ACL scons params:
BUILD_ARGS="--target-arch=aarch64 --tflite-classic-delegate --cl-backend --acl-scons-params='compress_kernels=1,benchmark_examples=1'"

Setup for aarch64 with all Arm NN dependencies, OpenCL enabled and additional Arm NN cmake args:
BUILD_ARGS="--target-arch=aarch64 --all --cl-backend --armnn-cmake-args='-DBUILD_SAMPLE_APP=1,-DBUILD_UNIT_TESTS=0'"

Example valid combination of SETUP_ARGS and BUILD_ARGS:

SETUP_ARGS="--target-arch=aarch64 --all"
BUILD_ARGS="--target-arch=aarch64 --tflite-classic-delegate --neon-backend --cl-backend"

Example invalid combination of SETUP_ARGS and BUILD_ARGS:

SETUP_ARGS="--target-arch=aarch64 --tflite-classic-delegate"
BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend"

The example above is invalid since it tries to build all Arm NN components after only building the dependencies required by the TF Lite Delegate.

Docker Build Steps

First clone the Arm NN repo to obtain the build-tool. This automatically downloads the latest release version of Arm NN.

git clone https://github.com/ARM-software/armnn.git armnn

Enter into the Arm NN build-tool directory where the Dockerfile and associated scripts are located.

cd armnn/build-tool

Run the docker build, which downloads and builds Arm NN and its dependencies. This process is isolated from the host machine's filesystem and the result is a Docker Image being created.
Dockerfile arguments are provided using --build-arg, the relative path to the Dockerfile is given with --file and the current directory is given with .
Provide a descriptive name for the Image with --tag in the form of image_name:tag (example given below). The backslashes \ tell Bash to expect a continuation of the command on the next line.

This example's choice of SETUP_ARGS and BUILD_ARGS builds all Arm NN components with NEON and OpenCL accelerated backends for aarch64 target. This process should take less than an hour with a modern machine but will vary depending on the arguments chosen and host machine specification.

docker build \
--build-arg SETUP_ARGS="--target-arch=aarch64 --all" \
--build-arg BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend" \
--tag armnn:aarch64 \
--file docker/Dockerfile .

Arm NN and all of its dependencies are now built within a Docker Image. Built Docker images can be listed with docker images.

docker images

REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
armnn        aarch64   cc623174fd98   7 seconds ago    5.02GB

A tarball archive of the Arm NN build is located inside the Docker home directory (/home/arm-user/), named armnn_<target-arch>_build.tar.gz. If the --debug build flag is provided in the BUILD_ARGS above, the tarball will be named armnn_<target-arch>_build_debug.tar.gz.
The docker-copy-to-host.sh script will copy a file from the Docker Image (in arm-user's home directory) to the host machine.
The script copies the tarball into a new directory on the host in build-tool/docker_output. It takes two arguments: image_name:tag and filename.
The filename is a relative path from the home directory created inside the Docker Image (/home/arm-user/).

./scripts/docker-copy-to-host.sh armnn:aarch64 armnn_aarch64_build.tar.gz

This tarball can now be used for integration in an ML application. The method of extraction is given below.
If the --target-arch chosen in the arguments above matches the host machine, the build can be tested locally (else copy the tarball to a remote device).

cd docker_output

# Extract the tarball into a directory called <target_arch>_build
# If --debug is enabled, the extracted build directory will be called <target_arch>_build_debug
tar -xzf armnn_aarch64_build.tar.gz
cd aarch64_build

# Set LD_LIBRARY_PATH to the current aarch64_build directory (.)
export LD_LIBRARY_PATH=.; ./UnitTests

# If the Arm NN TF Lite Delegate is built, we can also run DelegateUnitTests
cd delegate

# Set LD_LIBRARY_PATH to the current delegate directory (.) and the aarch64_build directory (..)
export LD_LIBRARY_PATH=.:..; ./DelegateUnitTests

Note: Repeated Docker builds may result in an accumulation of unwanted, dangling images. To remove dangling images (good practice), run the following command: docker image prune

Build Arm NN without Docker

For ease of use, we recommend that the official Arm NN Dockerfile is used to build Arm NN. If you'd like to build locally on a Ubuntu host machine without Docker, execute the build-tool scripts directly instead. This involves running install-packages.sh followed by setup-armnn.sh and build-armnn.sh. The arguments for setup-armnn.sh and build-armnn.sh are very much the same as SETUP_ARGS and BUILD_ARGS. Please use --help for more specific information.

cd build-tool/scripts

sudo ./install-packages.sh

./setup-armnn.sh --target-arch=aarch64 --all
./build-armnn.sh --target-arch=aarch64 --all --neon-backend --cl-backend

./setup-armnn.sh --help
./build-armnn.sh --help

Advanced Usage

Use custom Arm NN and ACL repository versions during build

By default, the docker build process (specifically, during build-armnn.sh) will download the latest release versions of Arm NN and ACL. If you'd like to use different versions during the build, check them out in the build-tool directory on the host.
When providing custom repositories, the following docker build argument must be provided --build-arg BUILD_TYPE=dev. This will trigger Docker to copy the custom repos into the Docker Image during build. The ACL repo is only required if supplying the --neon-backend or --cl-backend BUILD_ARGS options.

Note: the Arm NN version used for build-tool (Dockerfile and scripts) is not the same version of Arm NN that is used during the build. This means that separate versions of Arm NN can be used for the build-tool and for building Arm NN itself.
If you repeat the docker build process multiple times (e.g. with different BUILD_ARGS), the setup process will be skipped as long as SETUP_ARGS remains the same.

cd build-tool

git clone https://github.com/ARM-software/armnn.git armnn
cd armnn
git checkout <branch or SHA>

cd ..

# custom ACL repo only required when supplying --neon-backend or --cl-backend BUILD_ARGS options
git clone https://github.com/ARM-software/ComputeLibrary.git acl
cd acl
git checkout <tag or SHA>

cd ..

# Example docker build with BUILD_TYPE=dev, ran inside the build-tool directory
docker build \
--build-arg BUILD_TYPE=dev \
--build-arg SETUP_ARGS="--target-arch=aarch64 --all" \
--build-arg BUILD_ARGS="--target-arch=aarch64 --all --neon-backend --cl-backend" \
--tag armnn:aarch64 \
--file docker/Dockerfile .

Additional Docker Build Arguments

UBUNTU_VERSION

The default base Image used during docker build is ubuntu:20.04. Building Arm NN with this default image should be sufficient for a wide range of target devices. To use a different Ubuntu base Image, provide UBUNTU_VERSION during docker build e.g. --build-arg UBUNTU_VERSION=20.04.

BUILD_TYPE

By default, BUILD_TYPE is set to production. This means that the latest release versions of Arm NN and ACL are used during builds. To use custom repositories as explained above, provide --build-arg BUILD_TYPE=dev.

Additional SETUP_ARGS and BUILD_ARGS

The following are additional SETUP_ARGS and BUILD_ARGS that can be provided to the docker build.

USER_ID and GROUP_ID

By default, the Docker Image creates a user called arm-user with user/group ID of 1000. This ensures the build outputs can be used by a wide range of users on target devices.
To provide a more specific user and group ID to be associated with the build, provide --build-arg USER_ID=XXXX and --build-arg GROUP_ID=YYYY.

Inspect the contents of the Docker Image

To inspect the full contents of the Docker Image, the Image must be run as a Docker Container. A Container is a runtime version of the Image, like a lightweight virtual machine with its own filesystem.

The following shows how to run a container with an interactive terminal based on a specified Docker Image image_name:tag. The ID of the generated container in this example is 4da7c575a95a. An ls command shows the contents of the /home/arm-user directory.

docker run --interactive --tty armnn:aarch64

arm-user@4da7c575a95a:~$ ls
armnn_aarch64_build.tar.gz  build  build-armnn.sh  common.sh  setup-armnn.sh  source  validation.sh

The source directory contains the downloaded dependencies for building Arm NN.
The build directory contains the built Arm NN dependencies and the Arm NN build itself.
The armnn_aarch64_build.tar.gz tarball is an archive of the build located at build/armnn/aarch64_build.
The scripts (e.g. build-armnn.sh) in this directory can be run as you wish, changes will persist within the container but not the Docker Image. To exit the docker container, run CTRL-D. A list of Docker containers can be obtained with the following command:

# --all shows both running and non-running containers
docker ps --all

CONTAINER ID   IMAGE           COMMAND    CREATED          STATUS                     PORTS    NAMES
4da7c575a95a   armnn:aarch64   "bash"     7 minutes ago    Exited (0) 7 seconds ago            pedantic_joliot

To restart the same container (and any changes made to it) in a terminal again, use the following commands:

# 4da7c575a95a in this case is the CONTAINER ID from above

docker start 4da7c575a95a # restart container in background
docker attach 4da7c575a95a # re-attach to terminal

# After exiting with CTRL-D
docker stop 4da7c575a95a

To save space over time, stopped Docker containers can be deleted with the following command:
docker container prune.

Bind Mounts

Bind Mounts may be used to mount directories from the host machine to inside a Docker container. This could be useful when frequently changing the contents of the Arm NN or ACL source repositories (in /home/arm-user/source) between runs of build-armnn.sh inside the container. Note that bind mounts are a feature that apply to runtime versions of Docker images i.e. Docker Containers (docker run, not docker build). Please refer to the Docker documentation for more information about bind mounts.