1 # How to Cross-Compile Arm NN on x86_64 for arm64
3 - [Introduction](#introduction)
4 - [Cross-compiling ToolChain](#cross-compiling-toolchain)
5 - [Build and install Google's Protobuf library](#build-and-install-google-s-protobuf-library)
6 - [Build Compute Library](#build-compute-library)
7 - [Download ArmNN](#download-armnn)
8 - [Build Flatbuffer](#build-flatbuffer)
9 - [Build Onnx](#build-onnx)
10 - [Build TfLite](#build-tflite)
11 - [Build Arm NN](#build-armnn)
12 - [Build Standalone Sample Dynamic Backend](#build-standalone-sample-dynamic-backend)
13 - [Run Unit Tests](#run-unit-tests)
14 - [Troubleshooting and Errors:](#troubleshooting-and-errors-)
18 These are the step by step instructions on Cross-Compiling Arm NN under an x86_64 system to target an Arm64 system. This build flow has been tested with Ubuntu 18.04 and it depends on the same version of Ubuntu or Debian being installed on both the build host and target machines. The instructions assume you are using a bash shell and show how to build the Arm NN core library, Protobuf, Tflite, Flatbuffer and Compute Libraries.
19 Start by creating a directory to contain all components:
22 mkdir $HOME/armnn-devenv
27 ## Cross-compiling ToolChain
28 * Install the standard cross-compilation libraries for arm64:
30 sudo apt install crossbuild-essential-arm64
33 ## Build and install Google's Protobuf library
35 We support protobuf version 3.12.0
36 * Get protobuf from here: https://github.com/protocolbuffers/protobuf :
38 git clone -b v3.12.0 https://github.com/google/protobuf.git protobuf
40 git submodule update --init --recursive
43 * Build a native (x86_64) version of the protobuf libraries and compiler (protoc):
44 (Requires cUrl, autoconf, llibtool, and other build dependencies if not previously installed: sudo apt install curl autoconf libtool build-essential g++)
48 ../configure --prefix=$HOME/armnn-devenv/google/x86_64_pb_install
52 * Build the arm64 version of the protobuf libraries:
56 CC=aarch64-linux-gnu-gcc \
57 CXX=aarch64-linux-gnu-g++ \
58 ../configure --host=aarch64-linux \
59 --prefix=$HOME/armnn-devenv/google/arm64_pb_install \
60 --with-protoc=$HOME/armnn-devenv/google/x86_64_pb_install/bin/protoc
66 ## Build Compute Library
67 * Building the Arm Compute Library:
70 git clone https://github.com/ARM-software/ComputeLibrary.git
72 git checkout <tag_name>
73 scons arch=arm64-v8a neon=1 opencl=1 embed_kernels=1 extra_cxx_flags="-fPIC" -j4 internal_only=0
76 For example, if you want to checkout release tag of 21.02:
85 git clone https://github.com/ARM-software/armnn.git
87 git checkout <branch_name>
91 For example, if you want to checkout release branch of 21.02:
93 git checkout branches/armnn_21_02
98 * Building Flatbuffer version 1.12.0
100 cd $HOME/armnn-devenv
101 wget -O flatbuffers-1.12.0.tar.gz https://github.com/google/flatbuffers/archive/v1.12.0.tar.gz
102 tar xf flatbuffers-1.12.0.tar.gz
103 cd flatbuffers-1.12.0
107 cmake .. -DFLATBUFFERS_BUILD_FLATC=1 \
108 -DCMAKE_INSTALL_PREFIX:PATH=$HOME/armnn-devenv/flatbuffers \
109 -DFLATBUFFERS_BUILD_TESTS=0
113 * Build arm64 version of flatbuffer
118 # Add -fPIC to allow us to use the libraries in shared objects.
119 CXXFLAGS="-fPIC" cmake .. -DCMAKE_C_COMPILER=/usr/bin/aarch64-linux-gnu-gcc \
120 -DCMAKE_CXX_COMPILER=/usr/bin/aarch64-linux-gnu-g++ \
121 -DFLATBUFFERS_BUILD_FLATC=1 \
122 -DCMAKE_INSTALL_PREFIX:PATH=$HOME/armnn-devenv/flatbuffers-arm64 \
123 -DFLATBUFFERS_BUILD_TESTS=0
130 cd $HOME/armnn-devenv
131 git clone https://github.com/onnx/onnx.git
133 git fetch https://github.com/onnx/onnx.git 553df22c67bee5f0fe6599cff60f1afc6748c635 && git checkout FETCH_HEAD
134 LD_LIBRARY_PATH=$HOME/armnn-devenv/google/x86_64_pb_install/lib:$LD_LIBRARY_PATH \
135 $HOME/armnn-devenv/google/x86_64_pb_install/bin/protoc \
136 onnx/onnx.proto --proto_path=. --proto_path=../google/x86_64_pb_install/include --cpp_out $HOME/armnn-devenv/onnx
140 * Building TfLite (Tensorflow version 2.5.1)
142 cd $HOME/armnn-devenv
143 git clone https://github.com/tensorflow/tensorflow.git
145 git checkout tags/v2.5.1
149 cp ../tensorflow/tensorflow/lite/schema/schema.fbs .
150 ../flatbuffers-1.12.0/build/flatc -c --gen-object-api --reflect-types --reflect-names schema.fbs
154 * Compile Arm NN for arm64:
156 cd $HOME/armnn-devenv/armnn
161 * Use CMake to configure your build environment, update the following script and run it from the armnn/build directory to set up the Arm NN build:
164 CXX=aarch64-linux-gnu-g++ CC=aarch64-linux-gnu-gcc cmake .. \
165 -DARMCOMPUTE_ROOT=$HOME/armnn-devenv/ComputeLibrary \
166 -DARMCOMPUTE_BUILD_DIR=$HOME/armnn-devenv/ComputeLibrary/build/ \
167 -DARMCOMPUTENEON=1 -DARMCOMPUTECL=1 -DARMNNREF=1 \
168 -DONNX_GENERATED_SOURCES=$HOME/armnn-devenv/onnx \
169 -DBUILD_ONNX_PARSER=1 \
170 -DBUILD_TF_LITE_PARSER=1 \
171 -DTF_LITE_GENERATED_PATH=$HOME/armnn-devenv/tflite \
172 -DFLATBUFFERS_ROOT=$HOME/armnn-devenv/flatbuffers-arm64 \
173 -DFLATC_DIR=$HOME/armnn-devenv/flatbuffers-1.12.0/build \
174 -DPROTOBUF_ROOT=$HOME/armnn-devenv/google/x86_64_pb_install \
175 -DPROTOBUF_ROOT=$HOME/armnn-devenv/google/x86_64_pb_install/ \
176 -DPROTOBUF_LIBRARY_DEBUG=$HOME/armnn-devenv/google/arm64_pb_install/lib/libprotobuf.so.23.0.0 \
177 -DPROTOBUF_LIBRARY_RELEASE=$HOME/armnn-devenv/google/arm64_pb_install/lib/libprotobuf.so.23.0.0
180 * If you want to include standalone sample dynamic backend tests, add the argument to enable the tests and the dynamic backend path to the CMake command:
182 -DSAMPLE_DYNAMIC_BACKEND=1 \
183 -DDYNAMIC_BACKEND_PATHS=$SAMPLE_DYNAMIC_BACKEND_PATH
190 ## Build Standalone Sample Dynamic Backend
191 * The sample dynamic backend is located in armnn/src/dynamic/sample
193 cd $HOME/armnn-devenv/armnn/src/dynamic/sample
198 * Use CMake to configure your build environment, update the following script and run it from the armnn/src/dynamic/sample/build directory to set up the Arm NN build:
201 CXX=aarch64-linux-gnu-g++ CC=aarch64-linux-gnu-gcc cmake .. \
202 -DCMAKE_CXX_FLAGS=--std=c++14 \
203 -DARMNN_PATH=$HOME/armnn-devenv/armnn/build/libarmnn.so
212 * Copy the build folder to an arm64 linux machine
213 * Copy the libprotobuf.so.23.0.0 library file to the build folder
214 * If you enable the standalone sample dynamic tests, also copy libArm_SampleDynamic_backend.so library file to the folder specified as $SAMPLE_DYNAMIC_BACKEND_PATH when you build Arm NN
215 * cd to the build folder on your arm64 machine and set your LD_LIBRARY_PATH to its current location:
221 * Create a symbolic link to libprotobuf.so.23.0.0:
224 ln -s libprotobuf.so.23.0.0 ./libprotobuf.so.23
230 LD_LIBRARY_PATH=./:$LD_LIBRARY_PATH ./UnitTests
231 Running 4493 test cases...
233 *** No errors detected
236 ## Troubleshooting and Errors:
237 ### Missing libz.so.1
238 * When compiling armNN:
240 /usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: warning: libz.so.1, needed by /home/<username>/armNN/usr/lib64/libprotobuf.so.23.0.0, not found (try using -rpath or -rpath-link)
243 * Missing arm64 libraries for libz.so.1, these can be added by adding a second architecture to dpkg and explicitly installing them:
245 sudo dpkg --add-architecture arm64
246 sudo apt-get install zlib1g:arm64
250 * If apt-get update returns 404 errors for arm64 repos refer to section 5 below.
251 * Alternatively the missing arm64 version of libz.so.1 can be downloaded and installed from a .deb package here:
252 https://launchpad.net/ubuntu/wily/arm64/zlib1g/1:1.2.8.dfsg-2ubuntu4
254 sudo dpkg -i zlib1g_1.2.8.dfsg-2ubuntu4_arm64.deb
258 ### Unable to install arm64 packages after adding arm64 architecture
259 * Using sudo apt-get update should add all of the required repos for arm64 but if it does not or you are getting 404 errors the following instructions can be used to add the repos manually:
260 * From stackoverflow:
261 https://askubuntu.com/questions/430705/how-to-use-apt-get-to-download-multi-arch-library/430718
262 * Open /etc/apt/sources.list with your preferred text editor.
264 * Mark all the current (default) repos as \[arch=<current_os_arch>], e.g.
266 deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ xenial main restricted
268 * Then add the following:
270 deb [arch=arm64] http://ports.ubuntu.com/ xenial main restricted
271 deb [arch=arm64] http://ports.ubuntu.com/ xenial-updates main restricted
272 deb [arch=arm64] http://ports.ubuntu.com/ xenial universe
273 deb [arch=arm64] http://ports.ubuntu.com/ xenial-updates universe
274 deb [arch=arm64] http://ports.ubuntu.com/ xenial multiverse
275 deb [arch=arm64] http://ports.ubuntu.com/ xenial-updates multiverse
276 deb [arch=arm64] http://ports.ubuntu.com/ xenial-backports main restricted universe multiverse
278 * Update and install again:
280 sudo apt-get install zlib1g:arm64
286 ### Undefined references to google::protobuf:: functions
287 * Missing or out of date protobuf compilation libraries.
288 Use the command 'protoc --version' to check which version of protobuf is available (version 3.12.0 is required).
289 Follow the instructions above to install protobuf 3.12.0
292 ### Errors on strict-aliasing rules when compiling the Compute Library
293 * When compiling the Compute Library there are multiple errors on strict-aliasing rules:
295 cc1plus: error: unrecognized command line option ‘-Wno-implicit-fallthrough’ [-Werror]
297 * Add Werror=0 to the scons command:
299 scons arch=arm64-v8a neon=1 opencl=1 embed_kernels=1 extra_cxx_flags="-fPIC" -j8 internal_only=0 Werror=0