1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
|
<!-- Copyright (c) 2020 ARM Limited. -->
<!-- -->
<!-- SPDX-License-Identifier: MIT -->
<!-- -->
<!-- HTML header for doxygen 1.8.13-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.13"/>
<meta name="robots" content="NOINDEX, NOFOLLOW" />
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>ArmNN: How to Cross-Compile Arm NN on x86_64 for arm64</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtreedata.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
</script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js"],
jax: ["input/TeX","output/HTML-CSS"],
});
</script><script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
<link href="stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<img alt="ArmNN" src="Arm_NN_horizontal_blue.png" style="max-width: 10rem; margin-top: .5rem; margin-left 10px"/>
<td style="padding-left: 0.5em;">
<div id="projectname">
 <span id="projectnumber">21.08</span>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.13 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "search",false,'Search');
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
$(function() {
initMenu('',true,false,'search.php','Search');
$(document).ready(function() { init_search(); });
});
</script>
<div id="main-nav"></div>
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
$(document).ready(function(){initNavTree('md__build_guide_cross_compilation.xhtml','');});
</script>
<div id="doc-content">
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>
<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0"
name="MSearchResults" id="MSearchResults">
</iframe>
</div>
<div class="header">
<div class="headertitle">
<div class="title">How to Cross-Compile Arm NN on x86_64 for arm64 </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><ul>
<li><a href="#introduction">Introduction</a></li>
<li><a href="#cross-compiling-toolchain">Cross-compiling ToolChain</a></li>
<li><a href="#build-and-install-google-s-protobuf-library">Build and install Google's Protobuf library</a></li>
<li><a href="#build-compute-library">Build Compute Library</a></li>
<li><a href="#download-armnn">Download ArmNN</a></li>
<li><a href="#build-flatbuffer">Build Flatbuffer</a></li>
<li><a href="#build-onnx">Build Onnx</a></li>
<li><a href="#build-tflite">Build TfLite</a></li>
<li><a href="#build-armnn">Build Arm NN</a></li>
<li><a href="#build-standalone-sample-dynamic-backend">Build Standalone Sample Dynamic Backend</a></li>
<li><a href="#run-unit-tests">Run Unit Tests</a></li>
<li><a href="#troubleshooting-and-errors-">Troubleshooting and Errors:</a></li>
</ul>
<h2>Introduction</h2>
<p>These are the step by step instructions on Cross-Compiling Arm NN under an x86_64 system to target an Arm64 system. This build flow has been tested with Ubuntu 16.04 and it depends on the same version of Ubuntu or Debian being installed on both the build host and target machines. The instructions assume you are using a bash shell and show how to build the Arm NN core library, Protobuf, Tflite, Flatbuffer and Compute Libraries. Start by creating a directory to contain all components:</p>
<p>''' mkdir $HOME/armnn-devenv cd $HOME/armnn-devenv '''</p>
<h2>Cross-compiling ToolChain</h2>
<ul>
<li>Install the standard cross-compilation libraries for arm64: <div class="fragment"><div class="line">sudo apt install crossbuild-essential-arm64</div></div><!-- fragment --></li>
</ul>
<h2>Build and install Google's Protobuf library</h2>
<p>We support protobuf version 3.12.0</p><ul>
<li>Get protobuf from here: <a href="https://github.com/protocolbuffers/protobuf">https://github.com/protocolbuffers/protobuf</a> : <div class="fragment"><div class="line">git clone -b v3.12.0 https://github.com/google/protobuf.git protobuf</div><div class="line">cd protobuf</div><div class="line">git submodule update --init --recursive</div><div class="line">./autogen.sh</div></div><!-- fragment --></li>
<li>Build a native (x86_64) version of the protobuf libraries and compiler (protoc): (Requires cUrl, autoconf, llibtool, and other build dependencies if not previously installed: sudo apt install curl autoconf libtool build-essential g++) <div class="fragment"><div class="line">mkdir x86_64_build</div><div class="line">cd x86_64_build</div><div class="line">../configure --prefix=$HOME/armnn-devenv/google/x86_64_pb_install</div><div class="line">make install -j16</div><div class="line">cd ..</div></div><!-- fragment --></li>
<li>Build the arm64 version of the protobuf libraries: <div class="fragment"><div class="line">mkdir arm64_build</div><div class="line">cd arm64_build</div><div class="line">CC=aarch64-linux-gnu-gcc \</div><div class="line">CXX=aarch64-linux-gnu-g++ \</div><div class="line">../configure --host=aarch64-linux \</div><div class="line">--prefix=$HOME/armnn-devenv/google/arm64_pb_install \</div><div class="line">--with-protoc=$HOME/armnn-devenv/google/x86_64_pb_install/bin/protoc</div><div class="line">make install -j16</div><div class="line">cd ..</div></div><!-- fragment --></li>
</ul>
<h2>Build Compute Library</h2>
<ul>
<li>Building the Arm Compute Library: <div class="fragment"><div class="line">cd $HOME/armnn-devenv</div><div class="line">git clone https://github.com/ARM-software/ComputeLibrary.git</div><div class="line">cd ComputeLibrary/</div><div class="line">git checkout <tag_name></div><div class="line">scons arch=arm64-v8a neon=1 opencl=1 embed_kernels=1 extra_cxx_flags="-fPIC" -j4 internal_only=0</div></div><!-- fragment --></li>
</ul>
<p>For example, if you want to checkout release tag of 21.02: </p><div class="fragment"><div class="line">git checkout v21.02</div></div><!-- fragment --><h2>Download ArmNN</h2>
<div class="fragment"><div class="line">cd $HOME/armnn-devenv</div><div class="line">git clone https://github.com/ARM-software/armnn.git</div><div class="line">cd armnn</div><div class="line">git checkout <branch_name></div><div class="line">git pull</div></div><!-- fragment --><p>For example, if you want to checkout release branch of 21.02: </p><div class="fragment"><div class="line">git checkout branches/armnn_21_02</div><div class="line">git pull</div></div><!-- fragment --><h2>Build Flatbuffer</h2>
<ul>
<li>Building Flatbuffer version 1.12.0 <div class="fragment"><div class="line">cd $HOME/armnn-devenv</div><div class="line">wget -O flatbuffers-1.12.0.tar.gz https://github.com/google/flatbuffers/archive/v1.12.0.tar.gz</div><div class="line">tar xf flatbuffers-1.12.0.tar.gz</div><div class="line">cd flatbuffers-1.12.0</div><div class="line">rm -f CMakeCache.txt</div><div class="line">mkdir build</div><div class="line">cd build</div><div class="line">cmake .. -DFLATBUFFERS_BUILD_FLATC=1 \</div><div class="line"> -DCMAKE_INSTALL_PREFIX:PATH=$HOME/armnn-devenv/flatbuffers \</div><div class="line"> -DFLATBUFFERS_BUILD_TESTS=0</div><div class="line">make all install</div></div><!-- fragment --></li>
<li>Build arm64 version of flatbuffer <div class="fragment"><div class="line">cd ..</div><div class="line">mkdir build-arm64</div><div class="line">cd build-arm64</div><div class="line"># Add -fPIC to allow us to use the libraries in shared objects.</div><div class="line">CXXFLAGS="-fPIC" cmake .. -DCMAKE_C_COMPILER=/usr/bin/aarch64-linux-gnu-gcc \</div><div class="line"> -DCMAKE_CXX_COMPILER=/usr/bin/aarch64-linux-gnu-g++ \</div><div class="line"> -DFLATBUFFERS_BUILD_FLATC=1 \</div><div class="line"> -DCMAKE_INSTALL_PREFIX:PATH=$HOME/armnn-devenv/flatbuffers-arm64 \</div><div class="line"> -DFLATBUFFERS_BUILD_TESTS=0</div><div class="line">make all install</div></div><!-- fragment --></li>
</ul>
<h2>Build Onnx</h2>
<ul>
<li>Building Onnx <div class="fragment"><div class="line">cd $HOME/armnn-devenv</div><div class="line">git clone https://github.com/onnx/onnx.git</div><div class="line">cd onnx</div><div class="line">git fetch https://github.com/onnx/onnx.git 553df22c67bee5f0fe6599cff60f1afc6748c635 && git checkout FETCH_HEAD</div><div class="line">LD_LIBRARY_PATH=$HOME/armnn-devenv/google/x86_64_pb_install/lib:$LD_LIBRARY_PATH \</div><div class="line">$HOME/armnn-devenv/google/x86_64_pb_install/bin/protoc \</div><div class="line">onnx/onnx.proto --proto_path=. --proto_path=../google/x86_64_pb_install/include --cpp_out $HOME/armnn-devenv/onnx</div></div><!-- fragment --></li>
</ul>
<h2>Build TfLite</h2>
<ul>
<li>Building TfLite (Tensorflow version 2.3.1) <div class="fragment"><div class="line">cd $HOME/armnn-devenv</div><div class="line">git clone https://github.com/tensorflow/tensorflow.git</div><div class="line">cd tensorflow/</div><div class="line">git checkout fcc4b966f1265f466e82617020af93670141b009</div><div class="line">cd ..</div><div class="line">mkdir tflite</div><div class="line">cd tflite</div><div class="line">cp ../tensorflow/tensorflow/lite/schema/schema.fbs .</div><div class="line">../flatbuffers-1.12.0/build/flatc -c --gen-object-api --reflect-types --reflect-names schema.fbs</div></div><!-- fragment --></li>
</ul>
<h2>Build Arm NN</h2>
<ul>
<li>Compile Arm NN for arm64: <div class="fragment"><div class="line">cd $HOME/armnn-devenv/armnn</div><div class="line">mkdir build</div><div class="line">cd build</div></div><!-- fragment --></li>
<li>Use CMake to configure your build environment, update the following script and run it from the armnn/build directory to set up the Arm NN build: <div class="fragment"><div class="line">#!/bin/bash</div><div class="line">CXX=aarch64-linux-gnu-g++ CC=aarch64-linux-gnu-gcc cmake .. \</div><div class="line">-DARMCOMPUTE_ROOT=$HOME/armnn-devenv/ComputeLibrary \</div><div class="line">-DARMCOMPUTE_BUILD_DIR=$HOME/armnn-devenv/ComputeLibrary/build/ \</div><div class="line">-DARMCOMPUTENEON=1 -DARMCOMPUTECL=1 -DARMNNREF=1 \</div><div class="line">-DONNX_GENERATED_SOURCES=$HOME/armnn-devenv/onnx \</div><div class="line">-DBUILD_ONNX_PARSER=1 \</div><div class="line">-DBUILD_TF_LITE_PARSER=1 \</div><div class="line">-DTF_LITE_GENERATED_PATH=$HOME/armnn-devenv/tflite \</div><div class="line">-DFLATBUFFERS_ROOT=$HOME/armnn-devenv/flatbuffers-arm64 \</div><div class="line">-DFLATC_DIR=$HOME/armnn-devenv/flatbuffers-1.12.0/build \</div><div class="line">-DPROTOBUF_ROOT=$HOME/armnn-devenv/google/x86_64_pb_install \</div><div class="line">-DPROTOBUF_ROOT=$HOME/armnn-devenv/google/x86_64_pb_install/ \</div><div class="line">-DPROTOBUF_LIBRARY_DEBUG=$HOME/armnn-devenv/google/arm64_pb_install/lib/libprotobuf.so.23.0.0 \</div><div class="line">-DPROTOBUF_LIBRARY_RELEASE=$HOME/armnn-devenv/google/arm64_pb_install/lib/libprotobuf.so.23.0.0</div></div><!-- fragment --></li>
<li>If you want to include standalone sample dynamic backend tests, add the argument to enable the tests and the dynamic backend path to the CMake command: <div class="fragment"><div class="line">-DSAMPLE_DYNAMIC_BACKEND=1 \</div><div class="line">-DDYNAMIC_BACKEND_PATHS=$SAMPLE_DYNAMIC_BACKEND_PATH</div></div><!-- fragment --></li>
<li>Run the build <div class="fragment"><div class="line">make -j32</div></div><!-- fragment --></li>
</ul>
<h2>Build Standalone Sample Dynamic Backend</h2>
<ul>
<li>The sample dynamic backend is located in armnn/src/dynamic/sample <div class="fragment"><div class="line">cd $HOME/armnn-devenv/armnn/src/dynamic/sample</div><div class="line">mkdir build</div><div class="line">cd build</div></div><!-- fragment --></li>
<li>Use CMake to configure your build environment, update the following script and run it from the armnn/src/dynamic/sample/build directory to set up the Arm NN build: <div class="fragment"><div class="line">#!/bin/bash</div><div class="line">CXX=aarch64-linux-gnu-g++ CC=aarch64-linux-gnu-gcc cmake .. \</div><div class="line">-DCMAKE_CXX_FLAGS=--std=c++14 \</div><div class="line">-DARMNN_PATH=$HOME/armnn-devenv/armnn/build/libarmnn.so</div></div><!-- fragment --></li>
<li>Run the build <div class="fragment"><div class="line">make</div></div><!-- fragment --></li>
</ul>
<h2>Run Unit Tests</h2>
<ul>
<li>Copy the build folder to an arm64 linux machine</li>
<li>Copy the libprotobuf.so.23.0.0 library file to the build folder</li>
<li>If you enable the standalone sample dynamic tests, also copy libArm_SampleDynamic_backend.so library file to the folder specified as $SAMPLE_DYNAMIC_BACKEND_PATH when you build Arm NN</li>
<li>cd to the build folder on your arm64 machine and set your LD_LIBRARY_PATH to its current location:</li>
</ul>
<div class="fragment"><div class="line">cd build/</div></div><!-- fragment --><ul>
<li>Create a symbolic link to libprotobuf.so.23.0.0:</li>
</ul>
<div class="fragment"><div class="line">ln -s libprotobuf.so.23.0.0 ./libprotobuf.so.23</div></div><!-- fragment --><ul>
<li>Run the UnitTests:</li>
</ul>
<div class="fragment"><div class="line">LD_LIBRARY_PATH=./:$LD_LIBRARY_PATH ./UnitTests</div><div class="line">Running 4493 test cases...</div><div class="line"></div><div class="line">*** No errors detected</div></div><!-- fragment --><h2>Troubleshooting and Errors:</h2>
<h3>Missing libz.so.1</h3>
<ul>
<li>When compiling armNN: <div class="fragment"><div class="line">/usr/lib/gcc-cross/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/bin/ld: warning: libz.so.1, needed by /home/<username>/armNN/usr/lib64/libprotobuf.so.23.0.0, not found (try using -rpath or -rpath-link)</div></div><!-- fragment --></li>
<li>Missing arm64 libraries for libz.so.1, these can be added by adding a second architecture to dpkg and explicitly installing them: <div class="fragment"><div class="line">sudo dpkg --add-architecture arm64</div><div class="line">sudo apt-get install zlib1g:arm64</div><div class="line">sudo apt-get update</div><div class="line">sudo ldconfig</div></div><!-- fragment --></li>
<li>If apt-get update returns 404 errors for arm64 repos refer to section 5 below.</li>
<li>Alternatively the missing arm64 version of libz.so.1 can be downloaded and installed from a .deb package here: <a href="https://launchpad.net/ubuntu/wily/arm64/zlib1g/1:1.2.8.dfsg-2ubuntu4">https://launchpad.net/ubuntu/wily/arm64/zlib1g/1:1.2.8.dfsg-2ubuntu4</a> <div class="fragment"><div class="line">sudo dpkg -i zlib1g_1.2.8.dfsg-2ubuntu4_arm64.deb</div></div><!-- fragment --> <br />
<br />
</li>
</ul>
<h3>Unable to install arm64 packages after adding arm64 architecture</h3>
<ul>
<li>Using sudo apt-get update should add all of the required repos for arm64 but if it does not or you are getting 404 errors the following instructions can be used to add the repos manually:</li>
<li>From stackoverflow: <a href="https://askubuntu.com/questions/430705/how-to-use-apt-get-to-download-multi-arch-library/430718">https://askubuntu.com/questions/430705/how-to-use-apt-get-to-download-multi-arch-library/430718</a></li>
<li>Open /etc/apt/sources.list with your preferred text editor.</li>
<li>Mark all the current (default) repos as [arch=<current_os_arch>], e.g. <div class="fragment"><div class="line">deb [arch=amd64] http://archive.ubuntu.com/ubuntu/ xenial main restricted</div></div><!-- fragment --></li>
<li>Then add the following: <div class="fragment"><div class="line">deb [arch=arm64] http://ports.ubuntu.com/ xenial main restricted</div><div class="line">deb [arch=arm64] http://ports.ubuntu.com/ xenial-updates main restricted</div><div class="line">deb [arch=arm64] http://ports.ubuntu.com/ xenial universe</div><div class="line">deb [arch=arm64] http://ports.ubuntu.com/ xenial-updates universe</div><div class="line">deb [arch=arm64] http://ports.ubuntu.com/ xenial multiverse</div><div class="line">deb [arch=arm64] http://ports.ubuntu.com/ xenial-updates multiverse</div><div class="line">deb [arch=arm64] http://ports.ubuntu.com/ xenial-backports main restricted universe multiverse</div></div><!-- fragment --></li>
<li>Update and install again: <div class="fragment"><div class="line">sudo apt-get install zlib1g:arm64</div><div class="line">sudo apt-get update</div><div class="line">sudo ldconfig</div></div><!-- fragment --> <br />
<br />
</li>
</ul>
<h3>Undefined references to google::protobuf:: functions</h3>
<ul>
<li>Missing or out of date protobuf compilation libraries. Use the command 'protoc –version' to check which version of protobuf is available (version 3.12.0 is required). Follow the instructions above to install protobuf 3.12.0 <br />
<br />
</li>
</ul>
<h3>Errors on strict-aliasing rules when compiling the Compute Library</h3>
<ul>
<li>When compiling the Compute Library there are multiple errors on strict-aliasing rules: <div class="fragment"><div class="line">cc1plus: error: unrecognized command line option ‘-Wno-implicit-fallthrough’ [-Werror]</div></div><!-- fragment --></li>
<li>Add Werror=0 to the scons command: <div class="fragment"><div class="line">scons arch=arm64-v8a neon=1 opencl=1 embed_kernels=1 extra_cxx_flags="-fPIC" -j8 internal_only=0 Werror=0</div></div><!-- fragment --> </li>
</ul>
</div></div><!-- contents -->
</div><!-- doc-content -->
<!-- start footer part -->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
<ul>
<li class="navelem"><a class="el" href="buildguides.xhtml">Installation and build Guides</a></li>
<li class="footer">Generated on Tue Aug 24 2021 16:18:47 for ArmNN by
<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.13 </li>
</ul>
</div>
</body>
</html>
|
