1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
|
<!-- HTML header for doxygen 1.8.17-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.17"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>Arm NN: PyArmNN</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtreedata.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js"],
jax: ["input/TeX","output/HTML-CSS"],
});
</script>
<script type="text/javascript" async="async" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
<link href="customdoxygen.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<img alt="ArmNN" src="Arm_NN_horizontal_blue.png" style="max-width: 15rem; margin-top: .5rem; margin-left 13px"/>
<td id="projectalign" style="padding-left: 0.9em;">
<div id="projectname">
 <span id="projectnumber">24.02</span>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.17 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
var searchBox = new SearchBox("searchBox", "search",false,'Search');
/* @license-end */
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
$(function() {
initMenu('',true,false,'search.php','Search');
$(document).ready(function() { init_search(); });
});
/* @license-end */</script>
<div id="main-nav"></div>
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
$(document).ready(function(){initNavTree('md_python_pyarmnn__r_e_a_d_m_e.html',''); initResizable(); });
/* @license-end */
</script>
<div id="doc-content">
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>
<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0"
name="MSearchResults" id="MSearchResults">
</iframe>
</div>
<div class="PageDoc"><div class="header">
<div class="headertitle">
<div class="title">PyArmNN </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>PyArmNN is a python extension for <a href="https://developer.arm.com/ip-products/processors/machine-learning/arm-nn">Arm NN SDK</a>. PyArmNN provides interface similar to Arm NN C++ Api. Before you proceed with the project setup, you will need to checkout and build a corresponding Arm NN version.</p>
<p>PyArmNN is built around public headers from the armnn/include folder of Arm NN. PyArmNN does not implement any computation kernels itself, all operations are delegated to the Arm NN library.</p>
<p>The <a href="http://www.swig.org/">SWIG</a> project is used to generate the Arm NN python shadow classes and C wrapper.</p>
<p>The following diagram shows the conceptual architecture of this library: <img src="../../docs/pyarmnn.png" alt="PyArmNN" class="inline"/></p>
<h1>Setup development environment</h1>
<p>Before, proceeding to the next steps, make sure that:</p>
<ol type="1">
<li>You have Python 3.6+ installed system-side. The package is not compatible with older Python versions.</li>
<li>You have python3.6-dev installed system-side. This contains header files needed to build PyArmNN extension module.</li>
<li>In case you build Python from sources manually, make sure that the following libraries are installed and available in you system: <code>python3.6-dev build-essential checkinstall libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev</code></li>
<li>Install SWIG 4.x. Only 3.x version is typically available in Linux package managers, so you will have to build it and install it from sources. It can be downloaded from the <a href="http://www.swig.org/download.html">SWIG project website</a> or from <a href="https://github.com/swig/swig">SWIG GitHub</a>. To install it follow the guide on <a href="https://github.com/swig/swig/wiki/Getting-Started">SWIG GitHub</a>.</li>
</ol>
<h2>Setup virtual environment</h2>
<p>Now you can proceed with setting up workspace. It is recommended to create a python virtual environment, so you do not pollute your working folder: </p><div class="fragment"><div class="line">python -m venv env</div>
<div class="line">source env/bin/activate</div>
</div><!-- fragment --><p>You may run into missing python modules such as <em>wheel</em>. Make sure to install those using pip: </p><div class="fragment"><div class="line">pip install wheel</div>
</div><!-- fragment --><h2>Build python distr</h2>
<p>Python supports source and binary distribution packages.</p>
<p>Source distr contains setup.py script that is executed on the users machine during package installation. When preparing binary distr (wheel), setup.py is executed on the build machine and the resulting package contains only the result of the build (generated files and resources, test results etc).</p>
<p>In our case, PyArmNN depends on Arm NN installation. Thus, binary distr will be linked with the local build machine libraries and runtime.</p>
<p>The recommended way to build the python packages is by CMake.</p>
<h3>CMake build</h3>
<p>The recommended approach is to build PyArmNN together with Arm NN by adding the following options to your CMake command: </p><div class="fragment"><div class="line">-DBUILD_PYTHON_SRC=1</div>
</div><!-- fragment --><p>This will build the source package. Current project headers and build libraries will be used, so there is no need to provide them.</p>
<p>SWIG is required to generate the wrappers. If CMake did not find the executable during the configure step or it has found an older version, you may provide it manually: </p><div class="fragment"><div class="line">-DSWIG_EXECUTABLE=<path_to_swig_executable></div>
</div><!-- fragment --><p>After the build finishes, you will find the python packages in <code><build_folder>/python/pyarmnn/dist</code>.</p>
<h1>PyArmNN installation</h1>
<p>PyArmNN can be distributed as a source package or a binary package (wheel).</p>
<p>Binary package is platform dependent, the name of the package will indicate the platform it was built for, e.g.:</p>
<ul>
<li>Linux x86 64bit machine: pyarmnn-33.1.0-cp36-cp36m-*linux_x86_64*.whl</li>
<li>Linux Aarch 64 bit machine: pyarmnn-33.1.0-cp36-cp36m-*linux_aarch64*.whl</li>
</ul>
<p>The source package is platform independent but installation involves compilation of Arm NN python extension. You will need to have g++ compatible with C++ 14 standard and a python development library installed on the build machine.</p>
<p>Both of them, source and binary package, require the Arm NN library to be present on the target/build machine.</p>
<p>It is strongly suggested to work within a python virtual environment. The further steps assume that the virtual environment was created and activated before running PyArmNN installation commands.</p>
<p>PyArmNN also depends on the NumPy python library. It will be automatically downloaded and installed alongside PyArmNN. If your machine does not have access to Python pip repositories you might need to install NumPy in advance by following public instructions: <a href="https://scipy.org/install.html">https://scipy.org/install.html</a></p>
<h2>Installing from source package</h2>
<p>Installing from source is the most reliable way.</p>
<p>While installing from sources, you have the freedom of choosing Arm NN libraries location. Set environment variables <em>ARMNN_LIB</em> and <em>ARMNN_INCLUDE</em> to point to Arm NN libraries and headers. If you want to use system default locations, just set <em>ARMNN_INCLUDE</em> to point to Arm NN headers. Additionally, <em>LD_LIBRARY_PATH</em> may need to be updated to the Arm NN libraries location due to dependencies of the same shared library files being 'not found'.</p>
<div class="fragment"><div class="line">$ export ARMNN_LIB=/path/to/libs</div>
<div class="line">$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/libs</div>
<div class="line">$ export ARMNN_INCLUDE=/full/path/to/armnn/include:/full/path/to/armnn/profiling/common/include</div>
</div><!-- fragment --><p>Install PyArmNN as follows: </p><div class="fragment"><div class="line">$ pip install /path/to/armnn/python/pyarmnn</div>
</div><!-- fragment --><p>If PyArmNN installation script fails to find Arm NN libraries it will raise an error like this</p>
<p>‘RuntimeError: ArmNN library was not found in (’/usr/lib/gcc/aarch64-linux-gnu/8/', <...> ,'/lib/', '/usr/lib/'). Please install ArmNN to one of the standard locations or set correct ARMNN_INCLUDE and ARMNN_LIB env variables.`</p>
<p>You can now verify that PyArmNN library is installed and check PyArmNN version using: </p><div class="fragment"><div class="line">$ pip show pyarmnn</div>
</div><!-- fragment --><p>You can also verify it by running the following and getting output similar to below: </p><div class="fragment"><div class="line">$ python -c "import pyarmnn as ann;print(ann.GetVersion())"</div>
<div class="line">'33.1.0'</div>
</div><!-- fragment --><h2>Installing PyArmNN while using ArmNN prebuilt binaries</h2>
<p>If you wish to use ArmNN prebuilt binaries from our release page, you will need to generate the PyArmNN SWIG wrappers.</p>
<p>Again setup your environmental variables along with a virtual environment:</p>
<div class="fragment"><div class="line">$ python -m venv env</div>
<div class="line">$ source env/bin/activate</div>
<div class="line"> </div>
<div class="line">$ export ARMNN_LIB=/path/to/libs</div>
<div class="line">$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/libs</div>
<div class="line">$ export ARMNN_INCLUDE=/full/path/to/armnn/include:/full/path/to/armnn/profiling/common/include</div>
</div><!-- fragment --><p>Then generate the SWIG wrappers: </p><div class="fragment"><div class="line">$ cd armnn/python/pyarmnn/</div>
<div class="line">$ python swig_generate.py -v</div>
</div><!-- fragment --><p>Then install PyArmNN: </p><div class="fragment"><div class="line"># From directory armnn/python/pyarmnn/</div>
<div class="line">$ pip install .</div>
</div><!-- fragment --><p>As above, you can verify that PyArmNN library is installed and check PyArmNN version using: </p><div class="fragment"><div class="line">$ pip show pyarmnn</div>
</div><!-- fragment --><p>You can also verify it by running the following and getting output similar to below: </p><div class="fragment"><div class="line">$ python -c "import pyarmnn as ann;print(ann.GetVersion())"</div>
<div class="line">'33.1.0'</div>
</div><!-- fragment --><h1>PyArmNN API overview</h1>
<h4>Getting started</h4>
<p>The easiest way to begin using PyArmNN is by using the Parsers. We will demonstrate how to use them below:</p>
<p>Create a parser object and load your model file. </p><div class="fragment"><div class="line">import pyarmnn as ann</div>
<div class="line">import imageio</div>
<div class="line"> </div>
<div class="line"># An ONNX parser also exists.</div>
<div class="line">parser = ann.ITfLiteParser()</div>
<div class="line">network = parser.CreateNetworkFromBinaryFile('./model.tflite')</div>
</div><!-- fragment --><p>Get the input binding information by using the name of the input layer. </p><div class="fragment"><div class="line">input_binding_info = parser.GetNetworkInputBindingInfo(0, 'model/input')</div>
<div class="line"> </div>
<div class="line"># Create a runtime object that will perform inference.</div>
<div class="line">options = ann.CreationOptions()</div>
<div class="line">runtime = ann.IRuntime(options)</div>
</div><!-- fragment --><p>Choose preferred backends for execution and optimize the network. </p><div class="fragment"><div class="line"># Backend choices earlier in the list have higher preference.</div>
<div class="line">preferredBackends = [ann.BackendId('CpuAcc'), ann.BackendId('CpuRef')]</div>
<div class="line">opt_network, messages = ann.Optimize(network, preferredBackends, runtime.GetDeviceSpec(), ann.OptimizerOptions())</div>
<div class="line"> </div>
<div class="line"># Load the optimized network into the runtime.</div>
<div class="line">net_id, _ = runtime.LoadNetwork(opt_network)</div>
</div><!-- fragment --><p>Make workload tensors using input and output binding information. </p><div class="fragment"><div class="line"># Load an image and create an inputTensor for inference.</div>
<div class="line">img = imageio.imread('./image.png')</div>
<div class="line">input_tensors = ann.make_input_tensors([input_binding_info], [img])</div>
<div class="line"> </div>
<div class="line"># Get output binding information for an output layer by using the layer name.</div>
<div class="line">output_binding_info = parser.GetNetworkOutputBindingInfo(0, 'model/output')</div>
<div class="line">output_tensors = ann.make_output_tensors([output_binding_info])</div>
</div><!-- fragment --><p>Perform inference and get the results back into a numpy array. </p><div class="fragment"><div class="line">runtime.EnqueueWorkload(0, input_tensors, output_tensors)</div>
<div class="line"> </div>
<div class="line">results = ann.workload_tensors_to_ndarray(output_tensors)</div>
<div class="line">print(results)</div>
</div><!-- fragment --><h4>Examples</h4>
<p>To further explore PyArmNN API there are several examples provided in the <code>/examples</code> folder for you to explore.</p>
<h5>Image Classification</h5>
<p>This sample application performs image classification on an image and outputs the <em>Top N</em> results, listing the classes and probabilities associated with the classified image. All resources are downloaded during execution, so if you do not have access to the internet, you may need to download these manually.</p>
<p>Sample scripts are provided for performing image classification with TFLite and ONNX models with <code>tflite_mobilenetv1_quantized.py</code> and <code>onnx_mobilenetv2.py</code>.</p>
<h5>Object Detection</h5>
<p>This sample application guides the user and shows how to perform object detection using PyArmNN API. By taking a model and video file or camera feed as input, and running inference on each frame, we are able to interpret the output to draw bounding boxes around detected objects and overlay the corresponding labels and confidence scores.</p>
<p>Sample scripts are provided for performing object detection from video file and video stream with <code>run_video_file.py</code> and <code>run_video_stream.py</code>.</p>
<h2>Tox for automation</h2>
<p>To make things easier <em>tox</em> is available for automating individual tasks or running multiple commands at once such as generating wrappers, running unit tests using multiple python versions or generating documentation. To run it use:</p>
<div class="fragment"><div class="line">$ tox <task_name></div>
</div><!-- fragment --><p>See <em>tox.ini</em> for the list of tasks. You may also modify it for your own purposes. To dive deeper into tox read through <a href="https://tox.readthedocs.io/en/latest/">https://tox.readthedocs.io/en/latest/</a></p>
<h2>Running unit-tests</h2>
<p>Download resources required to run unit tests by executing the script in the scripts folder:</p>
<div class="fragment"><div class="line">$ python ./scripts/download_test_resources.py</div>
</div><!-- fragment --><p>The script will download an archive from the Linaro server and extract it. A folder <code>test/testdata/shared</code> will be created. Execute <code>pytest</code> from the project root dir: </p><div class="fragment"><div class="line">$ python -m pytest test/ -v</div>
</div><!-- fragment --><p>or run tox which will do both: </p><div class="fragment"><div class="line">$ tox</div>
</div><!-- fragment --> </div></div><!-- contents -->
</div><!-- PageDoc -->
</div><!-- doc-content -->
<!-- start footer part -->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
<ul>
<li class="navelem"><a class="el" href="swtools.html">Software Components</a></li>
<li class="footer">Generated on Wed Feb 14 2024 16:36:20 for Arm NN by
<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.17 </li>
</ul>
</div>
</body>
</html>
|
