1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
|
<!-- Copyright (c) 2020 ARM Limited. -->
<!-- -->
<!-- SPDX-License-Identifier: MIT -->
<!-- -->
<!-- HTML header for doxygen 1.8.13-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.8.17"/>
<meta name="robots" content="NOINDEX, NOFOLLOW" />
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>ArmNN: Delegate Build Guide</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtreedata.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
extensions: ["tex2jax.js"],
jax: ["input/TeX","output/HTML-CSS"],
});
</script>
<script type="text/javascript" async="async" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
<link href="stylesheet.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<img alt="ArmNN" src="Arm_NN_horizontal_blue.png" style="max-width: 10rem; margin-top: .5rem; margin-left 10px"/>
<td style="padding-left: 0.5em;">
<div id="projectname">
 <span id="projectnumber">23.05</span>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.8.17 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
var searchBox = new SearchBox("searchBox", "search",false,'Search');
/* @license-end */
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
$(function() {
initMenu('',true,false,'search.php','Search');
$(document).ready(function() { init_search(); });
});
/* @license-end */</script>
<div id="main-nav"></div>
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
$(document).ready(function(){initNavTree('md_delegate__build_guide_native.xhtml',''); initResizable(); });
/* @license-end */
</script>
<div id="doc-content">
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>
<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0"
name="MSearchResults" id="MSearchResults">
</iframe>
</div>
<div class="PageDoc"><div class="header">
<div class="headertitle">
<div class="title">Delegate Build Guide </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><p>This guide assumes that Arm NN has been built with the Arm NN TF Lite Delegate with the ../build-tool/README.md "Arm NN Build Tool".<br />
The Arm NN TF Lite Delegate can also be obtained from downloading the ../README.md "Pre-Built Binaries on the GitHub homepage".</p>
<p><b>Table of Contents:</b></p><ul>
<li><a href="#running-delegateunittests">Running DelegateUnitTests</a></li>
<li><a href="#run-the-tflite-model-benchmark-tool">Run the TF Lite Benchmark Tool</a><ul>
<li><a href="#download-the-tflite-model-benchmark-tool">Download the TFLite Model Benchmark Tool</a></li>
<li><a href="#execute-the-benchmarking-tool-with-the-arm-nn-tf-lite-delegate">Execute the benchmarking tool with the Arm NN TF Lite Delegate</a></li>
</ul>
</li>
<li><a href="#integrate-the-arm-nn-tflite-delegate-into-your-project">Integrate the Arm NN TfLite Delegate into your project</a></li>
</ul>
<h1>Running DelegateUnitTests</h1>
<p>To ensure that the build was successful you can run the unit tests for the delegate that can be found in the build directory for the delegate. <a href="https://github.com/onqtam/doctest">Doctest</a> was used to create those tests. Using test filters you can filter out tests that your build is not configured for. In this case, we run all test suites that have <code>CpuAcc</code> in their name. </p><div class="fragment"><div class="line">cd <PATH_TO_ARMNN_BUILD_DIRECTORY>/delegate/build</div>
<div class="line">./DelegateUnitTests --test-suite=*CpuAcc*</div>
</div><!-- fragment --><p>If you have built for Gpu acceleration as well you might want to change your test-suite filter: </p><div class="fragment"><div class="line">./DelegateUnitTests --test-suite=*CpuAcc*,*GpuAcc*</div>
</div><!-- fragment --><h1>Run the TFLite Model Benchmark Tool</h1>
<p>The <a href="https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark">TFLite Model Benchmark</a> Tool has a useful command line interface to test the TF Lite Delegate. We can use this to demonstrate the use of the Arm NN TF Lite Delegate and its options.</p>
<p>Some examples of this can be viewed in this <a href="https://www.youtube.com/watch?v=NResQ1kbm-M&t=920s">YouTube demonstration</a>.</p>
<h2>Download the TFLite Model Benchmark Tool</h2>
<p>Binary builds of the benchmarking tool for various platforms are available <a href="https://www.tensorflow.org/lite/performance/measurement#native_benchmark_binary">here</a>. In this example I will target an aarch64 Linux environment. I will also download a sample uint8 tflite model from the <a href="https://github.com/ARM-software/ML-zoo">Arm ML Model Zoo</a>.</p>
<div class="fragment"><div class="line">mkdir $BASEDIR/benchmarking</div>
<div class="line">cd $BASEDIR/benchmarking</div>
<div class="line"># Get the benchmarking binary.</div>
<div class="line">wget https://storage.googleapis.com/tensorflow-nightly-public/prod/tensorflow/release/lite/tools/nightly/latest/linux_aarch64_benchmark_model -O benchmark_model</div>
<div class="line"># Make it executable.</div>
<div class="line">chmod +x benchmark_model</div>
<div class="line"># and a sample model from model zoo.</div>
<div class="line">wget https://github.com/ARM-software/ML-zoo/blob/master/models/image_classification/mobilenet_v2_1.0_224/tflite_uint8/mobilenet_v2_1.0_224_quantized_1_default_1.tflite?raw=true -O mobilenet_v2_1.0_224_quantized_1_default_1.tflite</div>
</div><!-- fragment --><h2>Execute the benchmarking tool with the Arm NN TF Lite Delegate</h2>
<p>You are already at $BASEDIR/benchmarking from the previous stage. </p><div class="fragment"><div class="line">LD_LIBRARY_PATH=<PATH_TO_ARMNN_BUILD_DIRECTORY> ./benchmark_model --graph=mobilenet_v2_1.0_224_quantized_1_default_1.tflite --external_delegate_path="<PATH_TO_ARMNN_BUILD_DIRECTORY>/delegate/libarmnnDelegate.so" --external_delegate_options="backends:CpuAcc;logging-severity:info"</div>
</div><!-- fragment --><p>The "external_delegate_options" here are specific to the Arm NN delegate. They are used to specify a target Arm NN backend or to enable/disable various options in Arm NN. A full description can be found in the parameters of function tflite_plugin_create_delegate.</p>
<h1>Integrate the Arm NN TfLite Delegate into your project</h1>
<p>The delegate can be integrated into your c++ project by creating a TfLite Interpreter and instructing it to use the Arm NN delegate for the graph execution. This should look similar to the following code snippet. </p><div class="fragment"><div class="line">// Create TfLite Interpreter</div>
<div class="line">std::unique_ptr<Interpreter> armnnDelegateInterpreter;</div>
<div class="line">InterpreterBuilder(tfLiteModel, ::tflite::ops::builtin::BuiltinOpResolver())</div>
<div class="line"> (&armnnDelegateInterpreter)</div>
<div class="line"> </div>
<div class="line">// Create the Arm NN Delegate</div>
<div class="line">armnnDelegate::DelegateOptions delegateOptions(backends);</div>
<div class="line">std::unique_ptr<TfLiteDelegate, decltype(&armnnDelegate::TfLiteArmnnDelegateDelete)></div>
<div class="line"> theArmnnDelegate(armnnDelegate::TfLiteArmnnDelegateCreate(delegateOptions),</div>
<div class="line"> armnnDelegate::TfLiteArmnnDelegateDelete);</div>
<div class="line"> </div>
<div class="line">// Instruct the Interpreter to use the armnnDelegate</div>
<div class="line">armnnDelegateInterpreter->ModifyGraphWithDelegate(theArmnnDelegate.get());</div>
</div><!-- fragment --><p>For further information on using TfLite Delegates please visit the <a href="https://www.tensorflow.org/lite/guide">TensorFlow website</a>.</p>
<p>For more details of the kind of options you can pass to the Arm NN delegate please check the parameters of function tflite_plugin_create_delegate. </p>
</div></div><!-- contents -->
</div><!-- PageDoc -->
</div><!-- doc-content -->
<!-- start footer part -->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
<ul>
<li class="navelem"><a class="el" href="buildguides.xhtml">Build Guides</a></li>
<li class="footer">Generated on Thu May 18 2023 10:35:44 for ArmNN by
<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.17 </li>
</ul>
</div>
</body>
</html>
|
