1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
|
/// Copyright (c) 2022-2023 ARM Limited and Contributors. All rights reserved.
///
/// SPDX-License-Identifier: MIT
///
namespace armnn
{
/**
@page delegate TfLite Delegate
@section delegateintro About the delegate
'armnnDelegate' is a library for accelerating certain TensorFlow Lite (TfLite) operators on Arm hardware. It can be
integrated in TfLite using its delegation mechanism. TfLite will then delegate the execution of operators supported by
Arm NN to Arm NN.
The main difference to our @ref S6_tf_lite_parser is the amount of operators you can run with it. If none of the active
backends support an operation in your model you won't be able to execute it with our parser. In contrast to that, TfLite
only delegates operations to the armnnDelegate if it does support them and otherwise executes them itself. In other
words, every TfLite model can be executed and every operation in your model that we can accelerate will be accelerated.
That is the reason why the armnnDelegate is our recommended way to accelerate TfLite models.
If you need help building the armnnDelegate, please take a look at our [build guide](delegate/BuildGuideNative.md).
@section delegatesupport Supported Operators
This reference guide provides a list of TensorFlow Lite operators the Arm NN SDK currently supports.
@section delegatefullysupported Fully supported
The Arm NN SDK TensorFlow Lite delegate currently supports the following operators:
- ABS
- ADD
- ARG_MAX
- ARG_MIN
- AVERAGE_POOL_2D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE
- AVERAGE_POOL_3D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, SIGN_BIT, TANH, NONE
- BATCH_MATMUL
- BATCH_TO_SPACE_ND
- BROADCAST_TO
- CAST
- CEIL
- CONCATENATION, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE
- CONV_2D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE
- CONV_3D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE
- DEPTH_TO_SPACE
- DEPTHWISE_CONV_2D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE
- DEQUANTIZE
- DIV
- EQUAL
- ELU
- EXP
- EXPAND_DIMS
- FILL
- FLOOR
- FLOOR_DIV
- FULLY_CONNECTED, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE
- GATHER
- GATHER_ND
- GELU
- GREATER
- GREATER_EQUAL
- HARD_SWISH
- L2_NORMALIZATION
- L2_POOL_2D
- LESS
- LESS_EQUAL
- LOCAL_RESPONSE_NORMALIZATION
- LOG
- LOGICAL_AND
- LOGICAL_NOT
- LOGICAL_OR
- LOGISTIC
- LOG_SOFTMAX
- LSTM
- MAXIMUM
- MAX_POOL_2D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, TANH, NONE
- MAX_POOL_3D, Supported Fused Activation: RELU, RELU6, RELU_N1_TO_1, SIGMOID, SIGN_BIT, TANH, NONE
- MEAN
- MINIMUM
- MIRROR_PAD
- MUL
- NEG
- NOT_EQUAL
- PACK
- PAD
- PADV2
- POW
- PRELU
- QUANTIZE
- RANK
- REDUCE_MAX
- REDUCE_MIN
- REDUCE_PROD
- RELU
- RELU6
- RELU_N1_TO_1
- RESHAPE
- RESIZE_BILINEAR
- RESIZE_NEAREST_NEIGHBOR
- REVERSEV2
- RSQRT
- SCATTERND
- SHAPE
- SIN
- SOFTMAX
- SPACE_TO_BATCH_ND
- SPACE_TO_DEPTH
- SPLIT
- SPLIT_V
- SQRT
- SQUARE_DIFFERENCE
- SQUEEZE
- SLICE
- STRIDED_SLICE
- SUB
- SUM
- TANH
- TILE
- TRANSPOSE
- TRANSPOSE_CONV
- UNIDIRECTIONAL_SEQUENCE_LSTM
- UNPACK
More machine learning operators will be supported in future releases.
@section delegateoptions Delegate Options
The general list of runtime options are described in @subpage runtimeoptions
@subsection opaquedelegateoptions Opaque Delegate Options
In Opaque Delegate, delegate options are passed via ArmNNSettings which is a FlatBuffer of tflite::TFLiteSettings.
Arm NN Settings | Possible Values | Description
:--------------- ------------- | :----- | :----------
backends | ["GpuAcc"/"CpuAcc"] | A comma separated list without whitespaces of backends which should be used for execution. Falls back to next backend in list if previous does not provide support for operation.
fastmath | [true/false] | Allows the use of optimisation techniques e.g. Winograd that will reduce execution time with the possibility of a drop in accuracy.
additional_parameters | JSON string of additional Arm NN delegate options | JSON string of additional Arm NN delegate options. The general list of runtime options are described in [Runtime options](\ref runtimeoptions).
**/
}
|
