1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
|
//
// This confidential and proprietary software may be used only as
// authorised by a licensing agreement from ARM Limited
// (C) COPYRIGHT 2020 ARM Limited
// ALL RIGHTS RESERVED
// The entire notice above must be reproduced on all authorised
// copies and copies may only be made to the extent permitted
// by a licensing agreement from ARM Limited.
=== Activation Functions
==== CLAMP
Clamp to an arbitrary minimum and maximum value. Note that the maximum and minimum values are specified as signed quantized values, no scaling happens before or after this operation.
*Arguments:*
|===
|Argument|Type|Name|Shape|Description
|Input|in_t*|Input|shape|Input tensor with rank from 1 to 4
|Attribute|in_t|min_val|-|minimum clip value
|Attribute|in_t|max_val|-|maximum clip value
|Output|out_t*|Output|shape|Output tensor of same type and shape as input
|===
*Operation Function:*
....
assert(rank(shape)<=4)
for_each (index in shape) {
value = tensor_read<in_t>(input, shape, index)
acc = apply_clip(value, min_val, max_val)
tensor_write<out_t>(output, shape, index, acc)
}
....
*Supported Data Types:*
|===
|Profile|Mode|in_t|out_t
|Any|signed 8|aint8 |aint8
|Any|signed 16|int16|int16
|MI, MT|float|float|float
|===
==== RELUN
ReLU with a scalar maximum value.
*Arguments:*
|===
|Argument|Type|Name|Shape|Description
|Input|in_t*|Input|shape|Input tensor with rank from 1 to 4
|Attribute|in_t|max_val|-|maximum clip value
|Output|out_t*|Output|shape|Output tensor of same type and shape as input
|===
*Operation Function:*
[source,c]
----
for_each (index in shape) {
in_t value = tensor_read<in_t>(input, shape, index)
acc = apply_clip<in_t>(value, 0, max_val)
tensor_write<in_t>(output, shape, index, acc)
}
----
*Supported Data Types:*
|===
|Profile|Mode|in_t
|Any|signed 32|int32
|MI, MT|float|float
|===
==== SIGMOID
Sigmoid function: output = 1 / (1 + exp(-input))
For quantized integer data types, the TABLE operator should be used instead with
the following definition.
The sigmoid table has 513 entries each of 16-bit precision and covering the input range -16.0 to +16.0 in steps of 1/16.
[source,c]
....
int sigmoid_reference(int x) {|// input x range is -256 to + 256 inclusive
F64 v = (double)x/(double)16;
v = 1.0/(1.0+exp(-v));
return round_to_nearest(32768.0 * v);
}
generate_lookup_table(&sigmoid_table, &sigmoid_reference);
....
*Arguments:*
|===
|Argument|Type|Name|Shape|Description
|Input|in_t*|Input|shape|Input tensor with rank from 1 to 4
|Output|out_t*|Output|shape|Output tensor of same type and shape as input
|===
*Supported Data Types:*
|===
|Profile|Mode|in_t|out_t
|MI, MT|float|float|float
|===
==== TANH
Parameterized hyperbolic tangent.
For quantized integer data types, the TABLE operator should be used instead with
the following definition.
The tanh_table has 513 entries each of 16-bit precision and covering the input range -8.0 to +8.0 in steps of 1/32. The table is specified by:
[source,c]
----
int tanh_reference(int x) { // input x range is -256 to +256 inclusive
F64 v = (double)x/(double)32;
v = exp(-2.0*v);
v = (1.0-v)/(1.0+v);
return round_to_nearest(32768.0 * v);
}
generate_lookup_table(&tanh_table, &tanh_reference);
----
*Arguments:*
|===
|Argument|Type|Name|Shape|Description
|Input|in_t*|Input|shape|Input tensor with rank from 1 to 4
|Output|out_t*|Output|shape|Output tensor of same type and shape as input
|===
*Supported Data Types:*
|===
|Profile|Mode|in_t|out_t
|MI, MT|float|float|float
|===
|
