1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
|
//
// This confidential and proprietary software may be used only as
// authorised by a licensing agreement from ARM Limited
// (C) COPYRIGHT 2020-2021 ARM Limited
// ALL RIGHTS RESERVED
// The entire notice above must be reproduced on all authorised
// copies and copies may only be made to the extent permitted
// by a licensing agreement from ARM Limited.
=== Activation Functions
==== CLAMP
Clamp to an arbitrary minimum and maximum value.
Maximum and minimum values are specified as values in the range of the input type.
No zero point subtraction is done to the values, thus to clamp to the zero point value, the zero point itself should be supplied as the minimum value.
*Arguments:*
|===
|Argument|Type|Name|Shape|Description
|Input|in_out_t*|Input|shape|Input tensor
|Attribute|in_out_t|min_val|-|minimum clip value
|Attribute|in_out_t|max_val|-|maximum clip value
|Output|in_out_t*|Output|shape|Output tensor of same type and shape as input
|===
*Operation Function:*
[source,c++]
----
ERROR_IF(max_val < min_val);
for_each(index in shape) {
in_out_t value = tensor_read<in_out_t>(input, shape, index);
value = apply_clip<in_out_t>(value, min_val, max_val);
tensor_write<in_out_t>(output, shape, index, value);
}
----
*Supported Data Types:*
|===
|Profile|Mode|in_out_t
|Any|signed 8|int8_t
|Any|signed 16|int16_t
|MI, MT|fp16|fp16_t
|MI, MT|bf16|bf16_t
|MI, MT|fp32|fp32_t
|===
==== SIGMOID
Sigmoid function: output = 1 / (1 + exp(-input))
For quantized integer data types, the TABLE operator should be used instead with
the following definition.
The sigmoid table has 513 entries each of 16-bit precision and covering the input range -16.0 to +16.0 in steps of 1/16.
[source,c++]
----
int16_t sigmoid_reference(int16_t x) { // input x range is -256 to + 256 inclusive
F64 v = (double)x / (double)16;
v = 1.0/(1.0 + exp(-v));
return round_to_nearest_int(32768.0 * v);
}
generate_lookup_table(&sigmoid_table, &sigmoid_reference);
----
*Arguments:*
|===
|Argument|Type|Name|Shape|Description
|Input|in_out_t*|Input|shape|Input tensor
|Output|in_out_t*|Output|shape|Output tensor of same type and shape as input
|===
*Supported Data Types:*
|===
|Profile|Mode|in_out_t
|MI, MT|fp16|fp16_t
|MI, MT|bf16|bf16_t
|MI, MT|fp32|fp32_t
|===
==== TANH
Parameterized hyperbolic tangent.
For quantized integer data types, the TABLE operator should be used instead with
the following definition.
The tanh_table has 513 entries each of 16-bit precision and covering the input range -8.0 to +8.0 in steps of 1/32. The table is specified by:
[source,c++]
----
int16_t tanh_reference(int16_t x) { // input x range is -256 to +256 inclusive
F64 v = (double)x/(double)32;
v = exp(-2.0*v);
v = (1.0-v)/(1.0+v);
return round_to_nearest_int(32768.0 * v);
}
generate_lookup_table(&tanh_table, &tanh_reference);
----
*Arguments:*
|===
|Argument|Type|Name|Shape|Description
|Input|in_out_t*|Input|shape|Input tensor
|Output|in_out_t*|Output|shape|Output tensor of same type and shape as input
|===
*Supported Data Types:*
|===
|Profile|Mode|in_out_t
|MI, MT|fp16|fp16_t
|MI, MT|bf16|bf16_t
|MI, MT|fp32|fp32_t
|===
|
