1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
|
//
// This confidential and proprietary software may be used only as
// authorised by a licensing agreement from ARM Limited
// (C) COPYRIGHT 2020-2021 ARM Limited
// ALL RIGHTS RESERVED
// The entire notice above must be reproduced on all authorised
// copies and copies may only be made to the extent permitted
// by a licensing agreement from ARM Limited.
=== Activation Functions
==== CLAMP
Clamp to an arbitrary minimum and maximum value. Note that the maximum and minimum values are specified as signed quantized values, no scaling happens before or after this operation.
*Arguments:*
|===
|Argument|Type|Name|Shape|Description
|Input|in_t*|Input|shape|Input tensor
|Attribute|in_t|min_val|-|minimum clip value
|Attribute|in_t|max_val|-|maximum clip value
|Output|in_t*|Output|shape|Output tensor of same type and shape as input
|===
*Operation Function:*
[source,c++]
----
ERROR_IF(max_val < min_val);
for_each(index in shape) {
acc_t value = tensor_read<in_t>(input, shape, index);
acc = (in_t)apply_clip<acc_t>(value, min_val, max_val);
tensor_write<in_t>(output, shape, index, acc);
}
----
*Supported Data Types:*
|===
|Profile|Mode|in_t|acc_t
|Any|signed 8|int8_t|int16_t
|Any|signed 16|int16_t|int16_t
|MI, MT|floating-point|float_t|float_t
|===
==== SIGMOID
Sigmoid function: output = 1 / (1 + exp(-input))
For quantized integer data types, the TABLE operator should be used instead with
the following definition.
The sigmoid table has 513 entries each of 16-bit precision and covering the input range -16.0 to +16.0 in steps of 1/16.
[source,c++]
----
int sigmoid_reference(int x) {|// input x range is -256 to + 256 inclusive
F64 v = (double)x / (double)16;
v = 1.0/(1.0 + exp(-v));
return round_to_nearest_int(32768.0 * v);
}
generate_lookup_table(&sigmoid_table, &sigmoid_reference);
----
*Arguments:*
|===
|Argument|Type|Name|Shape|Description
|Input|in_t*|Input|shape|Input tensor
|Output|in_t*|Output|shape|Output tensor of same type and shape as input
|===
*Supported Data Types:*
|===
|Profile|Mode|in_t
|MI, MT|floating-point|float_t
|===
==== TANH
Parameterized hyperbolic tangent.
For quantized integer data types, the TABLE operator should be used instead with
the following definition.
The tanh_table has 513 entries each of 16-bit precision and covering the input range -8.0 to +8.0 in steps of 1/32. The table is specified by:
[source,c++]
----
int tanh_reference(int x) { // input x range is -256 to +256 inclusive
F64 v = (double)x/(double)32;
v = exp(-2.0*v);
v = (1.0-v)/(1.0+v);
return round_to_nearest_int(32768.0 * v);
}
generate_lookup_table(&tanh_table, &tanh_reference);
----
*Arguments:*
|===
|Argument|Type|Name|Shape|Description
|Input|in_t*|Input|shape|Input tensor
|Output|in_t*|Output|shape|Output tensor of same type and shape as input
|===
*Supported Data Types:*
|===
|Profile|Mode|in_t
|MI, MT|floating-point|float_t
|===
|
