1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
|
//
// This confidential and proprietary software may be used only as
// authorised by a licensing agreement from ARM Limited
// (C) COPYRIGHT 2020-2021 ARM Limited
// ALL RIGHTS RESERVED
// The entire notice above must be reproduced on all authorised
// copies and copies may only be made to the extent permitted
// by a licensing agreement from ARM Limited.
=== Activation Functions
==== CLAMP
Clamp to an arbitrary minimum and maximum value.
Maximum and minimum values are specified as values in the range of the input type.
No zero point subtraction is done to the values, thus to clamp to the zero point value, the zero point itself should be supplied as the minimum value.
include::{generated}/operators/CLAMP.adoc[]
*Operation Function:*
[source,c++]
----
ERROR_IF(max_val < min_val);
for_each(index in shape) {
in_out_t value = tensor_read<in_out_t>(input, shape, index);
value = apply_clip<in_out_t>(value, min_val, max_val);
tensor_write<in_out_t>(output, shape, index, value);
}
----
==== SIGMOID
Sigmoid function: output = 1 / (1 + exp(-input))
For quantized integer data types, the TABLE operator should be used instead with
the following definition.
The sigmoid table has 513 entries each of 16-bit precision and covering the input range -16.0 to +16.0 in steps of 1/16.
[source,c++]
----
int16_t sigmoid_reference(int16_t x) { // input x range is -256 to + 256 inclusive
F64 v = (double)x / (double)16;
v = 1.0/(1.0 + exp(-v));
return round_to_nearest_int(32768.0 * v);
}
generate_lookup_table(&sigmoid_table, &sigmoid_reference);
----
include::{generated}/operators/SIGMOID.adoc[]
==== TANH
Parameterized hyperbolic tangent.
For quantized integer data types, the TABLE operator should be used instead with
the following definition.
The tanh_table has 513 entries each of 16-bit precision and covering the input range -8.0 to +8.0 in steps of 1/32. The table is specified by:
[source,c++]
----
int16_t tanh_reference(int16_t x) { // input x range is -256 to +256 inclusive
F64 v = (double)x/(double)32;
v = exp(-2.0*v);
v = (1.0-v)/(1.0+v);
return round_to_nearest_int(32768.0 * v);
}
generate_lookup_table(&tanh_table, &tanh_reference);
----
include::{generated}/operators/TANH.adoc[]
|
