1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
|
//
// This confidential and proprietary software may be used only as
// authorised by a licensing agreement from ARM Limited
// (C) COPYRIGHT 2020-2021,2024 ARM Limited
// ALL RIGHTS RESERVED
// The entire notice above must be reproduced on all authorised
// copies and copies may only be made to the extent permitted
// by a licensing agreement from ARM Limited.
== Operators
=== Operator Arguments
Operators process input arguments to produce output arguments.
Their behavior can be configured using attribute arguments.
Arguments may have one of the following types:
* `tensor_t<element_type>`, abbreviated `T<element_type>`, represents a tensor whose elements are of type `element_type` where `element_type` can be any of the data types supported in TOSA.
* `tensor_list_t` represents a list of tensors. When lists are homogeneous, i.e. contain tensors of the same type, their type is further qualified as follows: `tensor_list_t<T<element_type>>`.
* `tosa_graph_t` represents a TOSA graph (see <<operator-graphs>>).
Arguments belong to one of three categories: Input, Output, or Attribute. The category to which an argument belongs further constrains its type:
* An Input argument must be a tensor or a list of tensors used to provide the data read by the operation.
* An Output argument must be a tensor or a list of tensors into which the data produced by the operation is written.
* An Attribute argument is constant, i.e. its value is known at compilation time. It may have any data type supported by TOSA.
[[operator-graphs]]
=== Operator Graphs
A TOSA graph is a collection of TOSA operators where:
* The output of an operator in the graph may be connected to one or more inputs of other operators in the graph
* When an output is connected to an input the tensor list shapes must match
* The attributes of the operators are defined and considered part of the graph
* The attributes must be in the valid range permitted for the operator
* The tensor dimensions must be in the valid range permitted for the operator
Some operators, such as control flow operators, take a graph of other operators as an attribute. The type `tosa_graph_t` will denote a graph of operators and the following functions define the tensor shape list for the graph input and outputs:
[source,c++]
----
shape_list_t tosa_input_shape(tosa_graph_t graph);
shape_list_t tosa_output_shape(tosa_graph_t graph);
----
Similarly the type tensor_list_t will be used for a list of tensors and the following function returns the shape of a tensor list:
[source,c++]
----
shape_list_t tensor_list_shape(tosa_list_t tensor_list);
----
The following function denotes the execution of a TOSA graph within a TOSA context,
on an input tensor list to produce an output tensor list. A TOSA context, represented
by `tosa_context_t` provides the environment in which a TOSA graph is executed.
Any side-effects that result from the execution of a graph within a context are not
observable by graphs executing in a different context. Operators are executed in an
implementation-defined order that must be a topological ordering of the TOSA graph.
[source,c++]
----
tosa_execute_graph(tosa_context_t context, tosa_graph_t graph, tosa_list_t input_list, tosa_list_t output_list, tosa_level_t level) {
ERROR_IF(tensor_list_shape(input_list) != tosa_input_shape(graph));
ERROR_IF(tensor_list_shape(output_list) != tosa_output_shape(graph));
// Declare the global list for storing persistent variable tensors across multiple graphs
if (!variable_tensors) {
variable_tensors = list<tensor_t>();
} else { // Clear the "seen flag"
for (tensor_t var_tensor in variable_tensors) {
var_tensor.seen = false;
}
}
for_each(operator in graph order) {
ERROR_IF(operator input tensors do not meet requirement of operator Arguments inputs)
ERROR_IF(operator attributes do not meet requirement of operator Arguments attributes)
ERROR_IF(operator output tensors do not meet requirement of operator Arguments outputs)
ERROR_IF(operator data types do not meet requirement of operator Supported Data Types)
// Execute the operator as defined by the operation function pseduo-code
tosa_execute_operator(context, operator, level);
}
}
----
include::tensor_ops.adoc[]
include::activation_funcs.adoc[]
include::ewise_binary.adoc[]
include::ewise_unary.adoc[]
include::ewise_ternary.adoc[]
include::comparison.adoc[]
include::reduction.adoc[]
include::data_layout.adoc[]
include::scatter_gather.adoc[]
include::image.adoc[]
include::type_conversion.adoc[]
include::data_nodes.adoc[]
include::custom.adoc[]
include::control_flow.adoc[]
include::variable.adoc[]
include::shape.adoc[]
|
