From 5333c25baaf4fe136ed45edd6521dc7a2a5de851 Mon Sep 17 00:00:00 2001 From: Kevin Petit Date: Tue, 16 May 2023 09:08:48 +0100 Subject: Formalise the description of operator argument types - Standardise the terminology for operator arguments. Argument, Operand, and Parameter were used interchangeably. - Introduce a templatized tensor_t<> type for tensor arguments. Scalars are represented by rank-0 tensors. - Types can be checked with the XSD schema. Signed-off-by: Kevin Petit Change-Id: Ic57b9387950824e994c5e7f9ec1489c29159b974 --- chapters/custom.adoc | 6 +- chapters/introduction.adoc | 8 +- chapters/operators.adoc | 21 +- tools/genspec.py | 21 +- tools/tosa.py | 10 +- tosa.xml | 533 ++++++++++++++++++++++++--------------------- tosa.xsd | 25 ++- 7 files changed, 356 insertions(+), 268 deletions(-) diff --git a/chapters/custom.adoc b/chapters/custom.adoc index 4804e25..e748f38 100644 --- a/chapters/custom.adoc +++ b/chapters/custom.adoc @@ -12,9 +12,9 @@ Hardware implementing TOSA may choose to add additional custom operators that are not expressed in the existing TOSA operations. These operators are not expected to be portable across TOSA implementations. The input and output signatures must be expressed in the corresponding TOSA node. ==== CUSTOM -Input Operands: +Input arguments: -* Num input operands – Scalar number of input operands -* Num output operands – Scalar number of output operands +* Num input arguments – Scalar number of input arguments +* Num output arguments – Scalar number of output arguments * Operator code – untyped data consisting of the operator data * Affine transform description for each tensor diff --git a/chapters/introduction.adoc b/chapters/introduction.adoc index 9cfccb7..b369070 100644 --- a/chapters/introduction.adoc +++ b/chapters/introduction.adoc @@ -79,7 +79,7 @@ The following principles govern the selection of operators within TOSA. |Consistent sub-operation definition reduces the operator implementation cost. |P4 -|The valid input and output ranges for all operands shall be specified. +|The valid input and output ranges for all arguments shall be specified. |Ranges are required to make consistent (numerically agreeing) implementations possible. |P5 @@ -108,11 +108,11 @@ The following table summarizes the three profiles: === Levels -A TOSA level defines operator parameter ranges that an implementation shall support. +A TOSA level defines operator argument ranges that an implementation shall support. This is distinct from a profile that defines the operations and data-types supported. This version of the specification defines two TOSA levels: -* No level : allows the full range of parameters specified by the operations according to the operation data types. +* No level : allows the full range of arguments specified by the operations according to the operation data types. * Level 8K : ranges are expected to be sufficient for applications with frame sizes up to 8K. Later versions of the specification may define additional levels. @@ -522,7 +522,7 @@ To convert a network containing quantized tensors to TOSA, generate explicit RES This reduces quantized operations to purely integer operations. As an example, an ADD between two quantized tensors requires the integer values represent the same range. -The scale parameters for RESCALE can be calculated to ensure that the resulting tensors represent the same range. +The scale arguments for RESCALE can be calculated to ensure that the resulting tensors represent the same range. Then the ADD is performed, and a RESCALE can be used to ensure that the result is scaled properly. RESCALE provides support for per-tensor and per-channel scaling values to ensure compatibility with a range of possible quantization implementations. diff --git a/chapters/operators.adoc b/chapters/operators.adoc index 897ff17..3a4c831 100644 --- a/chapters/operators.adoc +++ b/chapters/operators.adoc @@ -9,14 +9,23 @@ == Operators -=== Operator Parameters +=== Operator Arguments -An operator processes input operands to produce output operands. An operator can have three types of parameters: +Operators process input arguments to produce output arguments. +Their behavior can be configured using attribute arguments. +Arguments may have one of the following types: -* An input operand. This must be a tensor or a list of tensors and data is read by the operation. -* An output operand. This must be a tensor or a list of tensors and data is written by the operation. -* An attribute. This is a parameter that is constant for a particular instance of the operator. It may have any data type supported by TOSA. It is expected to be set at compile time. +* `tensor_t`, abbreviated `T`, represents a tensor whose elements are of type `element_type` where `element_type` can be any of the data types supported in TOSA. +* `tensor_list_t` represents a list of tensors. When lists are homogeneous, i.e. contain tensors of the same type, their type is further qualified as follows: `tensor_list_t>`. +* `tosa_graph_t` represents a TOSA graph (see <>). +Arguments belong to one of three categories: Input, Output, or Attribute. The category to which an argument belongs further constrains its type: + +* An Input argument must be a tensor or a list of tensors used to provide the data read by the operation. +* An Output argument must be a tensor or a list of tensors into which the data produced by the operation is written. +* An Attribute argument is constant, i.e. its value is known at compilation time. It may have any data type supported by TOSA. + +[[operator-graphs]] === Operator Graphs A TOSA graph is a collection of TOSA operators where: @@ -27,7 +36,7 @@ A TOSA graph is a collection of TOSA operators where: * The attributes must be in the valid range permitted for the operator * The tensor dimensions must be in the valid range permitted for the operator -Some operators, such as control flow operators, take a graph of other operators as an attribute. The type tosa_graph_t will denote a graph of operators and the following functions define the tensor shape list for the graph input and outputs: +Some operators, such as control flow operators, take a graph of other operators as an attribute. The type `tosa_graph_t` will denote a graph of operators and the following functions define the tensor shape list for the graph input and outputs: [source,c++] ---- diff --git a/tools/genspec.py b/tools/genspec.py index c64f05b..11a3e72 100755 --- a/tools/genspec.py +++ b/tools/genspec.py @@ -18,10 +18,11 @@ class TOSASpecAsciidocGenerator: def generate_operator(self, op, file): file.write("\n*Arguments:*\n") - file.write("[cols='2,1,1,1,2,4']") + file.write("[cols='3,3,2,2,4,8']") file.write("\n|===\n") file.write("|Argument|Type|Name|Shape|Rank|Description\n\n") for arg in op.arguments: + # Argument cats = arg.categories if len(cats) > 1: cattext = "" @@ -33,6 +34,19 @@ class TOSASpecAsciidocGenerator: sep = " " else: cattext = cats[0].name.title() + + # Type + if arg.type == 'tensor_t': + argtype = 'T<{}>'.format(arg.tensor_element_type) + elif arg.type == 'tensor_list_t': + if arg.tensor_element_type == '-': + argtype = 'tensor_list_t' + else: + argtype = 'tensor_list_t>'.format(arg.tensor_element_type) + else: + argtype = arg.type + + # Rank if len(arg.rank) > 0: if (arg.rank[0] == arg.rank[1]): rank = f'{arg.rank[0]}' @@ -40,9 +54,12 @@ class TOSASpecAsciidocGenerator: rank = f'{arg.rank[0]} to {arg.rank[1]}' else: rank = "" + + # Format and write line file.write( - f"|{cattext}|{arg.type}|{arg.name}|{arg.shape}|{rank}|{arg.description}\n" + f"|{cattext}|{argtype}|{arg.name}|{arg.shape}|{rank}|{arg.description}\n" ) + file.write("|===\n") if op.typesupports: file.write("\n*Supported Data Types:*\n\n") diff --git a/tools/tosa.py b/tools/tosa.py index 218412f..d01d9c2 100644 --- a/tools/tosa.py +++ b/tools/tosa.py @@ -36,11 +36,12 @@ class TOSALevel: self.maximums = maximums class TOSAOperatorArgument: - def __init__(self, name, description, categories, ty, shape, levellimits, rank): + def __init__(self, name, description, categories, ty, elty, shape, levellimits, rank): self.name = name self.description = description self.categories = categories self.type = ty + self.tensor_element_type = elty self.shape = shape self.levellimits = levellimits self.rank = rank @@ -144,14 +145,15 @@ class TOSASpec: desc = arg.find("description").text.strip() argcats = [] argtype = arg.get("type") + argtelty = arg.get("tensor-element-type") shape = arg.get("shape") levellimits = [] rank = [] r = arg.find("rank") if r != None: - if shape == "-": - raise RuntimeError(f"rank is not empty, but shape is '-' for {op_name}: {name}") rank = [r.get('min'),r.get('max')] + if shape == "-" and (rank[0] != '0' or rank[1] != '0'): + raise RuntimeError(f"rank is not empty or non-zero, but shape is '-' for {op_name}: {name}") # validate rank against the shape argument (shape_check, shape_rank) = get_rank_from_shape(shape) if shape_check and (shape_rank < int(rank[0]) or shape_rank > int(rank[1])): @@ -170,7 +172,7 @@ class TOSASpec: for cat in cats: argcats.append(TOSAOperatorArgumentCategory(cat[0], cat[1].split(","))) - return TOSAOperatorArgument(name, desc, argcats, argtype, shape, levellimits, rank) + return TOSAOperatorArgument(name, desc, argcats, argtype, argtelty, shape, levellimits, rank) def __load_enum(self, arg): name = arg.get("name") diff --git a/tosa.xml b/tosa.xml index 430f5fd..678c6ac 100644 --- a/tosa.xml +++ b/tosa.xml @@ -16,15 +16,16 @@ ARGMAX - + Input tensor - + Axis in range from 0 to rank(shape1) - 1 + - + Output tensor, with rank = rank(shape1) - 1 @@ -51,23 +52,23 @@ AVG_POOL2D - + Input tensor - + [kernel_y, kernel_x] - + [stride_y, stride_x] - + [pad_top, pad_bottom, pad_left, pad_right] @@ -75,16 +76,19 @@ - + Enumerated type, must be one of INT32, FP16, FP32, as defined in the Supported Data Types table for this operation + - + Input tensor zero point. Must be zero for non-int8 types. + - + Output tensor zero point. Must be zero for non-int8 types. + - + Output tensor 4D @@ -115,21 +119,21 @@ CONV2D - + Input tensor - + Weight kernel size KH x KW - + Per output channel bias data. - + [pad_top, pad_bottom, pad_left, pad_right] @@ -137,23 +141,25 @@ - + [stride_y, stride_x] - + [dilation_y, dilation_x] - + Input tensor zero point. Must be zero for non-int8 types. + - + Weight zero point. Must be zero for non-int8 types. + - + Output tensor @@ -186,22 +192,22 @@ CONV3D - + Input tensor - + Weight kernel size KDxKHxKW - + Per output channel bias data. - + [pad_d0, pad_d1, pad_top, pad_bottom, pad_left, pad_right] @@ -211,24 +217,26 @@ - + [stride_d, stride_y, stride_x] - + [dilation_d, dilation_y, dilation_x] - + Input tensor zero point. Must be zero for non-int8 types. + - + Weight zero point. Must be zero for non-int8 types. + - + Output tensor @@ -261,21 +269,21 @@ DEPTHWISE_CONV2D - + Input tensor - + Weight kernel size KH x KW - + Per output channel bias data. - + [pad_top, pad_bottom, pad_left, pad_right] @@ -283,23 +291,25 @@ - + [stride_y, stride_x] - + [dilation_y, dilation_x] - + Input tensor zero point. Must be zero for non-int8 types. + - + Weight zero point. Must be zero for non-int8 types. + - + Output tensor @@ -332,24 +342,25 @@ FFT2D - + Real part of the complex input. H,W must be powers of two. - + Imaginary part of the complex input. H,W must be powers of two. - + false for forward FFT, true for inverse FFT + - + Real part of the complex output. - + Imaginary part of the complex output. @@ -365,25 +376,27 @@ FULLY_CONNECTED - + Input tensor - + Weights - + Per output channel bias data. - + Input tensor zero point. Must be zero for non-int8 types. + - + Weight zero point. Must be zero for non-int8 types. + - + Output tensor @@ -416,21 +429,23 @@ MATMUL - + Input tensor A, N matrices of size HxC - + Input tensor B, N matrices of size CxW - + Input tensor A zero point. Must be zero for non-int8 types. + - + Input tensor B zero point. Must be zero for non-int8 types. + - + Output tensor, N matrices of size HxW @@ -461,23 +476,23 @@ MAX_POOL2D - + Input tensor 4D - + [kernel_y, kernel_x] - + [stride_y, stride_x] - + [pad_top, pad_bottom, pad_left, pad_right] @@ -485,7 +500,7 @@ - + Output tensor 4D @@ -511,17 +526,17 @@ RFFT2D - + Real input. H,W must be powers of two. - + Real part of the complex output - + Imaginary part of the complex output. @@ -537,21 +552,21 @@ TRANSPOSE_CONV2D - + Input tensor - + Weight kernel size KH x KW - + Per output channel bias data. - + [out_pad_top, out_pad_bottom, out_pad_left, out_pad_right] @@ -559,23 +574,25 @@ - + [stride_y, stride_x] - + [N,OH,OW,OC] - + Input tensor zero point. Must be zero for non-int8 types. + - + Weight zero point. Must be zero for non-int8 types. + - + Output tensor @@ -610,18 +627,20 @@ CLAMP - + Input tensor - + Minimum clip value + - + Maximum clip value + - + Output tensor of same type and shape as input @@ -647,12 +666,12 @@ SIGMOID - + Input tensor - + Output tensor of same type and shape as input @@ -676,12 +695,12 @@ TANH - + Input tensor - + Output tensor of same type and shape as input @@ -707,15 +726,15 @@ ADD - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -741,18 +760,19 @@ ARITHMETIC_RIGHT_SHIFT - + Input tensor - + Input tensor with the same rank as input1 - + If true then the shift is rounded + - + Output tensor with broadcast shape if necessary @@ -768,15 +788,15 @@ BITWISE_AND - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -792,15 +812,15 @@ BITWISE_OR - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -816,15 +836,15 @@ BITWISE_XOR - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -840,15 +860,15 @@ INTDIV - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -862,15 +882,15 @@ LOGICAL_AND - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -884,15 +904,15 @@ LOGICAL_LEFT_SHIFT - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -908,15 +928,15 @@ LOGICAL_RIGHT_SHIFT - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -932,15 +952,15 @@ LOGICAL_OR - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -954,15 +974,15 @@ LOGICAL_XOR - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -976,15 +996,15 @@ MAXIMUM - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -1010,15 +1030,15 @@ MINIMUM - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -1044,18 +1064,19 @@ MUL - + Input tensor - + Input tensor with the same rank as input1 - + Result right shift (int32_t data type only) + - + Output tensor with broadcast shape if necessary @@ -1084,15 +1105,15 @@ POW - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -1117,15 +1138,15 @@ SUB - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -1151,15 +1172,15 @@ TABLE - + Input tensor - + Lookup table tensor - + Output tensor @@ -1179,12 +1200,12 @@ ABS - + Input tensor - + Output tensor of same type, size as the input tensor @@ -1209,12 +1230,12 @@ BITWISE_NOT - + Input tensor - + Output tensor of same type, size as the input tensor @@ -1229,12 +1250,12 @@ CEIL - + Input tensor - + Output tensor of same type, size as the input tensor @@ -1258,12 +1279,12 @@ CLZ - + Input tensor - + Output tensor of same type, size as the input tensor @@ -1276,12 +1297,12 @@ EXP - + Input tensor - + Output tensor of same type, size as the input tensor @@ -1305,12 +1326,12 @@ FLOOR - + Input tensor - + Output tensor of same type, size as the input tensor @@ -1334,12 +1355,12 @@ LOG - + Input tensor - + Output tensor of same type, size as the input tensor @@ -1363,12 +1384,12 @@ LOGICAL_NOT - + Input tensor - + Output tensor of same type, size as the input tensor @@ -1381,18 +1402,20 @@ NEGATE - + Input tensor - + Input 1 zero point. Must be zero for non-int8 types. + - + Output zero point. Must be zero for non-int8 types. + - + Output tensor of same type, size as the input tensor @@ -1420,12 +1443,12 @@ RECIPROCAL - + Input tensor - + Output tensor of same type, size as the input tensor @@ -1449,12 +1472,12 @@ RSQRT - + Input tensor - + Output tensor of same type, size as the input tensor @@ -1480,19 +1503,19 @@ SELECT - + Input selector tensor - + Input value tensor if input1 is True - + Input value tensor if input1 is False - + Output tensor of same type as input2 and input3, with broadcast shape if necessary @@ -1523,15 +1546,15 @@ EQUAL - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -1558,15 +1581,15 @@ GREATER - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -1593,15 +1616,15 @@ GREATER_EQUAL - + Input tensor - + Input tensor with the same rank as input1 - + Output tensor with broadcast shape if necessary @@ -1630,14 +1653,15 @@ REDUCE_ALL - + Input tensor - + Axis to reduce, in range from 0 to rank(shape1)-1 + - + Output tensor. Same rank as the input tensor. @@ -1650,14 +1674,15 @@ REDUCE_ANY - + Input tensor - + Axis to reduce, in range from 0 to rank(shape1)-1 + - + Output tensor. Same rank as the input tensor. @@ -1670,14 +1695,15 @@ REDUCE_MAX - + Input tensor - + Axis to reduce, in range from 0 to rank(shape1)-1 + - + Output tensor. Same rank as the input tensor. @@ -1704,14 +1730,15 @@ REDUCE_MIN - + Input tensor - + Axis to reduce, in range from 0 to rank(shape1)-1 + - + Output tensor. Same rank as the input tensor. @@ -1738,14 +1765,15 @@ REDUCE_PRODUCT - + Input tensor - + Axis to reduce, in range from 0 to rank(shape1)-1 + - + Output tensor. Same rank as the input tensor. @@ -1769,14 +1797,15 @@ REDUCE_SUM - + Input tensor with rank from 1 to 4 - + Axis to reduce, in range from 0 to rank(shape1)-1 + - + Output tensor. Same rank as the input tensor. @@ -1803,15 +1832,15 @@ CONCAT - - + List of input tensors. All inputs must have the same rank and data type - + Axis along which concatenation is to occur, in range from 0 to rank(shape)-1 + - + Output tensor @@ -1840,18 +1869,19 @@ PAD - + Input tensor - + Number of pad elements at the start and end of each dimension - + Constant value to be used as padding + - + Output tensor of same type as the input tensor @@ -1880,16 +1910,16 @@ RESHAPE - + Input tensor - + List of values, with each element giving the size of the result tensor for the given dimension. At most one dimension may be given as -1 to automatically calculate the dimension size. - + Output tensor of same type, size as the input tensor @@ -1918,15 +1948,16 @@ REVERSE - + Input tensor - + Axis to reverse, in range from 0 to rank(shape)-1 + - + Output tensor. Same shape as input tensor @@ -1954,21 +1985,21 @@ SLICE - + Input tensor - + List of integer coordinates, of length equal to the rank of input1. Start coordinate for slicing. - + List of integer size values, of length equal to the rank of input1. Size of the input to be used. - + Output tensor of same type as the input tensor @@ -1996,15 +2027,15 @@ used. TILE - + Input tensor - + Number of times to replicate input1 in each dimension - + Output tensor of same type, rank as the input tensor @@ -2033,15 +2064,15 @@ used. TRANSPOSE - + Input tensor - + List of integers of length equal to the rank of input1. Values must be valid dimensions within shape1, and may not be repeated. - + Output tensor of same type, rank as the input tensor @@ -2072,15 +2103,15 @@ used. GATHER - + 3D value tensor - + 2D index tensor - + 3D output tensor @@ -2107,19 +2138,19 @@ used. SCATTER - + 3D values in tensor - + 2D index tensor - + 3D input tensor - + 3D output tensor @@ -2148,28 +2179,29 @@ used. RESIZE - + Input tensor - + [scale_y_n, scale_y_d, scale_x_n, scale_x_d] - + [offset_y, offset_x] - + [border_y, border_x] - + BILINEAR or NEAREST + - + Output tensor @@ -2201,12 +2233,12 @@ used. CAST - + Input tensor - + Output tensor @@ -2319,37 +2351,42 @@ used. RESCALE - + Input tensor - + Output tensor with the same shape as input - + Input tensor zero point. Must be zero for non-int8 types. + - + Output tensor zero point. Must be zero for non-int8 types. + - + Scaling multiplier array - + Scaling shift array - + if (scale32) mul_t=int32_t else mul_t=int16_t + - + Select double round mode + - + if (per_channel) NC=shape[rank(shape)-1] else NC=1 + @@ -2380,11 +2417,11 @@ used. CONST - + Constant values - + Output tensor of the same type, size as the input tensor @@ -2413,11 +2450,11 @@ used. IDENTITY - + Input tensor - + Output tensor of the same type, size as the input tensor @@ -2447,20 +2484,20 @@ used. COND_IF - + Input condition as a size 1 tensor - + List of input tensors - + TOSA graph to execute if condition is true - + TOSA graph to execute if condition is false - + List of output tensors @@ -2468,16 +2505,16 @@ used. WHILE_LOOP - + List of input tensors - + TOSA graph to evaluate the condition - + TOSA graph to execute the loop body - + List of output tensors diff --git a/tosa.xsd b/tosa.xsd index fe08885..40fd613 100644 --- a/tosa.xsd +++ b/tosa.xsd @@ -30,6 +30,7 @@ + @@ -44,6 +45,13 @@ + + + + + + + @@ -53,6 +61,8 @@ + + @@ -66,6 +76,18 @@ + + + + + + + + + + + + @@ -179,7 +201,8 @@ - + + -- cgit v1.2.1