From 6dd341093507157aabbea00b90ca8902509cfd4f Mon Sep 17 00:00:00 2001 From: Eric Kunze Date: Sun, 25 Feb 2024 22:24:52 -0800 Subject: Modify TOSA profiles Create composable profiles and profile extensions. Define requirements for a TOSA implementation to fully implement at least one profile. Signed-off-by: Eric Kunze Change-Id: I02cfb0171b2d227727f530cb29108b479206b25b --- chapters/appendix_b.adoc | 12 ++++++++++++ chapters/introduction.adoc | 37 ++++++++++++++++++++++++------------- chapters/operators.adoc | 4 ++-- 3 files changed, 38 insertions(+), 15 deletions(-) create mode 100644 chapters/appendix_b.adoc (limited to 'chapters') diff --git a/chapters/appendix_b.adoc b/chapters/appendix_b.adoc new file mode 100644 index 0000000..64f9f3a --- /dev/null +++ b/chapters/appendix_b.adoc @@ -0,0 +1,12 @@ +// +// This confidential and proprietary software may be used only as +// authorised by a licensing agreement from ARM Limited +// (C) COPYRIGHT 2024 ARM Limited +// ALL RIGHTS RESERVED +// The entire notice above must be reproduced on all authorised +// copies and copies may only be made to the extent permitted +// by a licensing agreement from ARM Limited. + +== Appendix B - Profile operator tables + +include::{generated}/profile_ops.adoc[] diff --git a/chapters/introduction.adoc b/chapters/introduction.adoc index 0a94ef3..c34bf7b 100644 --- a/chapters/introduction.adoc +++ b/chapters/introduction.adoc @@ -89,27 +89,36 @@ The following principles govern the selection of operators within TOSA. === Profiles -TOSA supports three profiles that enable efficient implementation on different classes of device. -The Base Inference profile is intended for embedded integer/fixed-point designs performing inference only. -The Main Inference profile is intended for general inference functionality including integer and floating-point data types. -The Main Training profile adds training operators in addition to inference operators. -This version of the specification covers the Base Inference and Main Inference profiles. -Main Training profile is expected in a later version of the specification. -The following table summarizes the three profiles: +TOSA profiles enable efficient implementation on different classes of device. +Each profile is an independent set of operations and data type combinations. + +TOSA profile extensions define optional operation and data type combinations. + +Each operator's Supported Data Types table will define which profile or extension an operator and data type is in. +An operator / data type combination may be part of multiple profiles or extensions. +If so, each profile and extension will be listed in the Supported Data Types table. +In addition, a table listing all operations for each profile can be found in Appendix B. + +The following are required for compliant TOSA implementations: + +* A TOSA implementation must implement at least one profile. +* A TOSA implementation may choose to implement any extensions. +* If a TOSA implementation chooses to implement an extension, it must implement the complete extension. +* If a operator / data type combination requires multiple extensions, the combination is only required to be implemented if all extensions are implemented +** For example, a CAST from bf16 to fp8 is only required if both extensions are implemented. .Profiles -|=== -|Profile|Name|Integer Inference|Floating-point Inference|Training +include::{generated}/profiles.adoc[] -|Base Inference|TOSA-BI|Yes|No|No -|Main Inference|TOSA-MI|Yes|Yes|No -|Main Training|TOSA-MT|Yes|Yes|Yes -|=== +.Profile Extensions +include::{generated}/profile_extensions.adoc[] === Levels A TOSA level defines operator argument ranges that an implementation shall support. This is distinct from a profile that defines the operations and data-types supported. +One level must apply to all profiles and extensions supported by an implementation. + This version of the specification defines two TOSA levels: * No level : allows the full range of arguments specified by the operations according to the operation data types. @@ -487,6 +496,8 @@ To map indexes in an output tensor to that of an input tensor, see <`, abbreviated `T`, represents a tensor whose elements are of type `element_type` where `element_type` can be any of the data types supported in TOSA. -* `tensor_list_t` represents a list of tensors. When lists are homogeneous, i.e. contain tensors of the same type, their type is further qualified as follows: `tensor_list_t>`. +* `tensor_list_t` represents a list of tensors. When lists are homogeneous, containing tensors of the same type, their type is further qualified as follows: `tensor_list_t>`. * `tosa_graph_t` represents a TOSA graph (see <>). Arguments belong to one of three categories: Input, Output, or Attribute. The category to which an argument belongs further constrains its type: * An Input argument must be a tensor or a list of tensors used to provide the data read by the operation. * An Output argument must be a tensor or a list of tensors into which the data produced by the operation is written. -* An Attribute argument is constant, i.e. its value is known at compilation time. It may have any data type supported by TOSA. +* An Attribute argument is constant, its value is known at compilation time. It may have any data type supported by TOSA. [[operator-graphs]] === Operator Graphs -- cgit v1.2.1