From 6dd341093507157aabbea00b90ca8902509cfd4f Mon Sep 17 00:00:00 2001 From: Eric Kunze Date: Sun, 25 Feb 2024 22:24:52 -0800 Subject: Modify TOSA profiles Create composable profiles and profile extensions. Define requirements for a TOSA implementation to fully implement at least one profile. Signed-off-by: Eric Kunze Change-Id: I02cfb0171b2d227727f530cb29108b479206b25b --- chapters/introduction.adoc | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) (limited to 'chapters/introduction.adoc') diff --git a/chapters/introduction.adoc b/chapters/introduction.adoc index 0a94ef3..c34bf7b 100644 --- a/chapters/introduction.adoc +++ b/chapters/introduction.adoc @@ -89,27 +89,36 @@ The following principles govern the selection of operators within TOSA. === Profiles -TOSA supports three profiles that enable efficient implementation on different classes of device. -The Base Inference profile is intended for embedded integer/fixed-point designs performing inference only. -The Main Inference profile is intended for general inference functionality including integer and floating-point data types. -The Main Training profile adds training operators in addition to inference operators. -This version of the specification covers the Base Inference and Main Inference profiles. -Main Training profile is expected in a later version of the specification. -The following table summarizes the three profiles: +TOSA profiles enable efficient implementation on different classes of device. +Each profile is an independent set of operations and data type combinations. + +TOSA profile extensions define optional operation and data type combinations. + +Each operator's Supported Data Types table will define which profile or extension an operator and data type is in. +An operator / data type combination may be part of multiple profiles or extensions. +If so, each profile and extension will be listed in the Supported Data Types table. +In addition, a table listing all operations for each profile can be found in Appendix B. + +The following are required for compliant TOSA implementations: + +* A TOSA implementation must implement at least one profile. +* A TOSA implementation may choose to implement any extensions. +* If a TOSA implementation chooses to implement an extension, it must implement the complete extension. +* If a operator / data type combination requires multiple extensions, the combination is only required to be implemented if all extensions are implemented +** For example, a CAST from bf16 to fp8 is only required if both extensions are implemented. .Profiles -|=== -|Profile|Name|Integer Inference|Floating-point Inference|Training +include::{generated}/profiles.adoc[] -|Base Inference|TOSA-BI|Yes|No|No -|Main Inference|TOSA-MI|Yes|Yes|No -|Main Training|TOSA-MT|Yes|Yes|Yes -|=== +.Profile Extensions +include::{generated}/profile_extensions.adoc[] === Levels A TOSA level defines operator argument ranges that an implementation shall support. This is distinct from a profile that defines the operations and data-types supported. +One level must apply to all profiles and extensions supported by an implementation. + This version of the specification defines two TOSA levels: * No level : allows the full range of arguments specified by the operations according to the operation data types. @@ -487,6 +496,8 @@ To map indexes in an output tensor to that of an input tensor, see <