TOSA Conformance Tests

NOTE: this repository of TOSA conformance tests has been DEPRECATED, the latest tests now need to be generated using the TOSA Unit Test Infrastructure in the TOSA Reference Model https://git.mlplatform.org/tosa/reference_model.git - see the TOSA Conformance Generator section in the reference model README for details.

Introduction

This repository contains the conformance tests for the Tensor Operator Set Architecture (TOSA) Specification https://git.mlplatform.org/tosa/specification.git.

Coverage

These conformance tests cover the Base Inference profile (TOSA-BI) only.

There are tests included that cover examples of TensorFlow Lite framework operators converted to TOSA.

Versioning

The conformance tests repository has branches that correspond to each TOSA version (major.minor), and tags (major.minor.patch) that match to specific versions.

Perform the following command to check out a specific version:

git checkout --recurse-submodules VERSION

Where VERSION can be for example: v0.23 or v0.23.0

Test Contents

Typical test contents will be:

test*.json - TOSA test in flatbuffer JSON format
input-*.json or placeholder-*.json - input files (e.g. tensors) in JSON format
Conformance-*.json - expected output file in JSON format
desc.json - test meta-data in JSON format
model-* - optional framework model file, used to create the TOSA test

Usage

Flatbuffer Schema

A JSON flatbuffer TOSA test file needs to be used with a corresponding version of the schema file. The schema is linked via git submodules and should be checked out with the specific version chosen above. You can find the schema:

ls -l third_party/serialization_lib/schema/tosa.fbs

If you have not checked out a specific conformance version, then use the following command to initialise the necessary submodule:

git submodule update --init --recursive

Running

It is recommended to use these tests with the Unit Test Runner from the TOSA Unit Test Infrastructure in the TOSA Reference Model https://git.mlplatform.org/tosa/reference_model.git, please see that repository for more information.

License

The TOSA Conformance tests are licensed under Apache-2.0 unless otherwise noted.