Age | Commit message (Collapse) | Author |
|
Add --test-version option to tosa_verif_conformance_generator to
select the version for tests to output.
Add --output-type to allow json, binary or both files to be created
during conformance generation.
Fix passing schema_path to test runner.
Add explicit verify lib path arg to test runner.
Change-Id: I5f1ad137d713fca408a98470ea77bddf8916c5f3
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Updated to conformance generator to not generate tests with results for
compliance tests.
Updated test runner to run compliance mode version (precise & abs mode)
of reference model to create test results to use against SUT results.
Updated reference model to enable abs_mode on correct desc.json flags.
Updated test checker to support compliance checking using verifier lib.
Seperated color printing from test checker.
Change-Id: I7e2fbfc6883916caa5d94d4ece122c48bf45f530
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Make compliance meta-data data-type required for all.
Add data-type checking for all verifier modes.
Add initial enum support for new ROUND compliance mode.
Improve print out information from library.
Use numpy ctypes.data_as to get f16 support compared to ctypes_lib.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ie983ba4ea958a88556f30c09b3ebc19cd9ec96b7
|
|
tosa_verif_conformance_generator
- Move to using ref-model-path instead of ref-model-dir
- Add schema-path and flatc-path
- Add model_files module to locate default places for files
convert2conformance
- Remove default paths
verifier
- Switch to using exact path of verifier library
tosa_verif_run_tests
- Use conformance model_files to locate defaults
Change-Id: Ieca3b164670e2a7dcb047743667cc4e8317daa97
Signed-off-by: Josef Malmström <josef.malmstrom@arm.com>
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Signed-off-by: Dmitrii Agibov <dmitrii.agibov@arm.com>
Change-Id: I4cd818af0db5e6e8a96641246cd3263ba4878f56
|
|
Attempt to avoid API getting out of sync.
Signed-off-by: Eric Kunze <eric.kunze@arm.com>
Change-Id: Ic7b72c3f906e4a38cb26159bb67e9b1c4e22ca96
|
|
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I3620799a01784865b3de1407e70e02582966f6d1
|
|
- Updates TOSA specification to the latest version
- Updates generate_api.py to generate the operator API correctly
for ops with additional tensor inputs.
- Removes default arguments for func_debug and func_config to make
the API C compliant again.
- Updates model_runner_tests.cpp for operators that have changed.
- Adds a unit test for the Tile operator to check that generated code
for additional tensor inputs works correctly.
Signed-off-by: Grant Watson <grant.watson@arm.com>
Change-Id: I1e26065c6ed333b2ca4b3da39972d30f896fa6e5
|
|
Incorrectly used "tag" in test.desc instead of "tags"
Incorrectly selected all tests rather than just all positive
tests depending on operator selection option.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ib257bc57accf745beedfa4a3b2ece9539c0972c2
|
|
Add initial support for compliance and lazy data-gen meta data
added to desc.json for MATMUL.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I00c047814134a96d7c98d890e93b5884e25b8e64
|
|
Rename scripts utlities to tosa_* such as tosa_json2numpy
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ie8d584e2afb189fb74cf96b39590c7c27444ba14
|
|
Updates the generate_api.py script and associated
templates to allow func_config and debug_config
to be passed when running individual operators
on the API.
This will allow us, for example, to set precise_mode
and abs_mode when running individual operators.
Signed-off-by: Grant Watson <grant.watson@arm.com>
Change-Id: Ia3e7ffc146f876daa307558433177c68285843b7
|
|
Compress flatbuffer JSON files for conformance to allow while_loop
tests to fit in the 30MB file size limit.
Do not include new SHAPE type in usable DTypes so that selection of
ERROR_IF tests are not changed.
Also enable strict json mode for flatbuffers.
Change-Id: Id89a9963244c34769a4662f9951adde0aa3334d9
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Signed-off-by: Grant Watson <grant.watson@arm.com>
Change-Id: I1296f968baca335ea88691bc973e2d01b2aa2c5b
|
|
- Allows axis attributes to be treated differently to other arguments in attribute.def
Signed-off-by: Grant Watson <grant.watson@arm.com>
Change-Id: I1be2595c24bf22e5391a2911a5283391d310df37
|
|
Split generation of conformance tests into groups to avoid a change
in existing tests selection
* A standard group is used for the normal/default conformance tests
* An 8k level group covers the extension of testing for 8k level
* Other arbitrary groups can be created and the group names will be
used to tag the tests, so that in the future they could be filtered
Group selection criteria is in a new dictionary
* A default selection config is used for generation groups
* But a different config can be used for a particular generation group
Added parameter sparsity for pad of rank6 and above, and further
parameter sparsity options to test_select to limit number of tests
selected.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ib9f1185c0ae6d0957df2116411c17e958157c6d3
|
|
Signed-off-by: James Ward <james.ward@arm.com>
Change-Id: If67f503a1848967bc1671646c3011d055b622c52
|
|
Rationale for making this change:
- In the original design, for control flow operators like WhileOp,
child blocks couldn't read the tensor variables (global consts) in the root level block,
this patch added the machanism for child blocks to access their parent
level block's tensors.
- This change also relies on another serialization change on adding
another layer of abtraction called Region:
- Serialization patch: [region] Add TosaSerializationRegion to serialization_lib
- Updated the corresponding python version of the serialization code: TosaSerializerRegion to python version of serialization_lib
- This change also relies on the TOSA MLIR Translator change: Add RegionBuilder to TOSA MLIR Translator
- Added the WhileOp related test cases: While, LSTM, GRU, RNN
- Other related fixes
Signed-off-by: Jerry Ge <jerry.ge@arm.com>
Change-Id: I13ae33628ad07e41d248e88652ce1328654694ab
|
|
Add option to control negative tests dim range to stop ERROR_IF
test clashes between profiles.
Change-Id: I1f806934480cc32b95e80613c66fecb1e6ff5426
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
- Adds a script to generate operators.h and operators.cc
- Adds jinja2 templates for generating operators.h and operators.cc
- Adds unit tests for a subset of the operators generated
- Includes the TOSA specification as a submodule
- Adds supporting C++ and header files
Signed-off-by: Grant Watson <grant.watson@arm.com>
Change-Id: I5b60db4c56113110d8e75fe1152525d258233f9c
|
|
Change-Id: I72f21fcfa153046274969d327313e3349981dbe6
Signed-off-by: James Ward <james.ward@arm.com>
|
|
Add supported profiles for each test in convert2conformance and
tosa_verif_conformance_generator.
Enable filtering of profile tests on running in tosa_verif_run_tests.
Reorganize arguments in conformance_generator to have more important
ones first.
Change-Id: Ie6e5b68727adb3c39b04aa482dd6433788f7bcc9
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Change-Id: I5cb16ea1f47ee454f03a1d5182827e3fd9f49128
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Change-Id: Ia00a9fa0d260c68ce8532dec9925467558fe7dea
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Script converts framework and reference model unit tests into a
suitable format for including into a conformance suite.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ida1ec8a0a7ea31fd3a3f62c4cb52d7cc2bf0b439
|
|
Fix up issues in existing python scripts.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Id4adab404560c3129c66f31c21ff0ce148283c73
|
|
Rename to tosa_verif_run_tests to match build_tests
Improve output and system under test support
Improve xunit support
Add results checker
Add utilities json2numpy and json2fbbin
Add set of python tests
Update README.md
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ia09f8e6fd126579b3ba1c1cda95c1326802417ca
|
|
Split generator and runner scripts
Add package setup
Add py-dev-env.sh/.bash to allow editing source files during dev
Update README.md with installation info
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I172fe426d99e2e9aeeacedc8b8f3b6a79c8bd39d
|
|
Change-Id: I2f8e7fa63e2ae40203e57d2cc8814bde3b312cb6
Signed-off-by: Eric Kunze <eric.kunze@arm.com>
|