aboutsummaryrefslogtreecommitdiff
path: root/verif/checker
AgeCommit message (Collapse)Author
2023-11-23Verify library fix to only accept FP64 ref dataJeremy Johnson
Make test_verif_result_check output to only say result files are non-compliant as it may not be implmenetation result issues. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I14d10a7ee3aeff7802b6b173280d9f2054760a08
2023-10-31Allow more integer types in result check.Jeremy Johnson
Added int8, int16, uint8 and uint16. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I6b1174aa7369c34a613b322c4b05bf4fc586ee6e
2023-10-02Support for compliance checking testingJeremy Johnson
Updated to conformance generator to not generate tests with results for compliance tests. Updated test runner to run compliance mode version (precise & abs mode) of reference model to create test results to use against SUT results. Updated reference model to enable abs_mode on correct desc.json flags. Updated test checker to support compliance checking using verifier lib. Seperated color printing from test checker. Change-Id: I7e2fbfc6883916caa5d94d4ece122c48bf45f530 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-10-02Update verifier library data-type supportJeremy Johnson
Make compliance meta-data data-type required for all. Add data-type checking for all verifier modes. Add initial enum support for new ROUND compliance mode. Improve print out information from library. Use numpy ctypes.data_as to get f16 support compared to ctypes_lib. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie983ba4ea958a88556f30c09b3ebc19cd9ec96b7
2023-09-28Refactor path arguments to tosa-toolsJeremy Johnson
tosa_verif_conformance_generator - Move to using ref-model-path instead of ref-model-dir - Add schema-path and flatc-path - Add model_files module to locate default places for files convert2conformance - Remove default paths verifier - Switch to using exact path of verifier library tosa_verif_run_tests - Use conformance model_files to locate defaults Change-Id: Ieca3b164670e2a7dcb047743667cc4e8317daa97 Signed-off-by: Josef Malmström <josef.malmstrom@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-09-13Build compliance verifier library with a python interface.Jeremy Johnson
Build steps to create tosa_reference_verify_lib.so New python class to handle loading and calling verifier with pytests. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id42c71c398ed7364742b4a039977412ef3081c4a
2022-11-19Initial set up of Main Inference conformance test genJeremy Johnson
tosa-verif-build-tests - option for setting FP values range - option for recursively finding tests - change from os.path to Path tosa_verif_result_check - option to supply FP tolerance - output difference and max tolerance on contents mismatch - change from os.path to Path MI conformance - contains examples of AVG_POOL2D and CONV2D tests Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I8e1645cd8f10308604400ea53eef723ca163eed7
2022-11-09Add BF16 support to reference modelJames Ward
* Upgrade Eigen to 3.4.0 (for bfloat16 support) and add work- arounds for reduce.any() and reduce.all() bugs (introduced between 3.3.7 and 3.4.0) * Truncation to bfloat16 now performed in eval() methods Signed-off-by: James Ward <james.ward@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If5f5c988d76d3d30790acf3b97081726b89205fe
2022-10-11Reference model changes for fp16 supportJames Ward
Change-Id: I72f21fcfa153046274969d327313e3349981dbe6 Signed-off-by: James Ward <james.ward@arm.com>
2022-03-02Add framework unit test generation scriptsJeremy Johnson
And fixes in tosa_verif_run_tests: * support for no-color printing * stop double printing of error messages on verbose * differentiate result code pass from results check Change-Id: I26e957013a8d18f7d3d3691067dfb778008a1eea Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-01-06Update tosa_verif_run_refJeremy Johnson
Rename to tosa_verif_run_tests to match build_tests Improve output and system under test support Improve xunit support Add results checker Add utilities json2numpy and json2fbbin Add set of python tests Update README.md Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ia09f8e6fd126579b3ba1c1cda95c1326802417ca