aboutsummaryrefslogtreecommitdiff
path: root/verif/runner
AgeCommit message (Collapse)Author
2022-11-19Initial set up of Main Inference conformance test genJeremy Johnson
tosa-verif-build-tests - option for setting FP values range - option for recursively finding tests - change from os.path to Path tosa_verif_result_check - option to supply FP tolerance - output difference and max tolerance on contents mismatch - change from os.path to Path MI conformance - contains examples of AVG_POOL2D and CONV2D tests Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I8e1645cd8f10308604400ea53eef723ca163eed7
2022-10-13Minor fixes & add FP16 support to refmodel testing and conformance genJeremy Johnson
Improve base inference conformance generation: * Change to target specific dtypes required for conformance tests. * Reduce dimension sizes of ERROR_IF tests. NOTE: Will impact tensor, clamp and all ERROR_IF tests. Add option to change seed on conformance generation for extra testing. Stop creation of convolution tests with negative output dimensions. Improve reporting on failing to allocate tensor due to above issue. Fix runner to correctly pass ref model debug flags. Update reference_model examples. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I992180dcfe265a7d50edfb151c9f38eeaef5c369
2022-08-22Simplify the argument parsing and debug codeEric Kunze
Change-Id: Ia681aa0eb1d42fa31a4658badb376f7b32b2e338 Signed-off-by: Eric Kunze <eric.kunze@arm.com>
2022-07-14Add profile to all created conformance tests.Jeremy Johnson
Add supported profiles for each test in convert2conformance and tosa_verif_conformance_generator. Enable filtering of profile tests on running in tosa_verif_run_tests. Reorganize arguments in conformance_generator to have more important ones first. Change-Id: Ie6e5b68727adb3c39b04aa482dd6433788f7bcc9 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-03-02Add framework unit test generation scriptsJeremy Johnson
And fixes in tosa_verif_run_tests: * support for no-color printing * stop double printing of error messages on verbose * differentiate result code pass from results check Change-Id: I26e957013a8d18f7d3d3691067dfb778008a1eea Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-01-06Update tosa_verif_run_refJeremy Johnson
Rename to tosa_verif_run_tests to match build_tests Improve output and system under test support Improve xunit support Add results checker Add utilities json2numpy and json2fbbin Add set of python tests Update README.md Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ia09f8e6fd126579b3ba1c1cda95c1326802417ca
2022-01-06Reorganize verif and create packagesJeremy Johnson
Split generator and runner scripts Add package setup Add py-dev-env.sh/.bash to allow editing source files during dev Update README.md with installation info Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I172fe426d99e2e9aeeacedc8b8f3b6a79c8bd39d