aboutsummaryrefslogtreecommitdiff
path: root/verif/generator/tosa_verif_build_tests.py
AgeCommit message (Collapse)Author
5 daysUpdate FP Special/full tests for ACTIVATION & UNARY opsHEADmainJeremy Johnson
Make sure there is one FP Special or Full range test per op per data type Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I234cf1a56ee294039a7486bd76885d130b399a76
2024-04-09Generate tests with random combination of const/inputsJeremy Johnson
Override the default input/consts used for each op with biased random combos. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I7a5eb2b01a47248bc0db5a2f930ad2ac52cc0a67
2024-04-08Add support for multi args in tosa_verif_build_testsJeremy Johnson
Now supports shorter "--target-rank 0 1" and the original method of "--target-rank 0 --target-rank 1" Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ia45a168588c6fca4dcd4cbbf526ac49cb0bdf621
2024-03-28Improve PAD coverage for main conformanceJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ia3e0d871525840f26948b45658bba71e75d767ea
2024-03-28Improve CONV2D coverage for conformanceJeremy Johnson
Enable testing of broadcastable bias channel. Change convolution arguments function to calculate workable convolutions by adjusting PAD size as needed for FP and errors. Add --conv-kernel build_tests option to allow different sized convolution kernels Improve coverage with batch size, specific shapes and kernels of 7x7 and 9x9 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I59bb737f398f4363cc606b24fea7812b2059bc01
2024-03-20Fix missing/broken ERROR_IF testsJeremy Johnson
Fix CONV2D WrongOutputType FP32 & Pad/Stride/DilationSmallerZero issues. Fix PAD WrongInputType. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I57fc57c43e63685e05bf5e3d562c3167411fd57b
2024-03-19Updated build_tests to support different random generatorsJeremy Johnson
All generator functions now take RNG argument to allow different random number generators, rather than relying on global RNG Default behaviour is the same as before using global RNG Added stable random generation mode * shape rng based on operator, rank and datatype * arguments rng based on operator, shape and datatype * build operands and data rng based on op, shape, datatype and args Add optional stable RNG test generation to conformance_generator Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I5ee4ff85575a81177fd74ed1617e946bfa3a0769
2024-03-13Add quick testing selection defaultJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I5c240eec4385aaa3a6b4bf9533669b91c74c76cf
2024-03-12Improved test selection before test generationJeremy Johnson
Add test list output to tosa_verif_build_tests and test list capture to file for tosa_verif_conformance_generator Improve PAD & CONV2D test coverage for tosa-mi conformance Change to use logging for output to hide info from test lists Tweak verbosity levels of tosa_verif_conformance_generator Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ic29da5776b02e9ac610db6ee89d0ebfb4994e055
2024-02-12Update test builder internal interfacesJeremy Johnson
Move remaining ops from using testArgs to argsDict. All tvg/build_fcn function interfaces updated. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie886fd931bd74608bda621363100bf8bfd7385e6
2023-11-10Main Compliance testing support for ADD, SUB, MINIMUM and MAXIMUMJeremy Johnson
Added main inference compliance mode data generation and verification. Improved error reporting in tosa_verif_build_tests. Fixed pseudo random invalid range check. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ib538b5999444d67704ebc322be3c7becbe5ad206
2023-11-02Main compliance testing support for MULJeremy Johnson
Update verify ULP mode to allow fractions (e.g. 0.5). Update pseudo generator to accept ranges. Fix up pseudo random distribution based on ranges. Change-Id: I9168c5f7d37722678c0f1f9e906953c8cec367b1 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-10-16Data generator library python interface addedJeremy Johnson
Added support for using generate library in tosa_verif_build_tests and tosa_verif_run_tests tosa tool scripts. Reduced scope of compliance test creation and verification to the supported type of FP32. Fix missing virtual destructor warning in generate_dot_product.h and add config file for generate library. Simple pytests included to check python interface. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I6cdad9b00660d6ddc8bd07fdea813937fb48626a
2023-09-07Initial lazy data-gen and compliance test build supportJeremy Johnson
Add initial support for compliance and lazy data-gen meta data added to desc.json for MATMUL. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I00c047814134a96d7c98d890e93b5884e25b8e64
2023-04-258K level: RESIZE scaling at 64 maximumJeremy Johnson
Add support for producing scaling 64 tests in RESIZE and make sure default tests are not generated with scaling greater than 64. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I3e99cde7e6bec6ea7854a139c6fc2ed151af5b1e
2023-02-22Bug fixes for max-batch-size/ofm-depthJames Ward
* Conformance regen required for tosa-bi: conv2d conv3d depthwise_conv2d transpose_conv2d * Include bug fix for testGen.makeShape() * Include json changes to limit size of tests which now have ofm_depth > 1 Signed-off-by: James Ward <james.ward@arm.com> Change-Id: Ic8221b9a742f5737df523997bee7260f4dfcdef3
2022-11-19Initial set up of Main Inference conformance test genJeremy Johnson
tosa-verif-build-tests - option for setting FP values range - option for recursively finding tests - change from os.path to Path tosa_verif_result_check - option to supply FP tolerance - output difference and max tolerance on contents mismatch - change from os.path to Path MI conformance - contains examples of AVG_POOL2D and CONV2D tests Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I8e1645cd8f10308604400ea53eef723ca163eed7
2022-11-09Add BF16 support to reference modelJames Ward
* Upgrade Eigen to 3.4.0 (for bfloat16 support) and add work- arounds for reduce.any() and reduce.all() bugs (introduced between 3.3.7 and 3.4.0) * Truncation to bfloat16 now performed in eval() methods Signed-off-by: James Ward <james.ward@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If5f5c988d76d3d30790acf3b97081726b89205fe
2022-09-26Add CONST testing to Numpy refmodel testsJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id32f3b7c287c9e8b0fd42dd96922be5a255598e2
2022-09-21Add simple post commit reference model testing against NumpyJeremy Johnson
Only perform testing of this after build of ref model using postcommit pytest mark. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I771a18d2c9cd4051fecafad3e6079b44f2ed62fa
2022-08-11Update RESIZE operator in test generator for spec updatesJeremy Johnson
* Add common screen aspect ratios with borders to random pool of tests * Add up/downscaling to random pool Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Iee8e3f5ed6bd5c941816474df20a7fd433646d6b Signed-off-by: James Ward <james.ward@arm.com>
2022-02-10Minor formatting fixes for pythonJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Iff6eb54734e7a78eb22d1a64803b937063b7f65a
2022-01-26Fix for max arguments to be respected in build testsJeremy Johnson
The conformance oversize tests are not produced by default. Change-Id: I9f7d4a60b9d8991889a077b61613e13e288bb65d Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-01-13Add python pre-commit script checkersJeremy Johnson
Fix up issues in existing python scripts. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id4adab404560c3129c66f31c21ff0ce148283c73
2022-01-06Reorganize verif and create packagesJeremy Johnson
Split generator and runner scripts Add package setup Add py-dev-env.sh/.bash to allow editing source files during dev Update README.md with installation info Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I172fe426d99e2e9aeeacedc8b8f3b6a79c8bd39d
generated by cgit v1.2.1 (git 2.23.0) at 2024-06-03 12:17:21 +0000