aboutsummaryrefslogtreecommitdiff
path: root/verif
AgeCommit message (Collapse)Author
2022-05-31Remove RESHAPE -1 dimensions supportJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I098daf49c92da12c07143cdd23ac9bb58acebbb9
2022-05-26Add support for uint16_t to RESCALEJeremy Johnson
Update ref-model RESCALE op to support UINT16 conversions Add testing for RESCALE UINT16 and ERROR_IFs Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ic6e6e53de1f0b054bedb9e6ba3856e7475498aba
2022-05-19Update framework test generation for ERROR_IF criteriaJeremy Johnson
Update to tosa_verif_framework_generator to produce valid test ranges for pooling and convolution tests Fix up test filtering to only filter on test name not output directory Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ifff7e7604a37e8680d7237dc2d85cd806b20e384
2022-05-04Fix for NEGATE using 32-bit accumulatorJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie5d119dc317303a0d2a71d018ac94ce6800ecbf5
2022-05-04Control oversized tests for TRANSPOSE_CONV2DJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I464871b412b3dfc9a84bd3d8fffe4c1de3072b69
2022-04-28Update tensor ops ERROR_IF criteriaJeremy Johnson
Update to ref model to check ERROR_IF criteria for pooling and convolution ops to match specification Update to tosa_verif_build_tests to produce valid test ranges and new ERROR_IF tests Plus update pooling ops big kernel to 9 (from 6) for better testing coverage and set dilation to 1 and add out_pad bottom & right for transpose_conv2d to match specification Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ic5759872d40ae8d3f3d07043d9a0f2fa0244d72e
2022-04-11Refactor verif/generator/tosa_test_gen.py into different filesJeremy Johnson
Move all error & validation into tosa_error_if.py Move all argument and tensor generation into tosa_arg_gen.py Move utility functions into tosa_utils.py Create new TosaTensorValuesGen class for specialising tensor value generation. Change-Id: Ib9ac65e2308b14471a567c6f11d775c76585bc5b Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-04-06Fix NEGATE int8 test value rangesJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ied54f5064dfb44029136b0f0297ef35e6eb0be6e
2022-04-05Add missing REQUIREs check to REDUCE_SUM in refmodelJeremy Johnson
And limit REDUCE_SUM test values to within int32 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I4d902b245d17eb343cfb2bbc23d9db28c1d1f4c3
2022-03-31Update pre-commit tool versionsJeremy Johnson
* black needed updating for ImportError of new click version Change-Id: If8e2d13a09996e331d676af31bf3d3aee9403b4e Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-03-24Add missing REQUIRE to NEGATE opJeremy Johnson
And update test generation to create values in predictable range Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I4ba1ff445bf6caeec9f8782902fc45929fe0ee77
2022-03-23Improve EQUAL tests to have matching numbersJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If785fdaab3026eca5a31888115fba8a6750e0460
2022-03-22Fix input rank-match comparison, --no-ref-model flag, no test listJared Smolens
- Fixed input shape/expected input shape mismatches when due to a missing batch dimension. - Return from run_test() immediately after running compiler when --no-ref-model is set. This avoids doing result comparisons on results that were never generated. - Exit gracefully when no tests are specified instead of throwing TypeError on iterating NoneType Signed-off-by: Jared Smolens <jared.smolens@arm.com> Change-Id: I91d2c24fd60aa1cf8b3755894030ff14f1371ed0
2022-03-14Generate REDUCE_MIN tests (instead of REDUCE_MAX)Jeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I41c364905ca364d3446962b737735b9068a29a77
2022-03-02Add framework unit test generation scriptsJeremy Johnson
And fixes in tosa_verif_run_tests: * support for no-color printing * stop double printing of error messages on verbose * differentiate result code pass from results check Change-Id: I26e957013a8d18f7d3d3691067dfb778008a1eea Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-02-17Fix rescale test gen for scale32 & dtypeJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I65a7fac6e65d2b979139adaf6c5a42139b5493c2
2022-02-16Update refmodel apply_scale_32: adjust range checkingJeremy Johnson
Fix up generated values for rescale tests Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I28fc3b8f189d25a7ad8e5172d4d8a43b86820fcf
2022-02-10Minor formatting fixes for pythonJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Iff6eb54734e7a78eb22d1a64803b937063b7f65a
2022-01-26Fix for max arguments to be respected in build testsJeremy Johnson
The conformance oversize tests are not produced by default. Change-Id: I9f7d4a60b9d8991889a077b61613e13e288bb65d Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-01-24Fix COND_IF binary INT8/16 test generationJeremy Johnson
Limit input values to allowed for logical shift operations. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I78110c449274ab96a3f824890c3f03a0eeb345eb
2022-01-20Fix for LOGICAL_LEFT/RIGHT_SHIFT shift valuesJeremy Johnson
Added missing reference model REQUIRE check for shift value (0-31) Make sure result of LOGICAL_SHIFT_LEFT is masked to input size Fixed test generation to produce shift values in that range Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id511de0d989ea954fc1afd18dc2051341bce2cd0
2022-01-13Add python pre-commit script checkersJeremy Johnson
Fix up issues in existing python scripts. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id4adab404560c3129c66f31c21ff0ce148283c73
2022-01-06Update tosa_verif_run_refJeremy Johnson
Rename to tosa_verif_run_tests to match build_tests Improve output and system under test support Improve xunit support Add results checker Add utilities json2numpy and json2fbbin Add set of python tests Update README.md Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ia09f8e6fd126579b3ba1c1cda95c1326802417ca
2022-01-06Reorganize verif and create packagesJeremy Johnson
Split generator and runner scripts Add package setup Add py-dev-env.sh/.bash to allow editing source files during dev Update README.md with installation info Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I172fe426d99e2e9aeeacedc8b8f3b6a79c8bd39d
2022-01-05Fix crash with rank mismatch negative testsEric Kunze
When testing rank mismatch tests, could be accessing beyond the end a tensor's shape. Change-Id: I304463e50579d574d629abd13098017a26a1406b Signed-off-by: Eric Kunze <eric.kunze@arm.com>
2021-12-08Stop COND_IF neg tests creating bad arraysJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I267fa624241f60f1fa613e27e4fe46fc43768f8a
2021-11-24Do not generate tests that fail validation checksLes Bell
Change-Id: I33237ebfd946b9ec91352c2b0dc6298cc113cd77 Signed-off-by: Les Bell <les.bell@arm.com>
2021-11-18WrongRank negative test gen additions & fixesJeremy Johnson
REDUCE ops - test generation now limited to ranks 1-4 and so WrongRank tests created SCATTER/GATHER ops - enable WrongRank testing Change-Id: I0909573e0edabddbad8764e7542d654967a5c8b9 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2021-11-18ERRORIF test generation fixesJeremy Johnson
TRANSPOSE - remove WrongRank testing as no limit in spec RESIZE - make floating point test naming same format as int - fix MaxDimExceeeded by not restricting dims and fix max dim check to match spec - catch stride less than/equal zero and regen PAD - remove zero point testing as not in spec Change-Id: I4eb0b78d7f577469d202866d95833e7d5fd2f2a8 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2021-11-18Convolutions ERROR_IF testsLes Bell
Signed-off-by: Les Bell <les.bell@arm.com> Change-Id: I68a13e1b337b1afc2ab5e0edcffda2b4b0cecdda
2021-11-11Add Broadcast DimensionMismatch errorsJeremy Johnson
Add RankMismatch and DimensionMismatch support for SELECT Update RankMismatch ops to also support DimensionMismatch Update POW op to have proper broadcast testing A few other broadcastable ops missing Rank/Dimension testing Change-Id: I6566f45a7a0db4f9f008456ea7a8e23d4192f4f9 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2021-11-09Fix Transpose WrongRank test and add new test for ConcatMatthew Haddon
* Transpose WrongRank tests now use ranks 7, 8 * Concat ERROR_IF checks now test for inaccurate summation of output shape tensor dimension Change-Id: If32f43a4dbd872d0ef7625fa3d4969c863a11b8c Signed-off-by: Matthew Haddon <matthew.haddon@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Signed-off-by: Les Bell <les.bell@arm.com>
2021-11-09Add negative testing to cond_if, while_loopMatthew Haddon
Signed-off-by: Matthew Haddon <matthew.haddon@arm.com> Signed-off-by: Les Bell <les.bell@arm.com> Change-Id: Ie6c8c8653874f9eed6007a54a3ad526601a4a669
2021-11-09Add ERROR_IF checks to operators without specific ERROR_IFsMatthew Haddon
* Operators implemented: sigmoid, tanh, arthmetic_right_shift, mul, table, select, equal, greater, greater_equal, concat, reverse, tile, scatter, gather, case * Note that over the course of implementation some specific ERROR_IF checks have been added for some of the above operators Change-Id: I80595e6eb9a3e5efd1cc6fd7aa28bbc2dd614980 Signed-off-by: Matthew Haddon <matthew.haddon@arm.com> Signed-off-by: Les Bell <les.bell@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2021-11-01RESIZE test generator updatesJeremy Johnson
Allow random shift values rather than just 11 Check for values that cause unpredictable behaviour Change-Id: I508c9d0982808ac03b8400d47ad9445d6afb1b79 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2021-10-28Changes for 0.23.0 releaseKevin Cheng
- update serialization_lib hash - PAD: 1. make padding as an attribute instead of tensor. 2. add pad_const_int (for non-float type) / pad_const_fp (for float type) - TRANSPOSE: make perm as an attribute instead of tensor - TABLE: make table as attribute instead of tensor - update examples/ tests Signed-off-by: Kevin Cheng <kevin.cheng@arm.com> Change-Id: Iddc446db4b356ba2f36ea4a79b7220b9cfc2aa4e
2021-10-25Limit tensor values for COND_IF/WHILE_LOOP tests to stop saturationJeremy Johnson
Change-Id: Idb36b1f1c0d78ec101c168865a9c8d03221b4c84 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2021-10-21Fix pool2d generation and check to match specificationJeremy Johnson
Change output size error_if check in ref model to match specification. Remove size check from test generation as output shape is always correctly calculated. Change-Id: I5be64f31e6448b47e80fc0a4af11bb312f366a26 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2021-10-18Add Negative tests for pad, reshape, slice, transposeMatthew Haddon
Signed-off-by: Matthew Haddon <matthew.haddon@arm.com> Change-Id: I659337aadfd0498bf88a95737f69c51efec797cc
2021-10-18Add negative testing support to fully_connected, matmul, argmaxMatthew Haddon
Change-Id: I75f2a4ab6790dcbdfaec064f42f601d8f44da70b Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>
2021-10-18Add negative testing support to RESCALEMatthew Haddon
* Negative tests for rescale op added Signed-off-by: Matthew Haddon <matthew.haddon@arm.com> Change-Id: I70aead1c6a67f159c7b7c9a05f7d5f0b92521584
2021-10-14Remove tosa_serializer.py and use the one from serialization_lib now.Kevin Cheng
Signed-off-by: Kevin Cheng <kevin.cheng@arm.com> Change-Id: Idd12aab4d52365f3137a403d362790abc16e66d0
2021-10-12COND_IF int8, int16 testsLes Bell
Change-Id: Id1c2d2a41bdfff6b3fe45f79bf99abd0b5beb704 Signed-off-by: Les Bell <les.bell@arm.com>
2021-10-12Add negative testing for avg_pool2d, max_pool2dMatthew Haddon
* Negative tests for ERROR_IFs given in spec * Constrict dimension size of latter ranks if rank is larger than 4 Signed-off-by: Matthew Haddon <matthew.haddon@arm.com> Change-Id: Iffea1874e876dba83c8a7c63049283bf7b3ba74b
2021-10-11Add negative testing support for reduce operatorsMatthew Haddon
* Negative testing support added for the following operators: reduce_all, reduce_any, reduce_max, reduce_min, reduce_product, reduce_sum Change-Id: Ife4f0307b681510e8910a23a39efc127c82ae120 Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>
2021-10-11Fix rank and dtype filtering for ops like conv3d & fully_connectedJeremy Johnson
Change-Id: Ic2aebe40b5cce61d4576a64f4f48ff87b36475c2 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2021-10-07Add negative testing support for ew_unary operatorsMatthew Haddon
* Added negative testing support for the following operators: abs, bitwise_not, ceil, clz, exp, floor, log, logical_not, negate, reciprocal, rsqrt Change-Id: Icc6f146c6407502520330678420951749ba2a9ef Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>
2021-10-07Add negative testing support to (most) EW Binary OpsMatthew Haddon
* Negative testing support for the following operators: ADD, BITWISE_AND, BITWISE_OR, BITWISE_XOR, INTDIV, LOGICAL_AND, LOGICAL_LEFT_SHIFT, LOGICAL_RIGHT_SHIFT, LOGICAL_OR, LOGICAL_XOR, MAXIMUM, MINIMUM, POW, SUB Change-Id: I2271f00b0b619604e864e36e4a4f987f1b2a37d4 Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>
2021-10-07Add ERROR_IF checks for mismatched batch/channelMatthew Haddon
Change-Id: I7c670c5f9b97a18a6f586b16f31bc9fc301f6bc3 Signed-off-by: Matthew Haddon <matthew.haddon@arm.com>
2021-10-07Separate positive and negative test generation and refactorMatthew Haddon
* Positive and negative tests are now produced entirely in isolation, if 'both' test types are chosen all positive tests are generated, then all the negative tests are generated, without combining the two * Moved tensor generation out of serialize test and into it's own function Signed-off-by: Matthew Haddon <matthew.haddon@arm.com> Change-Id: Id8e9023d2c2966a0b14ae83d2d848a66cb125fcb
generated by cgit v1.2.1 (git 2.23.0) at 2024-06-05 01:15:25 +0000