aboutsummaryrefslogtreecommitdiff
path: root/verif/generator/tosa_error_if.py
AgeCommit message (Collapse)Author
2024-04-18[reference_model] Remove output_shape from transpose_conv2dSuraj Sudhir
Signed-off-by: Suraj Sudhir <suraj.sudhir@arm.com> Change-Id: Ib2b95e73b226d64c4db5ad1ed22c123e04d7e6f9
2024-03-21Reduce WrongRank tests to the minimum neededJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ideeb67ec4b33b9c6922d6765f4b0eb5ca69bdf89
2024-03-20Fix missing/broken ERROR_IF testsJeremy Johnson
Fix CONV2D WrongOutputType FP32 & Pad/Stride/DilationSmallerZero issues. Fix PAD WrongInputType. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I57fc57c43e63685e05bf5e3d562c3167411fd57b
2024-03-20[ref model] Add acc_type to Conv OpsTai Ly
This patch implements changes required by the new acc_type field in ConvAttribute and TransposeConvAttribute Signed-off-by: Tai Ly <tai.ly@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ib13dbeec4d8920e0ddbcca02b727e7277f2c8d62
2024-03-19Updated build_tests to support different random generatorsJeremy Johnson
All generator functions now take RNG argument to allow different random number generators, rather than relying on global RNG Default behaviour is the same as before using global RNG Added stable random generation mode * shape rng based on operator, rank and datatype * arguments rng based on operator, shape and datatype * build operands and data rng based on op, shape, datatype and args Add optional stable RNG test generation to conformance_generator Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I5ee4ff85575a81177fd74ed1617e946bfa3a0769
2024-03-12Improved test selection before test generationJeremy Johnson
Add test list output to tosa_verif_build_tests and test list capture to file for tosa_verif_conformance_generator Improve PAD & CONV2D test coverage for tosa-mi conformance Change to use logging for output to hide info from test lists Tweak verbosity levels of tosa_verif_conformance_generator Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ic29da5776b02e9ac610db6ee89d0ebfb4994e055
2024-02-21Add support for FP8 to reference modelWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: I99b70f94aff2ccd4af64875697e124eb60bc5b08
2024-02-08Improve Avg_Pool2D ErrorIf TestingJeremy Johnson
* Add test for invalid accumulator dtype Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I506e2047623372670b82db6e9c0010fa89802851
2024-02-08Main Compliance: Add RESIZE supportJeremy Johnson
Add RELATIVE verify mode for RESIZE. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I4fe352579507211dae7a048bf080c24426ce42a2
2024-02-01[testing] Switch to using shape type for start and size of sliceTatWai Chong
The type of both start and size of slice are tosa shape type since 0.90. Update the type of these arguments and its usage in testing. Change-Id: Iff582982ad55d64a792183c6d46c9266510f924c Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2024-01-30Main Compliance: TRANSPOSE_CONV2D supportJeremy Johnson
Update data generator for main compliance values. Add test generation support. Fixed test set by including large 65k tests that were missing. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I8668c774e01c17e5d999aadf99c317e2dd893857
2024-01-24Add conformance testing for shape operatorsWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Ie80570146601c470a3be7c04a9d6e1016a7c547c
2023-12-14Remove inferred dimension from RESHAPEJeremy Johnson
Test generation changed to only produce static reshape tests Reference model changed to produce ERROR_IF on inferred shapes Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I92c92a40e7c0e457961bc654630040dff79a750b
2023-12-14Main Compliance testing for SCATTER and GATHERJeremy Johnson
Added indices shuffling and random INT32 support to generate lib with testing of these new random generator modes Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I058d8b092470228075e8fe69c2ededa639163003
2023-11-16Main Compliance testing for simple UNARY opsJeremy Johnson
For RECIPROCAL, RSQRT, CEIL, FLOOR, ABS, NEGATE & IDENTITY. Improved ULP informational output. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I49644573b4c9a30b2b9d6c9624f2a1d46976a378
2023-10-26Compliance testing support for MAX_POOL2D & PADJeremy Johnson
Added Pseudo Random number generator in generate library. Enabled MAX_POOL2D, PAD FP32 tests to use new generator and compliance. Fixed verify library exact mode to expect reference data as FP64. Simplified tosa_verif_build_tests internal interfaces for new tests. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Icc0ffa924cf38107c3a212efd452c47a650c9d98
2023-06-15Add ERROR_IF to incorrect broadcast shapesJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I7460ad9eed3ed5c7cec6e855a0303753ed28eb1c
2023-05-17Add support for one dimension of size -1 in ReshapeOpJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I0ef7607f4266296a1204c5cccdb5be36f345b5ba
2023-04-268K levels: Tensor op tests kernel/stride at 8192 maximumsJeremy Johnson
Operators updated: AVG_POOL2D, MAX_POOL2D, CONV2D, CONV3D, DEPTHWISE_CONV2D & TRANSPOSE_CONV2D tosa_verif_build_tests argument --level-8k-sizes used to allow kernel/stride maximum boundary testing Fixed bugs in height/width validator function meaning some esixting avg_pool2d float tests need regening. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I7aeab82d3bd3c49d02d54708f2c9d995cd3cf2df
2023-02-28Update rank limits for SLICE, TILE and TRANSPOSELuke Hutton
Updated to align with corresponding changes to the spec. In addition, some ERROR_IF tests have been updated to match the checks specified by the spec, including: PAD, SLICE, TILE, TRANSPOSE. Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: Ie2c5f48e79a5610eb82739170e25057a63dac1d8
2023-02-10Add FFT2d to the reference modelLuke Hutton
Includes: * FFT2d reference implementation * Basic TOSA tests Change-Id: Ie79fcb713542345d550ec013646810c1e890e388 Signed-off-by: Luke Hutton <luke.hutton@arm.com>
2023-01-31Create MI tests for Type Conversion: CASTJames Ward
* Add exclusion regex's to conformance generation Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I15bef7451efd5662065060242d35bd7fa3381487
2023-01-24Add RFFT2d to the reference modelLuke Hutton
Includes: * RFFT2d reference implementation * TFLite framework tests * Basic TOSA tests * Serialization submodule upgrade with support for FFT/RFFT Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: I2a687e9cf87fb62a26160ea52439ba9830bea36e
2022-12-15Add extra control flow ERROR_IF testsJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I7276dc686d8d18ba44663b73e35ceca2a1cbaadf
2022-11-09Add BF16 support to reference modelJames Ward
* Upgrade Eigen to 3.4.0 (for bfloat16 support) and add work- arounds for reduce.any() and reduce.all() bugs (introduced between 3.3.7 and 3.4.0) * Truncation to bfloat16 now performed in eval() methods Signed-off-by: James Ward <james.ward@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If5f5c988d76d3d30790acf3b97081726b89205fe
2022-10-13Rename FLOAT type to FP32Jeremy Johnson
Update tensor operations naming to state input type as TxT in all cases. Effects CONV2D, CONV3D, DEPTHWISE_CONV2D, FULLY_CONNECTED, TRANSPOSE_CONV2D. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ic959acfcb3aa0a910b33b774a5a85fac08219205
2022-10-11Reference model changes for fp16 supportJames Ward
Change-Id: I72f21fcfa153046274969d327313e3349981dbe6 Signed-off-by: James Ward <james.ward@arm.com>
2022-08-26Align padding for transpose_conv2d to match specEric Kunze
Increasing out pad values now leads to increasing pad. Reference model changes, and test generator changes to match specification definition Change-Id: I4f3ebfbca5048354fb15bedc7ab640ff28ed853a Signed-off-by: Eric Kunze <eric.kunze@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-08-25Add PAD ERROR_IF test for output shapeJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I25a13540734fa30c0c21b46708dfabbec8c4d1e5
2022-08-11Update RESIZE operator in test generator for spec updatesJeremy Johnson
* Add common screen aspect ratios with borders to random pool of tests * Add up/downscaling to random pool Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Iee8e3f5ed6bd5c941816474df20a7fd433646d6b Signed-off-by: James Ward <james.ward@arm.com>
2022-06-15Remove quantization info from serialization attributesEric Kunze
Any needed information moves into the attributes for each operator. New serialization library version removes teh quantization information attributes from the schema Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: Icf6165687ab1fd34a01f64c01b0b92b2820e72fa
2022-06-13Update transpose_conv2d to align with TOSA specTatWai Chong
Rename outpad to out_pad, and also fix the dilation in the generator. Change-Id: I4c1599871f0d0b41856e819d8c644a85ca6d8267 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2022-05-26Add support for uint16_t to RESCALEJeremy Johnson
Update ref-model RESCALE op to support UINT16 conversions Add testing for RESCALE UINT16 and ERROR_IFs Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ic6e6e53de1f0b054bedb9e6ba3856e7475498aba
2022-04-28Update tensor ops ERROR_IF criteriaJeremy Johnson
Update to ref model to check ERROR_IF criteria for pooling and convolution ops to match specification Update to tosa_verif_build_tests to produce valid test ranges and new ERROR_IF tests Plus update pooling ops big kernel to 9 (from 6) for better testing coverage and set dilation to 1 and add out_pad bottom & right for transpose_conv2d to match specification Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ic5759872d40ae8d3f3d07043d9a0f2fa0244d72e
2022-04-11Refactor verif/generator/tosa_test_gen.py into different filesJeremy Johnson
Move all error & validation into tosa_error_if.py Move all argument and tensor generation into tosa_arg_gen.py Move utility functions into tosa_utils.py Create new TosaTensorValuesGen class for specialising tensor value generation. Change-Id: Ib9ac65e2308b14471a567c6f11d775c76585bc5b Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-01-13Add python pre-commit script checkersJeremy Johnson
Fix up issues in existing python scripts. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id4adab404560c3129c66f31c21ff0ce148283c73
2022-01-06Reorganize verif and create packagesJeremy Johnson
Split generator and runner scripts Add package setup Add py-dev-env.sh/.bash to allow editing source files during dev Update README.md with installation info Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I172fe426d99e2e9aeeacedc8b8f3b6a79c8bd39d
generated by cgit v1.2.1 (git 2.23.0) at 2024-05-19 17:48:00 +0000