aboutsummaryrefslogtreecommitdiff
path: root/verif
AgeCommit message (Collapse)Author
2023-01-18Add framework tests for tf.sin and tf.cosLuke Hutton
Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: I342a162e252c8527d5b318ceaf5f1de70fa5dc34
2023-01-13Reference model update for control flow operators supportJerry Ge
Rationale for making this change: - In the original design, for control flow operators like WhileOp, child blocks couldn't read the tensor variables (global consts) in the root level block, this patch added the machanism for child blocks to access their parent level block's tensors. - This change also relies on another serialization change on adding another layer of abtraction called Region: - Serialization patch: [region] Add TosaSerializationRegion to serialization_lib - Updated the corresponding python version of the serialization code: TosaSerializerRegion to python version of serialization_lib - This change also relies on the TOSA MLIR Translator change: Add RegionBuilder to TOSA MLIR Translator - Added the WhileOp related test cases: While, LSTM, GRU, RNN - Other related fixes Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I13ae33628ad07e41d248e88652ce1328654694ab
2023-01-04Create MI tests for EW Binary: ADD, SUB, MULJeremy Johnson
Add option to control negative tests dim range to stop ERROR_IF test clashes between profiles. Change-Id: I1f806934480cc32b95e80613c66fecb1e6ff5426 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-01-04Create MI tests for EW Binary: POWJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I8d478b7bd13559ea37e27f77b05ee5db56c8f9e3
2022-12-15Add extra control flow ERROR_IF testsJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I7276dc686d8d18ba44663b73e35ceca2a1cbaadf
2022-12-15Create MI tests for Image: RESIZEJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I440590fa59433b9ddc1934188f2b34b377129c1f
2022-12-15Create MI tests for Activation: SIGMOID, TANHJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I75d6280ecb5219d50c91d14029e90ff3a2f6eb06
2022-12-14Create MI tests for EW Ternary: SELECTJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I1eb4fbadb9117e28f87a36ee828d65d896d834f0
2022-12-13Create MI tests for EW Unary: ABS, CEIL, FLOOR, NEGATEJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I2ee6d4f706958b6261fd41dd245a3d01d3f8cecf
2022-12-13Create MI tests for Reduction: REDUCE_MIN, REDUCE_MAXJames Ward
* Also include test_select improvements Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I306a8ed57975bc429d6cd5a8a24777b889593ddf
2022-12-09Create MI tests for EW Binary: MINIMUM, MAXIMUMJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I6824e30cece291fceaedd727132a6b211762baa9
2022-12-09Add framework tests for tfl.sin and tfl.cosLuke Hutton
Although the legalization is currently supported in the TFLite runtime, the tests were missing from the reference model. Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: Iae3cd54f145f69a84943867b418063a4d4109d87
2022-12-01Fix conformance desc.json contains correct profilesJeremy Johnson
Must seperate conformance generation by profile to stop desc.json being incorrectly overwritten by different profile information. Change-Id: Iead01c9fcc659e48ef1d350a860b03c2b0a05ffa Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-11-21Create MI tests for Data Nodes: CONST, IDENTITYJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I7280b3bf31aa377a6f6af7713627f34eb445e689
2022-11-21Create MI tests for Data Layout: CONCAT, PAD, RESHAPE, REVERSE, SLICE, TILE, ↵James Ward
TRANSPOSE Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I626d9ece0290c5104c9ac42d20be2f9db1e55fa7
2022-11-21Create MI tests for Scatter/Gather: SCATTER, GATHERJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I44c0843fa049546582ca1111795342d5035a329c
2022-11-21Create MI tests for Tensors: ARGMAX, MAX_POOL2DJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I908d88ddf7c52d98ce6d533396b5c439f15750f5
2022-11-19Initial set up of Main Inference conformance test genJeremy Johnson
tosa-verif-build-tests - option for setting FP values range - option for recursively finding tests - change from os.path to Path tosa_verif_result_check - option to supply FP tolerance - output difference and max tolerance on contents mismatch - change from os.path to Path MI conformance - contains examples of AVG_POOL2D and CONV2D tests Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I8e1645cd8f10308604400ea53eef723ca163eed7
2022-11-11Add framework test for tfl.preluTatWai Chong
Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: Ib894e50f4b869a3f4936885704995a417ea25eef
2022-11-09Add BF16 support to reference modelJames Ward
* Upgrade Eigen to 3.4.0 (for bfloat16 support) and add work- arounds for reduce.any() and reduce.all() bugs (introduced between 3.3.7 and 3.4.0) * Truncation to bfloat16 now performed in eval() methods Signed-off-by: James Ward <james.ward@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If5f5c988d76d3d30790acf3b97081726b89205fe
2022-10-26Add TILE bool tests & rename CAST/RESCALE testsJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If9639a94c8e7322d470a1ca97e9e95d821683629
2022-10-13Rename FLOAT type to FP32Jeremy Johnson
Update tensor operations naming to state input type as TxT in all cases. Effects CONV2D, CONV3D, DEPTHWISE_CONV2D, FULLY_CONNECTED, TRANSPOSE_CONV2D. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ic959acfcb3aa0a910b33b774a5a85fac08219205
2022-10-13Minor fixes & add FP16 support to refmodel testing and conformance genJeremy Johnson
Improve base inference conformance generation: * Change to target specific dtypes required for conformance tests. * Reduce dimension sizes of ERROR_IF tests. NOTE: Will impact tensor, clamp and all ERROR_IF tests. Add option to change seed on conformance generation for extra testing. Stop creation of convolution tests with negative output dimensions. Improve reporting on failing to allocate tensor due to above issue. Fix runner to correctly pass ref model debug flags. Update reference_model examples. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I992180dcfe265a7d50edfb151c9f38eeaef5c369
2022-10-11Reference model changes for fp16 supportJames Ward
Change-Id: I72f21fcfa153046274969d327313e3349981dbe6 Signed-off-by: James Ward <james.ward@arm.com>
2022-09-26Add CONST testing to Numpy refmodel testsJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id32f3b7c287c9e8b0fd42dd96922be5a255598e2
2022-09-22Add framework test for TF and TFL mirrorpadTatWai Chong
Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: Icc9b8f6a65ee54ddbb445c3a999ca49401b660c2
2022-09-21Add simple post commit reference model testing against NumpyJeremy Johnson
Only perform testing of this after build of ref model using postcommit pytest mark. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I771a18d2c9cd4051fecafad3e6079b44f2ed62fa
2022-08-30Update RESCALE testing to match no rank restrictionsEric Kunze
Matches changes to the specification loosening the rank requirements for the RESCALE operation. Includes removing wrong rank testing for RESCALE, and increasing conformance testing to rank 5 Signed-off-by: Eric Kunze <eric.kunze@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I4e15d4bd8f448e1ec81d9a8bbcd231c286b64a30
2022-08-29[Fix] Wrong dimension is inferred in tensor shape generationTatWai Chong
Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: I0bccfbe971f64986d71cef5a1d68daa7eb1697c4
2022-08-29Update framework test generator to support TF/TFL conv3d.TatWai Chong
Add a new attribute `rank` to indicate the testing dimension range of input tensor. Also fix a minor bug in the existing conv3d simulation. And relax rescale operator in the reference model to support 5-D input. Change-Id: Ib42fe513831dc83eb7f9af07e011787a6c752704 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2022-08-26Align padding for transpose_conv2d to match specEric Kunze
Increasing out pad values now leads to increasing pad. Reference model changes, and test generator changes to match specification definition Change-Id: I4f3ebfbca5048354fb15bedc7ab640ff28ed853a Signed-off-by: Eric Kunze <eric.kunze@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-08-25Add generator functions to support TFL geluTatWai Chong
Change-Id: I1056f4a3abe9079381fb2c7d02edf11312aba9d6 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2022-08-25Add PAD ERROR_IF test for output shapeJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I25a13540734fa30c0c21b46708dfabbec8c4d1e5
2022-08-24Enabled 16-bit TABLE REQUIRE statementJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ib6e81814e022f33e45430e47ca99d6d9f9e0e101
2022-08-22Simplify the argument parsing and debug codeEric Kunze
Change-Id: Ia681aa0eb1d42fa31a4658badb376f7b32b2e338 Signed-off-by: Eric Kunze <eric.kunze@arm.com>
2022-08-11Update RESIZE operator in test generator for spec updatesJeremy Johnson
* Add common screen aspect ratios with borders to random pool of tests * Add up/downscaling to random pool Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Iee8e3f5ed6bd5c941816474df20a7fd433646d6b Signed-off-by: James Ward <james.ward@arm.com>
2022-08-10Update TOSA resize to match specificationTatWai Chong
Attribute stride and shift are removed, and has new scale and border. Also add tests in the generator to test tf.resize with all option combinations. Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If0f330d04395762d2d907863235eda1532f5e1ff
2022-08-02Added framework tests for Relu1 Operator legalizationJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I36ea46e79a0ce42c2dbe47c816b5b5572058c0f2
2022-07-14Add profile to all created conformance tests.Jeremy Johnson
Add supported profiles for each test in convert2conformance and tosa_verif_conformance_generator. Enable filtering of profile tests on running in tosa_verif_run_tests. Reorganize arguments in conformance_generator to have more important ones first. Change-Id: Ie6e5b68727adb3c39b04aa482dd6433788f7bcc9 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-07-12Add conformance generator scriptsJeremy Johnson
Change-Id: I5cb16ea1f47ee454f03a1d5182827e3fd9f49128 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-07-11Modify input limits for apply_scale_32Eric Kunze
Aligns with change in specification, as the limit to value being between (-1 << shift-2) and (1 << shift-2) was overly constraining. Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I72cc5743344d4036920c3df366e0a3930701a0cb
2022-06-15Remove quantization info from serialization attributesEric Kunze
Any needed information moves into the attributes for each operator. New serialization library version removes teh quantization information attributes from the schema Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: Icf6165687ab1fd34a01f64c01b0b92b2820e72fa
2022-06-13Update transpose_conv2d to align with TOSA specTatWai Chong
Rename outpad to out_pad, and also fix the dilation in the generator. Change-Id: I4c1599871f0d0b41856e819d8c644a85ca6d8267 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2022-05-31Remove RESHAPE -1 dimensions supportJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I098daf49c92da12c07143cdd23ac9bb58acebbb9
2022-05-26Add support for uint16_t to RESCALEJeremy Johnson
Update ref-model RESCALE op to support UINT16 conversions Add testing for RESCALE UINT16 and ERROR_IFs Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ic6e6e53de1f0b054bedb9e6ba3856e7475498aba
2022-05-19Update framework test generation for ERROR_IF criteriaJeremy Johnson
Update to tosa_verif_framework_generator to produce valid test ranges for pooling and convolution tests Fix up test filtering to only filter on test name not output directory Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ifff7e7604a37e8680d7237dc2d85cd806b20e384
2022-05-04Fix for NEGATE using 32-bit accumulatorJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie5d119dc317303a0d2a71d018ac94ce6800ecbf5
2022-05-04Control oversized tests for TRANSPOSE_CONV2DJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I464871b412b3dfc9a84bd3d8fffe4c1de3072b69
2022-04-28Update tensor ops ERROR_IF criteriaJeremy Johnson
Update to ref model to check ERROR_IF criteria for pooling and convolution ops to match specification Update to tosa_verif_build_tests to produce valid test ranges and new ERROR_IF tests Plus update pooling ops big kernel to 9 (from 6) for better testing coverage and set dilation to 1 and add out_pad bottom & right for transpose_conv2d to match specification Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ic5759872d40ae8d3f3d07043d9a0f2fa0244d72e
2022-04-11Refactor verif/generator/tosa_test_gen.py into different filesJeremy Johnson
Move all error & validation into tosa_error_if.py Move all argument and tensor generation into tosa_arg_gen.py Move utility functions into tosa_utils.py Create new TosaTensorValuesGen class for specialising tensor value generation. Change-Id: Ib9ac65e2308b14471a567c6f11d775c76585bc5b Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
generated by cgit v1.2.1 (git 2.23.0) at 2024-07-10 03:18:23 +0000