aboutsummaryrefslogtreecommitdiff
path: root/verif
AgeCommit message (Collapse)Author
2023-08-02Update Framework compiler scriptEric Kunze
Reuse the name deduced from the test run in results. Also move most string formatting to f strings. Move path handling to pathlib from os.path Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I44c7786d4b7af44306e218ff49608df35d4521a7
2023-07-31Update serialization library submoduleJeremy Johnson
* Update Flatbuffers version * Allows consts to be saved as inputs Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ibb9e38d381f8229bfca65b559e35c9ded889c384
2023-07-26Update framework tools to add name in jsonEric Kunze
Name of the test can now come from the json description rather than inferred from the directory. Old method remains to keep functionality with older tests Change-Id: I79f872103ae6a101b5cc799af91c933839f28dfb
2023-07-26Add Verif Conformance READMEJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If30b329905aad9e520e4a9a3ac1dffccd37bb0e9
2023-07-25Update flatbuffer version and fix json2numpy testEric Kunze
Temporarily disable the json to numpy pytest Otherwise will load as the wrong dtype Also update the flatbuffer module version to match tosa_mlir_translator Change-Id: I40999f631adb1b230bc8a8b891e0955dd67371f0 Signed-off-by: Eric Kunze <eric.kunze@arm.com>
2023-07-11Fix size mismatch of placeholders for broadcast fuzzingWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Ib954db4f8f2fd5008e95d5651a6e84a8f5b7161d
2023-07-06Add reference model framework test for INT8 squared differenceWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: I6fba6907cef0616c18dc461dbb92d2aceb582f6c
2023-07-06Enable framework generator to save numpy files in native dtypesJerry Ge
With the following patch merged, the tosa serialization_lib is able to read u/int8, u/int16 in their native dtypes: https://review.mlplatform.org/c/tosa/serialization_lib/+/9871 Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I5e63fb70b055cc5ee33e7dc26237eef9bedeac30
2023-07-05Add a parameter to tensor generation function to disable fuzzingWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Iff31b33b818a181371904915d5477a169513aa2e
2023-06-29Add support for ERF operator to reference modelWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Ib42b867287b83a183a0d0fb1f1eb29974f58fae4
2023-06-21Add tf broadcast-to testingTai Ly
This patch adds BoradcastTo Op to the tf tests. Did not add tflite testing because the tf.lite.TFLiteConverter converts tf.broadcast-to to tfl.mul by 1. Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: Icd372e619c318121c19eca87d5716bcd5fbbbb23
2023-06-15Fix framework test generator with custom_shapesJerry Ge
shape_list is global while the custom_shape is only applicable for a single operator, need to make a copy for the whole shape_list when dealing with custom_shapes Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I2c5b87cb6fcaa153b0ccfe5827bae814b9cfe2b9
2023-06-15Add ERROR_IF to incorrect broadcast shapesJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I7460ad9eed3ed5c7cec6e855a0303753ed28eb1c
2023-06-15Add accumtype to tf/tfl framework testsJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Ie45cc27433f5dbce3fadc90014dc5cc8e36a9950
2023-06-02Support custom_shapes for framework test generationJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Ia29c73cb5d0a7f91914e2a94ca52d06f375722e9
2023-05-17Add support for one dimension of size -1 in ReshapeOpJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I0ef7607f4266296a1204c5cccdb5be36f345b5ba
2023-05-168K level: rank 6 ARGMAX testsJeremy Johnson
Change-Id: I81097ae263a64ee8e35b8bd798598671ac089c3f Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-05-10Refactor ref_model rank checking and add level check to argmaxJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Iad035b31d5e5e83040068e6311501490765bfff7
2023-05-05[reference model] Add precise modeTai Ly
This adds --precise_mode=1 option to tosa_referece_model, which will cause reference model to convert all floating point tensors to FP64 tensors and compute all operators accordingly. Also adds optional -p arguments to test runners tosa_verif_run_tests.py and tosa_verif_framework_compiler_runner.py to run tests in precise mode Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I156055216ad61710096497a8fa1a653be2a602a3
2023-04-268K levels: Tensor op tests kernel/stride at 8192 maximumsJeremy Johnson
Operators updated: AVG_POOL2D, MAX_POOL2D, CONV2D, CONV3D, DEPTHWISE_CONV2D & TRANSPOSE_CONV2D tosa_verif_build_tests argument --level-8k-sizes used to allow kernel/stride maximum boundary testing Fixed bugs in height/width validator function meaning some esixting avg_pool2d float tests need regening. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I7aeab82d3bd3c49d02d54708f2c9d995cd3cf2df
2023-04-258K level: RESIZE scaling at 64 maximumJeremy Johnson
Add support for producing scaling 64 tests in RESIZE and make sure default tests are not generated with scaling greater than 64. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I3e99cde7e6bec6ea7854a139c6fc2ed151af5b1e
2023-04-20Add level checking to TOSA Ref modelJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I5689d7c6b902a319a68fa4628b59e0bcc23aeca4
2023-04-13Add framework tests for tfl.real and tfl.imagLuke Hutton
Change-Id: I665acac9b5171efd0c5a2b68b516609048f6e187 Signed-off-by: Luke Hutton <luke.hutton@arm.com>
2023-04-118K level: rank 6 tests support using generator groupsJeremy Johnson
Split generation of conformance tests into groups to avoid a change in existing tests selection * A standard group is used for the normal/default conformance tests * An 8k level group covers the extension of testing for 8k level * Other arbitrary groups can be created and the group names will be used to tag the tests, so that in the future they could be filtered Group selection criteria is in a new dictionary * A default selection config is used for generation groups * But a different config can be used for a particular generation group Added parameter sparsity for pad of rank6 and above, and further parameter sparsity options to test_select to limit number of tests selected. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ib9f1185c0ae6d0957df2116411c17e958157c6d3
2023-03-30Add positive/negative random number generator for RsqrtJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I1e9e97ead447295e1252785106931b261df7bcea
2023-03-28Add CONST INT48 support to conformance.Jeremy Johnson
Remove ERRORIF tests for CONST as they do not create valid negative tests. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ia96b184f34010a452d2c4848b0653514352c751e
2023-03-13Update framework test choices - SPLIT, RESIZEv0.60.2Jeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ifce52ef0e082e9e6289938baa6c73a6ebd9ded62
2023-03-09Remove 1x7x7x479 from the test size listv0.60.1Eric Kunze
Was causing the generated set of tests to be too big. Change-Id: I45292e9317ab93786a1ec4670283385d06d52c0d Signed-off-by: Eric Kunze <eric.kunze@arm.com>
2023-03-08Prevent generating test case when align_corners and half_pixel are both true ↵Jerry Ge
in nearest mode Note: - align_corners and half_pixel can both be true in nearest mode for tfl but not tf - this is inconsistent between tf and tfl Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Ib396f994874904e017e0d696f70370bd51d60161
2023-03-03Add framework tests for TFL.Relu0To1OpJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I6fa14e9dbe7c0cd21156a5779d68fd473af322f2
2023-03-03Refactor resize test builderTatWai Chong
Also add input size = 1 in the shape list, and extend scaling to 1x, 2x and 3x, so that the cases of broadcasting, power-of-two scaling, no-scaling (e.g. 1x1 -> 1x1), scaling accuracy (3x) can be tested. Since the scalar tensor is tiny, should not noticeably impact the execution time of the framework test. Change-Id: Iec53da3cbb60e087077d6e2d8eb205e76e6c1313 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2023-03-02Add test case against Add Quantized Int8Jerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: If18fc61514cbc2d31a15ed7dd87147b0d8e69f16
2023-03-02Extend reduce_all to test with tfliteTatWai Chong
Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: Ifb3ed9c448005d6d53c5d0938b5146003a26e8a3
2023-03-01CONV2D/CONV3D/DEPTHWISE_CONV2D Test Regen for ofm_depth bugfixJames Ward
* Restricting size of tensors due to expanded values of ofm_depth Signed-off-by: James Ward <james.ward@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I582d8f0073c2d01244976f8e2e14783b506d8091
2023-02-28Update rank limits for SLICE, TILE and TRANSPOSELuke Hutton
Updated to align with corresponding changes to the spec. In addition, some ERROR_IF tests have been updated to match the checks specified by the spec, including: PAD, SLICE, TILE, TRANSPOSE. Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: Ie2c5f48e79a5610eb82739170e25057a63dac1d8
2023-02-24Extend pad tests to include quantized type and explicit pad value.TatWai Chong
Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: I4a49f45aa73044aff5b0a8b3dba58c1f52c1ae21
2023-02-22Create MI tests for Tensors: TRANSPOSE_CONV2DJames Ward
Change-Id: I4f967ce2dfb0cb44dc58307170fd9306a0ea1ed5 Signed-off-by: James Ward <james.ward@arm.com>
2023-02-22Bug fixes for max-batch-size/ofm-depthJames Ward
* Conformance regen required for tosa-bi: conv2d conv3d depthwise_conv2d transpose_conv2d * Include bug fix for testGen.makeShape() * Include json changes to limit size of tests which now have ofm_depth > 1 Signed-off-by: James Ward <james.ward@arm.com> Change-Id: Ic8221b9a742f5737df523997bee7260f4dfcdef3
2023-02-22Create MI tests for Tensor: FFT2D & RFFT2DJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I44ad6513fad831ff50fcc5ab801b46a5bf25d0a8
2023-02-22Create MI tests for Tensor: CONV3DJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I2c1e87ec586195f82e97bbc70e4e2bc18cd2cc88
2023-02-22Create MI tests for Tensors: DEPTHWISE_CONV2DJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I6c2087c94c2c024aabc2086d40cbca9c719c69dd
2023-02-22Add tests for INT48 CONST supportLuke Hutton
* Add TOSA test coverage * Add some basic error checking Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: Ib02df7319954a2dcb915e5f5a70699ca398fd483
2023-02-14Select new TFLite SPLIT testsJeremy Johnson
Due to change in generator creating different test patterns. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I58224dc117c766b22fccd418f64004aea31905b5
2023-02-10Add FFT2d to the reference modelLuke Hutton
Includes: * FFT2d reference implementation * Basic TOSA tests Change-Id: Ie79fcb713542345d550ec013646810c1e890e388 Signed-off-by: Luke Hutton <luke.hutton@arm.com>
2023-02-07Adjust ERROR_IF tests generated for tosa-mi profileJeremy Johnson
So they do not overwrite tosa-bi tests, ops affected: NEGATE, TILE, REDUCE_MIN, REDUCE_MAX, RESIZE, SCATTER, GATHER Update test-select exclude-patterns to work with ERRORIF tests. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Idb0d26512ce9a48f6d868663a630e343c8785899
2023-02-06Create MI tests for Tensors: AVG_POOL2D, FULLY_CONNECTED, MATMULJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I874c87745b929f9808a5e45490042420a57b22f7
2023-02-06Create MI tests for Tensor: CONV2DJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Iefd8490d9eedc76d199d36943b87c99e4dd97612
2023-02-02Remove accumulator attributes from all but AVG_POOL2DJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: If67f503a1848967bc1671646c3011d055b622c52
2023-02-02Create MI tests for Reduction: REDUCE_SUM, REDUCE_PRODUCTJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I21b8f8139ce069d5fc6bbeabe292e3a44d99fe10
2023-01-31Create MI tests for Type Conversion: CASTJames Ward
* Add exclusion regex's to conformance generation Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I15bef7451efd5662065060242d35bd7fa3381487
generated by cgit v1.2.1 (git 2.23.0) at 2024-07-03 07:11:00 +0000