aboutsummaryrefslogtreecommitdiff
path: root/verif
AgeCommit message (Collapse)Author
2023-12-15Update conformance tests to use new rescale attributesEric Kunze
Determine input/output signed/unsignedness appropriately. Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I83d95f37803dd9d0ab27be6785523271ad24cf99
2023-12-06[reference_model] add local_bound attribute to test_genTai Ly
Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I9b87d1ad0f09648d07a6ecfc2075a174a9d04a3d
2023-11-28[reference_model] Support StatefulOps and the tests for CallOnceOpTai Ly
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I03cb878736ccd7e1f5e1f780d7171949a19a9de2
2023-08-31[reference model] Fix dim test generationTai Ly
In tosa_test_gen.py, change output_shape for dim operators to [] instead of [1] Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I8e7901e02831e9a467bce17e55e9379cb5f1b739
2023-08-30Fix conformance regressions due to type and region changes.Jeremy Johnson
Compress flatbuffer JSON files for conformance to allow while_loop tests to fit in the 30MB file size limit. Do not include new SHAPE type in usable DTypes so that selection of ERROR_IF tests are not changed. Also enable strict json mode for flatbuffers. Change-Id: Id89a9963244c34769a4662f9951adde0aa3334d9 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-08-18Add DIM operator to reference modelWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Iea11ee5d3d98773e9c5e9b827593c05afb41ce3b
2023-08-16Remove `"` from the search pattern `"tosa.*"`TatWai Chong
The double quotation mark is not always appear in the assembly output of operations. Remove it to easier capture general printing pattern. Change-Id: I59301daa07abd905de7259f8ae527a74b1ce5a6c Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2023-08-08Re-write json2numpy testsJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I17227adfca203f353e51ae9297f7e01f8d7edbe9
2023-08-07Deprecation of parts of TOSA conformanceJeremy Johnson
* Add deprecation notice for conformance_tests repo * Add deprecation notice for framework tests in conformance Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I92b17f4d6d993e47f30d7996a18fc13b934c4cdf
2023-08-02Update Framework compiler scriptEric Kunze
Reuse the name deduced from the test run in results. Also move most string formatting to f strings. Move path handling to pathlib from os.path Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I44c7786d4b7af44306e218ff49608df35d4521a7
2023-07-31Update serialization library submoduleJeremy Johnson
* Update Flatbuffers version * Allows consts to be saved as inputs Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ibb9e38d381f8229bfca65b559e35c9ded889c384
2023-07-26Update framework tools to add name in jsonEric Kunze
Name of the test can now come from the json description rather than inferred from the directory. Old method remains to keep functionality with older tests Change-Id: I79f872103ae6a101b5cc799af91c933839f28dfb
2023-07-26Add Verif Conformance READMEJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If30b329905aad9e520e4a9a3ac1dffccd37bb0e9
2023-07-25Update flatbuffer version and fix json2numpy testEric Kunze
Temporarily disable the json to numpy pytest Otherwise will load as the wrong dtype Also update the flatbuffer module version to match tosa_mlir_translator Change-Id: I40999f631adb1b230bc8a8b891e0955dd67371f0 Signed-off-by: Eric Kunze <eric.kunze@arm.com>
2023-07-11Fix size mismatch of placeholders for broadcast fuzzingWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Ib954db4f8f2fd5008e95d5651a6e84a8f5b7161d
2023-07-06Add reference model framework test for INT8 squared differenceWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: I6fba6907cef0616c18dc461dbb92d2aceb582f6c
2023-07-06Enable framework generator to save numpy files in native dtypesJerry Ge
With the following patch merged, the tosa serialization_lib is able to read u/int8, u/int16 in their native dtypes: https://review.mlplatform.org/c/tosa/serialization_lib/+/9871 Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I5e63fb70b055cc5ee33e7dc26237eef9bedeac30
2023-07-05Add a parameter to tensor generation function to disable fuzzingWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Iff31b33b818a181371904915d5477a169513aa2e
2023-06-29Add support for ERF operator to reference modelWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Ib42b867287b83a183a0d0fb1f1eb29974f58fae4
2023-06-21Add tf broadcast-to testingTai Ly
This patch adds BoradcastTo Op to the tf tests. Did not add tflite testing because the tf.lite.TFLiteConverter converts tf.broadcast-to to tfl.mul by 1. Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: Icd372e619c318121c19eca87d5716bcd5fbbbb23
2023-06-15Fix framework test generator with custom_shapesJerry Ge
shape_list is global while the custom_shape is only applicable for a single operator, need to make a copy for the whole shape_list when dealing with custom_shapes Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I2c5b87cb6fcaa153b0ccfe5827bae814b9cfe2b9
2023-06-15Add ERROR_IF to incorrect broadcast shapesJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I7460ad9eed3ed5c7cec6e855a0303753ed28eb1c
2023-06-15Add accumtype to tf/tfl framework testsJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Ie45cc27433f5dbce3fadc90014dc5cc8e36a9950
2023-06-02Support custom_shapes for framework test generationJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Ia29c73cb5d0a7f91914e2a94ca52d06f375722e9
2023-05-17Add support for one dimension of size -1 in ReshapeOpJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I0ef7607f4266296a1204c5cccdb5be36f345b5ba
2023-05-168K level: rank 6 ARGMAX testsJeremy Johnson
Change-Id: I81097ae263a64ee8e35b8bd798598671ac089c3f Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-05-10Refactor ref_model rank checking and add level check to argmaxJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Iad035b31d5e5e83040068e6311501490765bfff7
2023-05-05[reference model] Add precise modeTai Ly
This adds --precise_mode=1 option to tosa_referece_model, which will cause reference model to convert all floating point tensors to FP64 tensors and compute all operators accordingly. Also adds optional -p arguments to test runners tosa_verif_run_tests.py and tosa_verif_framework_compiler_runner.py to run tests in precise mode Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I156055216ad61710096497a8fa1a653be2a602a3
2023-04-268K levels: Tensor op tests kernel/stride at 8192 maximumsJeremy Johnson
Operators updated: AVG_POOL2D, MAX_POOL2D, CONV2D, CONV3D, DEPTHWISE_CONV2D & TRANSPOSE_CONV2D tosa_verif_build_tests argument --level-8k-sizes used to allow kernel/stride maximum boundary testing Fixed bugs in height/width validator function meaning some esixting avg_pool2d float tests need regening. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I7aeab82d3bd3c49d02d54708f2c9d995cd3cf2df
2023-04-258K level: RESIZE scaling at 64 maximumJeremy Johnson
Add support for producing scaling 64 tests in RESIZE and make sure default tests are not generated with scaling greater than 64. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I3e99cde7e6bec6ea7854a139c6fc2ed151af5b1e
2023-04-20Add level checking to TOSA Ref modelJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I5689d7c6b902a319a68fa4628b59e0bcc23aeca4
2023-04-13Add framework tests for tfl.real and tfl.imagLuke Hutton
Change-Id: I665acac9b5171efd0c5a2b68b516609048f6e187 Signed-off-by: Luke Hutton <luke.hutton@arm.com>
2023-04-118K level: rank 6 tests support using generator groupsJeremy Johnson
Split generation of conformance tests into groups to avoid a change in existing tests selection * A standard group is used for the normal/default conformance tests * An 8k level group covers the extension of testing for 8k level * Other arbitrary groups can be created and the group names will be used to tag the tests, so that in the future they could be filtered Group selection criteria is in a new dictionary * A default selection config is used for generation groups * But a different config can be used for a particular generation group Added parameter sparsity for pad of rank6 and above, and further parameter sparsity options to test_select to limit number of tests selected. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ib9f1185c0ae6d0957df2116411c17e958157c6d3
2023-03-30Add positive/negative random number generator for RsqrtJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I1e9e97ead447295e1252785106931b261df7bcea
2023-03-28Add CONST INT48 support to conformance.Jeremy Johnson
Remove ERRORIF tests for CONST as they do not create valid negative tests. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ia96b184f34010a452d2c4848b0653514352c751e
2023-03-13Update framework test choices - SPLIT, RESIZEv0.60.2Jeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ifce52ef0e082e9e6289938baa6c73a6ebd9ded62
2023-03-09Remove 1x7x7x479 from the test size listv0.60.1Eric Kunze
Was causing the generated set of tests to be too big. Change-Id: I45292e9317ab93786a1ec4670283385d06d52c0d Signed-off-by: Eric Kunze <eric.kunze@arm.com>
2023-03-08Prevent generating test case when align_corners and half_pixel are both true ↵Jerry Ge
in nearest mode Note: - align_corners and half_pixel can both be true in nearest mode for tfl but not tf - this is inconsistent between tf and tfl Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Ib396f994874904e017e0d696f70370bd51d60161
2023-03-03Add framework tests for TFL.Relu0To1OpJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I6fa14e9dbe7c0cd21156a5779d68fd473af322f2
2023-03-03Refactor resize test builderTatWai Chong
Also add input size = 1 in the shape list, and extend scaling to 1x, 2x and 3x, so that the cases of broadcasting, power-of-two scaling, no-scaling (e.g. 1x1 -> 1x1), scaling accuracy (3x) can be tested. Since the scalar tensor is tiny, should not noticeably impact the execution time of the framework test. Change-Id: Iec53da3cbb60e087077d6e2d8eb205e76e6c1313 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2023-03-02Add test case against Add Quantized Int8Jerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: If18fc61514cbc2d31a15ed7dd87147b0d8e69f16
2023-03-02Extend reduce_all to test with tfliteTatWai Chong
Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: Ifb3ed9c448005d6d53c5d0938b5146003a26e8a3
2023-03-01CONV2D/CONV3D/DEPTHWISE_CONV2D Test Regen for ofm_depth bugfixJames Ward
* Restricting size of tensors due to expanded values of ofm_depth Signed-off-by: James Ward <james.ward@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I582d8f0073c2d01244976f8e2e14783b506d8091
2023-02-28Update rank limits for SLICE, TILE and TRANSPOSELuke Hutton
Updated to align with corresponding changes to the spec. In addition, some ERROR_IF tests have been updated to match the checks specified by the spec, including: PAD, SLICE, TILE, TRANSPOSE. Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: Ie2c5f48e79a5610eb82739170e25057a63dac1d8
2023-02-24Extend pad tests to include quantized type and explicit pad value.TatWai Chong
Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: I4a49f45aa73044aff5b0a8b3dba58c1f52c1ae21
2023-02-22Create MI tests for Tensors: TRANSPOSE_CONV2DJames Ward
Change-Id: I4f967ce2dfb0cb44dc58307170fd9306a0ea1ed5 Signed-off-by: James Ward <james.ward@arm.com>
2023-02-22Bug fixes for max-batch-size/ofm-depthJames Ward
* Conformance regen required for tosa-bi: conv2d conv3d depthwise_conv2d transpose_conv2d * Include bug fix for testGen.makeShape() * Include json changes to limit size of tests which now have ofm_depth > 1 Signed-off-by: James Ward <james.ward@arm.com> Change-Id: Ic8221b9a742f5737df523997bee7260f4dfcdef3
2023-02-22Create MI tests for Tensor: FFT2D & RFFT2DJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I44ad6513fad831ff50fcc5ab801b46a5bf25d0a8
2023-02-22Create MI tests for Tensor: CONV3DJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I2c1e87ec586195f82e97bbc70e4e2bc18cd2cc88
2023-02-22Create MI tests for Tensors: DEPTHWISE_CONV2DJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I6c2087c94c2c024aabc2086d40cbca9c719c69dd
generated by cgit v1.2.1 (git 2.23.0) at 2024-06-18 12:53:00 +0000