aboutsummaryrefslogtreecommitdiff
path: root/verif/frameworks/tosa_verif_framework_generator.py
AgeCommit message (Collapse)Author
2024-04-30Update serialization_lib for flatbuffer v24.3.7Tai Ly
also fixed traceback.format_exception calls to fix hang of framework testing due to failing tests. Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: Ic943aec7981a71dc8f9c898a16ac3f408f07a370
2024-03-26Add variable tensor fields for test descriptorsJerry Ge
- Add variable_name and variable_file to the desc.json file for writing variable tensors to numpy - Add the key of num_variables in the unit test declaration to specify the number of variable tensors in the graph Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I1109f66ffed52e49dbb14f4a8aca64baa2bea622
2024-01-23Add dynamic space_to_batch to the framework testTatWai Chong
Also fix the dimension mask out logic that only set batch dimension to unknown but others won't. Change-Id: I9e1d2c3bb1d24cba1242103aa2c7609ef0c2c0b3 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2024-01-23Shape infer dynamic model to static model prior to execution.TatWai Chong
Dynamic shape model cannot directly run on the refenence model as the concrete size of tensor is unknown therefore the volume of tensor is not able to be allocated. Furthemore, the operators also expect the input model is static-shaped. This change turns dynamic model to static model prior to execution. - Add `ifm_dynamic` field into json description to indicate whether the model has dynamic shape or not. - Add the shape inference pass into the compilation pipeline, firstly legalize the dynamic tf/tfl model to dynamic tosa model with unknown shapes, and then run the shape inference pass with static shapes input argument to resolve unknown dimensions. Change-Id: I5d2ffd452becc562dc30546789705bd01dd7a0b0 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2024-01-05Fix dynamic_input_shape referenced before assignment errorJerry Ge
- Found the error for framework test generator when it's not generating dynamic_shaped tests Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I54bf1422026f7f962d3f6ed0be9bd2e4dade14c5
2023-12-15Add basic framework test cases for dynamic shapesJerry Ge
- Added a basic infrastructure for allowing generate network with dynamic_shapes - Added tests cases for - batch_to_space - depth_to_space, space_to_depth - linear Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Ie3f13231a74485df64b852f554cfe65e995f0d03
2023-09-28Add 0-rank tensor support for concat in framework testWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Iff77091e4a57f487431ffbf7ac1c89301a153c8b
2023-09-15[reference_model] Support StatefulOps and the tests for CallOnceOpTai Ly
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I03cb878736ccd7e1f5e1f780d7171949a19a9de2
2023-07-26Update framework tools to add name in jsonEric Kunze
Name of the test can now come from the json description rather than inferred from the directory. Old method remains to keep functionality with older tests Change-Id: I79f872103ae6a101b5cc799af91c933839f28dfb
2023-07-11Fix size mismatch of placeholders for broadcast fuzzingWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Ib954db4f8f2fd5008e95d5651a6e84a8f5b7161d
2023-07-06Add reference model framework test for INT8 squared differenceWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: I6fba6907cef0616c18dc461dbb92d2aceb582f6c
2023-07-06Enable framework generator to save numpy files in native dtypesJerry Ge
With the following patch merged, the tosa serialization_lib is able to read u/int8, u/int16 in their native dtypes: https://review.mlplatform.org/c/tosa/serialization_lib/+/9871 Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I5e63fb70b055cc5ee33e7dc26237eef9bedeac30
2023-07-05Add a parameter to tensor generation function to disable fuzzingWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Iff31b33b818a181371904915d5477a169513aa2e
2023-06-29Add support for ERF operator to reference modelWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Ib42b867287b83a183a0d0fb1f1eb29974f58fae4
2023-06-21Add tf broadcast-to testingTai Ly
This patch adds BoradcastTo Op to the tf tests. Did not add tflite testing because the tf.lite.TFLiteConverter converts tf.broadcast-to to tfl.mul by 1. Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: Icd372e619c318121c19eca87d5716bcd5fbbbb23
2023-06-15Fix framework test generator with custom_shapesJerry Ge
shape_list is global while the custom_shape is only applicable for a single operator, need to make a copy for the whole shape_list when dealing with custom_shapes Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I2c5b87cb6fcaa153b0ccfe5827bae814b9cfe2b9
2023-06-02Support custom_shapes for framework test generationJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Ia29c73cb5d0a7f91914e2a94ca52d06f375722e9
2023-04-13Add framework tests for tfl.real and tfl.imagLuke Hutton
Change-Id: I665acac9b5171efd0c5a2b68b516609048f6e187 Signed-off-by: Luke Hutton <luke.hutton@arm.com>
2023-03-30Add positive/negative random number generator for RsqrtJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I1e9e97ead447295e1252785106931b261df7bcea
2023-03-09Remove 1x7x7x479 from the test size listv0.60.1Eric Kunze
Was causing the generated set of tests to be too big. Change-Id: I45292e9317ab93786a1ec4670283385d06d52c0d Signed-off-by: Eric Kunze <eric.kunze@arm.com>
2023-03-03Add framework tests for TFL.Relu0To1OpJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I6fa14e9dbe7c0cd21156a5779d68fd473af322f2
2023-03-03Refactor resize test builderTatWai Chong
Also add input size = 1 in the shape list, and extend scaling to 1x, 2x and 3x, so that the cases of broadcasting, power-of-two scaling, no-scaling (e.g. 1x1 -> 1x1), scaling accuracy (3x) can be tested. Since the scalar tensor is tiny, should not noticeably impact the execution time of the framework test. Change-Id: Iec53da3cbb60e087077d6e2d8eb205e76e6c1313 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2023-03-02Add test case against Add Quantized Int8Jerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: If18fc61514cbc2d31a15ed7dd87147b0d8e69f16
2023-03-02Extend reduce_all to test with tfliteTatWai Chong
Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: Ifb3ed9c448005d6d53c5d0938b5146003a26e8a3
2023-02-24Extend pad tests to include quantized type and explicit pad value.TatWai Chong
Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: I4a49f45aa73044aff5b0a8b3dba58c1f52c1ae21
2023-01-30Add framework tests for tfl.atan2Luke Hutton
Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: I11a891296a040f4e48187d0d51a4dd2c554805ac
2023-01-27Add RFFT2d to optimized only kernel listLuke Hutton
This fixes framework test generation when the "optimized" kernel mode option is not selected. As a result, the optimized RFFT2d kernel is used when generating tests in the reference kernel mode to prevent "op not found" errors. Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: Ice18bfa08c46e237e39c99f849622bf99d9b849c
2023-01-27Add framework test for math.signTatWai Chong
The result comparison between Tensorflow runtime and the reference model hasn't been checked as the sign operator is not supported by the native TFLite runtime. That said, since the generated tosa ops for tf.sign and tfl.sign is identical, the correctness presumably can be proved by the result from tf.sign. Change-Id: I72eb415df7fb6ca4dc9103f9ddc7104b0ba39234 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2023-01-24Add RFFT2d to the reference modelLuke Hutton
Includes: * RFFT2d reference implementation * TFLite framework tests * Basic TOSA tests * Serialization submodule upgrade with support for FFT/RFFT Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: I2a687e9cf87fb62a26160ea52439ba9830bea36e
2023-01-18Add framework tests for tf.sin and tf.cosLuke Hutton
Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: I342a162e252c8527d5b318ceaf5f1de70fa5dc34
2023-01-13Reference model update for control flow operators supportJerry Ge
Rationale for making this change: - In the original design, for control flow operators like WhileOp, child blocks couldn't read the tensor variables (global consts) in the root level block, this patch added the machanism for child blocks to access their parent level block's tensors. - This change also relies on another serialization change on adding another layer of abtraction called Region: - Serialization patch: [region] Add TosaSerializationRegion to serialization_lib - Updated the corresponding python version of the serialization code: TosaSerializerRegion to python version of serialization_lib - This change also relies on the TOSA MLIR Translator change: Add RegionBuilder to TOSA MLIR Translator - Added the WhileOp related test cases: While, LSTM, GRU, RNN - Other related fixes Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I13ae33628ad07e41d248e88652ce1328654694ab
2022-12-09Add framework tests for tfl.sin and tfl.cosLuke Hutton
Although the legalization is currently supported in the TFLite runtime, the tests were missing from the reference model. Signed-off-by: Luke Hutton <luke.hutton@arm.com> Change-Id: Iae3cd54f145f69a84943867b418063a4d4109d87
2022-11-11Add framework test for tfl.preluTatWai Chong
Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: Ib894e50f4b869a3f4936885704995a417ea25eef
2022-09-22Add framework test for TF and TFL mirrorpadTatWai Chong
Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: Icc9b8f6a65ee54ddbb445c3a999ca49401b660c2
2022-08-29Update framework test generator to support TF/TFL conv3d.TatWai Chong
Add a new attribute `rank` to indicate the testing dimension range of input tensor. Also fix a minor bug in the existing conv3d simulation. And relax rescale operator in the reference model to support 5-D input. Change-Id: Ib42fe513831dc83eb7f9af07e011787a6c752704 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2022-08-25Add generator functions to support TFL geluTatWai Chong
Change-Id: I1056f4a3abe9079381fb2c7d02edf11312aba9d6 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2022-08-10Update TOSA resize to match specificationTatWai Chong
Attribute stride and shift are removed, and has new scale and border. Also add tests in the generator to test tf.resize with all option combinations. Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If0f330d04395762d2d907863235eda1532f5e1ff
2022-08-02Added framework tests for Relu1 Operator legalizationJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I36ea46e79a0ce42c2dbe47c816b5b5572058c0f2
2022-05-19Update framework test generation for ERROR_IF criteriaJeremy Johnson
Update to tosa_verif_framework_generator to produce valid test ranges for pooling and convolution tests Fix up test filtering to only filter on test name not output directory Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ifff7e7604a37e8680d7237dc2d85cd806b20e384
2022-03-31Update pre-commit tool versionsJeremy Johnson
* black needed updating for ImportError of new click version Change-Id: If8e2d13a09996e331d676af31bf3d3aee9403b4e Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2022-03-02Add framework unit test generation scriptsJeremy Johnson
And fixes in tosa_verif_run_tests: * support for no-color printing * stop double printing of error messages on verbose * differentiate result code pass from results check Change-Id: I26e957013a8d18f7d3d3691067dfb778008a1eea Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
generated by cgit v1.2.1 (git 2.23.0) at 2024-06-01 16:03:46 +0000