aboutsummaryrefslogtreecommitdiff
path: root/reference_model
AgeCommit message (Collapse)Author
2023-12-12Main Conformance: Adjust TANH compliance valueJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I35d14e3e9f80198c1da3d267f12bc7a9a055e698
2023-12-11Enforce no output rewrite REQUIRE in SCATTERJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I3555e7216d403d436bf6e39d4b16bb000645c4bb
2023-12-07Add support for list of tensors as input parameterDmitrii Agibov
Some operators (e.g. Concat) expect list of tensor as an input parameter. Currently operators API does not support passing such parameters from the client code. In order to enable it: - Add new type tensor_list_t - Update operators API generation script to support new type - Add unit test for operator Concat Signed-off-by: Dmitrii Agibov <dmitrii.agibov@arm.com> Change-Id: Ib2f61bcea5e5ecabf56ce031d905cb46a4cc68ea
2023-12-06Add support for int16_t inputs in eager interfaceGeorgios Pinitas
Implement support of int16_t inputs through int32_t underlying storage buffers. Values are upcasted/downcasted as needed through explicit copy. Signed-off-by: Georgios Pinitas <georgios.pinitas@arm.com> Change-Id: I3752d98531c859002539bf1cb65413ceeff05e95
2023-12-05Fix Format Specifiers for MUL REQUIRES outputJack Frankland
Use platform agnostic format specifiers for `int64_t`. Change-Id: I002d94c1a0c0431ec09fc165a584a8f4b3ddc17d Signed-off-by: Jack Frankland <jack.frankland@arm.com>
2023-12-04Path fixes for ref model and librariesJeremy Johnson
Support relative desc.json paths in refmodel of just "desc.json". Catch when generate and verify library paths are None and provide help in tosa_verif_check_result. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie52ac399ea002e5fcdcc1eec3d9df6153a778e88
2023-12-04Change TANH, SIGMOID to ABS_ERROR complianceJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Icf04afc7fdae8f506ba4710aaa085d6ea53bb5bf
2023-12-04Main Compliance FP16 support - generate and verify.Jeremy Johnson
FP16 support for all existing operators for compliance: * DOT_PRODUCT * ULP * EXACT * ABS_ERROR Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I8d25448a793375b53880da3787d8f839767f02cf
2023-12-04Main Compliance testing support for RESHAPEJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I750ef5d3c95daca10e7717437ed375a81b434582
2023-11-30Main Compliance testing support for CASTJeremy Johnson
Limit CAST input tensor to maximums of output type to avoid saturation and infinity. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I33350a4ce0ec828da7d2e7aa8cd3183a89a97431
2023-11-30Adjust random data ranges for Main Compliance to avoid FP inf and nanJeremy Johnson
POW - there are now 3 test sets to cover random ranges. Also added ROUND mode to data generator to force integer exponent values. LOG, EXP, RSQRT, REDUCE_SUM & FULLY_CONNECTED - have had their ranges reduced for each test. Fix generate library configuration defaults and checks. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie5d3bd78f690cc787a2ca4eb9b4bd6808bd9238c
2023-11-28Fix Fast Fourier Transforms in operator APIDhruv Chauhan
* Change ignore list in generate_api.py to generate operators information. * Fix serialization attributes mapping for operator FFT and RFFT * Add a unit test for Fft2d and Rfft2d operator Change-Id: I3ad7a77a3c46aa586834188bab42cbdcc423e834 Signed-off-by: Dhruv Chauhan <dhruv.chauhan@arm.com>
2023-11-23Main Compliance testing support for COMPARISON opsJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id6229cfaccad866b110630119eb045dbf6453bf5
2023-11-23Main Compliance testing support for LOG and ACTIVATIONsJeremy Johnson
Increase exp2 allowed range to account for denormals. Minor adjustments to verify to match spec updates for pseudo code. Set ranges of activation function inputs to match spec. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I6fcf665932ac2c9080e284b865da8f7746801f59
2023-11-23Verify library fix to only accept FP64 ref dataJeremy Johnson
Make test_verif_result_check output to only say result files are non-compliant as it may not be implmenetation result issues. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I14d10a7ee3aeff7802b6b173280d9f2054760a08
2023-11-23Main Compliance testing support for AVG_POOL2DJeremy Johnson
Fix test set 4 to have more symmetrical data set for AVG_POOL2D. Increased size of AVG_POOL2D test shapes to meet MIN_DOT_PRODUCTS. Print out sign of bias magnitude in verify library to help debugging. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Icf139cf5788cbc68f2c537567bdfb4cfe31274df
2023-11-22Correct Fully Connected Validation LogicJack Frankland
The bias operand of the fully connected operator must be a 1D tensor either equal to the output channel size or of size 1. Previously we asserted the former case, we now include the second case. Signed-off-by: Jack Frankland <jack.frankland@arm.com> Change-Id: I07dbc8a3aa1650703e5c50e1e7f36bb9539fd5db
2023-11-16Support loading shared libraries for custom operatorsJerry Ge
- Add a new command line option to allow users to specify a custom defined dll library - Add a custom registry to store all registered libraries - Add a dummy example (custom_op_example.cpp) for demonstrating this new feature Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I7c360835933f77e33fcbd772cabfe01d82282d47
2023-11-16Main Compliance testing support for EXP & POWJeremy Johnson
Added new ABS_ERROR mode to verify lib and ref model. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ifb78290675833d3df7df91a4d6cef336b02b64a4
2023-11-16Main Compliance testing for simple UNARY opsJeremy Johnson
For RECIPROCAL, RSQRT, CEIL, FLOOR, ABS, NEGATE & IDENTITY. Improved ULP informational output. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I49644573b4c9a30b2b9d6c9624f2a1d46976a378
2023-11-14[reference_model] Add local_bound supportTai Ly
Add support for local_bound attributes. Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: Ie1acb65ca2495fb7d1512bf120568c695635d631
2023-11-10Main Compliance testing for FULLY_CONNECTEDJeremy Johnson
Updated shapes to meet MIN_DOT_PRODUCTS. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I82297917c009b3120306f8a9bb965209d109ef8d
2023-11-10Main Compliance testing support for ARGMAX, REDUCE_SUM/MAX/MINJeremy Johnson
Add extra tests for FP32 REDUCE_SUM that meet MIN_DOT_PRODUCTS. Plus improved dot product test generation skip information. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ia8198a9500ddddfc86c5bb84230b9a4edf5ffd50
2023-11-10Main Compliance testing support for ADD, SUB, MINIMUM and MAXIMUMJeremy Johnson
Added main inference compliance mode data generation and verification. Improved error reporting in tosa_verif_build_tests. Fixed pseudo random invalid range check. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ib538b5999444d67704ebc322be3c7becbe5ad206
2023-11-10[fix] Fix the parsing failure during splitting multiple files to the argumentTatWai Chong
The second parameter of std::string::substr holds the number of characters to include in the substring, rather than the index of the end of the capture. Change-Id: Ie8c36efcb1850fcc1f44d430569f18646f6bfd45 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2023-11-02Main compliance testing support for MULJeremy Johnson
Update verify ULP mode to allow fractions (e.g. 0.5). Update pseudo generator to accept ranges. Fix up pseudo random distribution based on ranges. Change-Id: I9168c5f7d37722678c0f1f9e906953c8cec367b1 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-11-02Compliance mode testing for CONV2DJeremy Johnson
Added CONV2D data generation. Updated verify dot product check to latest specification. Updated test generator and python datagenerator library to create const files during test generation. Add support for compliance test sets to conformance test_select. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I5be3b761a1e3ef259c058e493877cd5a89d5778b
2023-11-01Fix TransposeConv2d in operator APIDmitrii Agibov
- Change name of the TransposeConv2d attribute output_shape to out_shape in generate_api.py to match with TOSA specification - Fix serialization attributes mapping for operator TransposeConv2d - Add a unit test for TransposeConv2d operator Signed-off-by: Dmitrii Agibov <dmitrii.agibov@arm.com> Change-Id: I6613c0d093aeea0af30012bcc1c8e5d26dec746c
2023-10-31Fix Reshape in operator APIGrant Watson
- The API incorrectly requires the new shape to be passed in twice. - This fix changes the name of the attribute from new_shape to shape in the generate_api.py script. - Adds a unit test to verify that the reshape operator works correctly. Signed-off-by: Grant Watson <grant.watson@arm.com> Change-Id: I07dd0ef786c747896b6e54f4eada0e7b97c6cef3
2023-10-30Add environment variable to skip debug backtraceEric Kunze
set TOSA_MODEL_NO_BACKTRACE to avoid printing a backtrace on a model failure Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I9105169ac8cc0b5092b54b9e13c18fba18c520af
2023-10-26Compliance testing support for MAX_POOL2D & PADJeremy Johnson
Added Pseudo Random number generator in generate library. Enabled MAX_POOL2D, PAD FP32 tests to use new generator and compliance. Fixed verify library exact mode to expect reference data as FP64. Simplified tosa_verif_build_tests internal interfaces for new tests. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Icc0ffa924cf38107c3a212efd452c47a650c9d98
2023-10-25Improve verfiy and generate library validationJeremy Johnson
Improved libraries validation to catch unknown values. Improved verify output to match generate library. Refactored generate tests to use less code duplication. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I9c38745fbc8e70f46c19ddae6c62ee248d33b5f1
2023-10-24Catch when CONSTs aren't initializedJeremy Johnson
CONST data should be either data in the flatbuffer file or loaded in via an input file. This check catches if neither of these have been done, instead of marking uninitialized data as valid. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I97dc2254f0b58c05c39cc0281a331a394c2a4b3c
2023-10-23Fix AvgPool2D regressionJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I414899d0f504af00da185e0fc4119f3bde2bae3a
2023-10-20Fix AVG_POOL2D bad_alloc with large kernel sizesJames Ward
- Nested looping instead of using Eigen extract_image_patches to avoid OOM errors Signed-off-by: James Ward <james.ward@arm.com> Change-Id: Id4d78d5b5dd04a00134f29b1d29f814195515b1f
2023-10-17Add boolean input argument supportJack Frankland
Add support for tensors of booleans as input arguments to operators. Signed-off-by: Jack Frankland <jack.frankland@arm.com> Change-Id: I35bb5768cef11348c27ffbe5869c32c144485958
2023-10-16Data generator library python interface addedJeremy Johnson
Added support for using generate library in tosa_verif_build_tests and tosa_verif_run_tests tosa tool scripts. Reduced scope of compliance test creation and verification to the supported type of FP32. Fix missing virtual destructor warning in generate_dot_product.h and add config file for generate library. Simple pytests included to check python interface. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I6cdad9b00660d6ddc8bd07fdea813937fb48626a
2023-10-05Expand TOSA MI generator support for MATMULJeremy Johnson
Fixed PrimitiveGenerator starting point and added test sets 1-5. Fixed verify_test reduce_product missing data_type. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Iaf080ce5c1adb5819f70d1a285d04baa36016092
2023-10-04Add initial TOSA MI generator supportJeremy Johnson
Add support for dot-product MatMul - test set 0 Signed-off-by: Georgios Pinitas <georgios.pinitas@arm.com> Change-Id: Ifd15b42570014b634f59c94a1fd1cd56bac79ea4 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-10-03Add reduce product verifierJack Frankland
* Add verifiers to validate the result of a reduce produce operation. * Add test cases for the new validator. Change-Id: I666d1a67f498e7893e0f224bc5408a4134f2ef6c Signed-off-by: Jack Frankland <jack.frankland@arm.com>
2023-10-02Support for compliance checking testingJeremy Johnson
Updated to conformance generator to not generate tests with results for compliance tests. Updated test runner to run compliance mode version (precise & abs mode) of reference model to create test results to use against SUT results. Updated reference model to enable abs_mode on correct desc.json flags. Updated test checker to support compliance checking using verifier lib. Seperated color printing from test checker. Change-Id: I7e2fbfc6883916caa5d94d4ece122c48bf45f530 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-10-02Update verifier library data-type supportJeremy Johnson
Make compliance meta-data data-type required for all. Add data-type checking for all verifier modes. Add initial enum support for new ROUND compliance mode. Improve print out information from library. Use numpy ctypes.data_as to get f16 support compared to ctypes_lib. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie983ba4ea958a88556f30c09b3ebc19cd9ec96b7
2023-10-02Add ULP verification for fp32Jack Frankland
Add a verifier to check two results are correct within a certain ULP tolerance for IEEE-754 32-bit floating point values. Add a test to check the ULP verifier is correct. Signed-off-by: Jack Frankland <jack.frankland@arm.com> Change-Id: Iaf43069f300999479d998e7837746b247ca5177e
2023-09-28feat: Add exact verifierJack Frankland
Add a verifier to check that tensor results match exactly. Add a unit test to check the behavior of this new verifier. Change-Id: I9b80a6d57640fec67c6be80a97b3af484aeb935e Signed-off-by: Jack Frankland <jack.frankland@arm.com>
2023-09-25Pass parameter acc_size to AvgPool2d operatorDmitrii Agibov
Signed-off-by: Dmitrii Agibov <dmitrii.agibov@arm.com> Change-Id: I4cd818af0db5e6e8a96641246cd3263ba4878f56
2023-09-15Update reference model MAX_SCALE to match specEric Kunze
MAX_SCALE for the EIGHTK level is 256 now. Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I6f3a3623d18ff238888d994c2a4129091f5a92fe
2023-09-15[reference_model] Support StatefulOps and the tests for CallOnceOpTai Ly
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I03cb878736ccd7e1f5e1f780d7171949a19a9de2
2023-09-14Change version to 0.90 draftEric Kunze
Include 0.90 draft serialization library Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I290360bfd1c147507e092868b5546c9be08682c2
2023-09-14Upgrade to latest version of TOSA specificationGrant Watson
- Updates TOSA specification to the latest version - Updates generate_api.py to generate the operator API correctly for ops with additional tensor inputs. - Removes default arguments for func_debug and func_config to make the API C compliant again. - Updates model_runner_tests.cpp for operators that have changed. - Adds a unit test for the Tile operator to check that generated code for additional tensor inputs works correctly. Signed-off-by: Grant Watson <grant.watson@arm.com> Change-Id: I1e26065c6ed333b2ca4b3da39972d30f896fa6e5
2023-09-13Build compliance verifier library with a python interface.Jeremy Johnson
Build steps to create tosa_reference_verify_lib.so New python class to handle loading and calling verifier with pytests. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id42c71c398ed7364742b4a039977412ef3081c4a
generated by cgit v1.2.1 (git 2.23.0) at 2024-07-04 07:42:04 +0000