aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-10-30Add environment variable to skip debug backtraceEric Kunze
set TOSA_MODEL_NO_BACKTRACE to avoid printing a backtrace on a model failure Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I9105169ac8cc0b5092b54b9e13c18fba18c520af
2023-10-26Compliance testing support for MAX_POOL2D & PADJeremy Johnson
Added Pseudo Random number generator in generate library. Enabled MAX_POOL2D, PAD FP32 tests to use new generator and compliance. Fixed verify library exact mode to expect reference data as FP64. Simplified tosa_verif_build_tests internal interfaces for new tests. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Icc0ffa924cf38107c3a212efd452c47a650c9d98
2023-10-25Improve verfiy and generate library validationJeremy Johnson
Improved libraries validation to catch unknown values. Improved verify output to match generate library. Refactored generate tests to use less code duplication. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I9c38745fbc8e70f46c19ddae6c62ee248d33b5f1
2023-10-24Improve conformance backward version supportJeremy Johnson
Turn off ARGMAX rank 6 tests for v0.60.0 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I535b7887a9db099d39ad943e151f870ebc716791
2023-10-24Catch when CONSTs aren't initializedJeremy Johnson
CONST data should be either data in the flatbuffer file or loaded in via an input file. This check catches if neither of these have been done, instead of marking uninitialized data as valid. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I97dc2254f0b58c05c39cc0281a331a394c2a4b3c
2023-10-23Fix AvgPool2D regressionJames Ward
Signed-off-by: James Ward <james.ward@arm.com> Change-Id: I414899d0f504af00da185e0fc4119f3bde2bae3a
2023-10-20Fix AVG_POOL2D bad_alloc with large kernel sizesJames Ward
- Nested looping instead of using Eigen extract_image_patches to avoid OOM errors Signed-off-by: James Ward <james.ward@arm.com> Change-Id: Id4d78d5b5dd04a00134f29b1d29f814195515b1f
2023-10-17Add boolean input argument supportJack Frankland
Add support for tensors of booleans as input arguments to operators. Signed-off-by: Jack Frankland <jack.frankland@arm.com> Change-Id: I35bb5768cef11348c27ffbe5869c32c144485958
2023-10-16Data generator library python interface addedJeremy Johnson
Added support for using generate library in tosa_verif_build_tests and tosa_verif_run_tests tosa tool scripts. Reduced scope of compliance test creation and verification to the supported type of FP32. Fix missing virtual destructor warning in generate_dot_product.h and add config file for generate library. Simple pytests included to check python interface. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I6cdad9b00660d6ddc8bd07fdea813937fb48626a
2023-10-12Conformance simple backward version and binary files supportJeremy Johnson
Add --test-version option to tosa_verif_conformance_generator to select the version for tests to output. Add --output-type to allow json, binary or both files to be created during conformance generation. Fix passing schema_path to test runner. Add explicit verify lib path arg to test runner. Change-Id: I5f1ad137d713fca408a98470ea77bddf8916c5f3 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-10-05Expand TOSA MI generator support for MATMULJeremy Johnson
Fixed PrimitiveGenerator starting point and added test sets 1-5. Fixed verify_test reduce_product missing data_type. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Iaf080ce5c1adb5819f70d1a285d04baa36016092
2023-10-04Add initial TOSA MI generator supportJeremy Johnson
Add support for dot-product MatMul - test set 0 Signed-off-by: Georgios Pinitas <georgios.pinitas@arm.com> Change-Id: Ifd15b42570014b634f59c94a1fd1cd56bac79ea4 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-10-03Add reduce product verifierJack Frankland
* Add verifiers to validate the result of a reduce produce operation. * Add test cases for the new validator. Change-Id: I666d1a67f498e7893e0f224bc5408a4134f2ef6c Signed-off-by: Jack Frankland <jack.frankland@arm.com>
2023-10-02Support for compliance checking testingJeremy Johnson
Updated to conformance generator to not generate tests with results for compliance tests. Updated test runner to run compliance mode version (precise & abs mode) of reference model to create test results to use against SUT results. Updated reference model to enable abs_mode on correct desc.json flags. Updated test checker to support compliance checking using verifier lib. Seperated color printing from test checker. Change-Id: I7e2fbfc6883916caa5d94d4ece122c48bf45f530 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-10-02Update verifier library data-type supportJeremy Johnson
Make compliance meta-data data-type required for all. Add data-type checking for all verifier modes. Add initial enum support for new ROUND compliance mode. Improve print out information from library. Use numpy ctypes.data_as to get f16 support compared to ctypes_lib. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie983ba4ea958a88556f30c09b3ebc19cd9ec96b7
2023-10-02Add ULP verification for fp32Jack Frankland
Add a verifier to check two results are correct within a certain ULP tolerance for IEEE-754 32-bit floating point values. Add a test to check the ULP verifier is correct. Signed-off-by: Jack Frankland <jack.frankland@arm.com> Change-Id: Iaf43069f300999479d998e7837746b247ca5177e
2023-09-28Update .gitignore filesJack Frankland
* Ignore build-debug (this is a common sub-directory for debug builds). * Ignore .cache (this is a clangd LSP directory * Ignore compile_commands (can be generated by CMake used by LSP clients) * Update copyright date. Signed-off-by: Jack Frankland <jack.frankland@arm.com> Change-Id: I74a5bc8c03f51e2d534d4870b58d477e830f6f42
2023-09-28feat: Add exact verifierJack Frankland
Add a verifier to check that tensor results match exactly. Add a unit test to check the behavior of this new verifier. Change-Id: I9b80a6d57640fec67c6be80a97b3af484aeb935e Signed-off-by: Jack Frankland <jack.frankland@arm.com>
2023-09-28Refactor path arguments to tosa-toolsJeremy Johnson
tosa_verif_conformance_generator - Move to using ref-model-path instead of ref-model-dir - Add schema-path and flatc-path - Add model_files module to locate default places for files convert2conformance - Remove default paths verifier - Switch to using exact path of verifier library tosa_verif_run_tests - Use conformance model_files to locate defaults Change-Id: Ieca3b164670e2a7dcb047743667cc4e8317daa97 Signed-off-by: Josef Malmström <josef.malmstrom@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-09-28Add 0-rank tensor support for concat in framework testWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Iff77091e4a57f487431ffbf7ac1c89301a153c8b
2023-09-28Change version scheme to not guess next versionJosef Malmström
By default, setuptools_scm will guess the next version and supply that as the version number when the package is installed. For example, this meant that installing when tag "v0.60.2" was checked out would give package version "0.60.3.dev8+gf5a8283.d20230920". With this change, the package version would instead be "0.60.2.post1.dev8+gad80c03". Change-Id: I0bca9a6abec38e82839b52fe2933d2c0d14b5a84 Signed-off-by: Josef Malmström <josef.malmstrom@arm.com>
2023-09-25Pass parameter acc_size to AvgPool2d operatorDmitrii Agibov
Signed-off-by: Dmitrii Agibov <dmitrii.agibov@arm.com> Change-Id: I4cd818af0db5e6e8a96641246cd3263ba4878f56
2023-09-22Add check of operator API to precommitEric Kunze
Attempt to avoid API getting out of sync. Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: Ic7b72c3f906e4a38cb26159bb67e9b1c4e22ca96
2023-09-18Revert tag desc.json renamingJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I3620799a01784865b3de1407e70e02582966f6d1
2023-09-15Update reference model MAX_SCALE to match specEric Kunze
MAX_SCALE for the EIGHTK level is 256 now. Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I6f3a3623d18ff238888d994c2a4129091f5a92fe
2023-09-15[reference_model] Support StatefulOps and the tests for CallOnceOpTai Ly
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I03cb878736ccd7e1f5e1f780d7171949a19a9de2
2023-09-14Change version to 0.90 draftEric Kunze
Include 0.90 draft serialization library Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I290360bfd1c147507e092868b5546c9be08682c2
2023-09-14Upgrade to latest version of TOSA specificationGrant Watson
- Updates TOSA specification to the latest version - Updates generate_api.py to generate the operator API correctly for ops with additional tensor inputs. - Removes default arguments for func_debug and func_config to make the API C compliant again. - Updates model_runner_tests.cpp for operators that have changed. - Adds a unit test for the Tile operator to check that generated code for additional tensor inputs works correctly. Signed-off-by: Grant Watson <grant.watson@arm.com> Change-Id: I1e26065c6ed333b2ca4b3da39972d30f896fa6e5
2023-09-13Build compliance verifier library with a python interface.Jeremy Johnson
Build steps to create tosa_reference_verify_lib.so New python class to handle loading and calling verifier with pytests. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id42c71c398ed7364742b4a039977412ef3081c4a
2023-09-13Restore coverage for conformance MATMULJeremy Johnson
Increase size of tensor dimensions to meet compliance size needs. Also improve warning output about this size requirement. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I9dc2b8dab05a1bac0083cdca9d99845c287d3ce9
2023-09-13Add integer divide with floor for coordinate calculationTatWai Chong
Align with the change in the spec. Define idiv_floor to give equivalent behavior to the floating-point floor function for image coordinate calculation. Change-Id: Ice06d354d58b1bb0dfedab55c9cc9eac1598b50c Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2023-09-11Fixes for conformance generator tags and test selectionJeremy Johnson
Incorrectly used "tag" in test.desc instead of "tags" Incorrectly selected all tests rather than just all positive tests depending on operator selection option. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ib257bc57accf745beedfa4a3b2ece9539c0972c2
2023-09-08Support new RESCALE attributesEric Kunze
input_unsigned and output_unsigned were added to the specification. Older TOSA files with uint data types are still supported. Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I125886ffc92975d99971e56e2075dd5d96bdbdc4
2023-09-07Initial lazy data-gen and compliance test build supportJeremy Johnson
Add initial support for compliance and lazy data-gen meta data added to desc.json for MATMUL. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I00c047814134a96d7c98d890e93b5884e25b8e64
2023-09-07Add desc.json schema validation with compliance/data_gen extensionsJeremy Johnson
Rename scripts utlities to tosa_* such as tosa_json2numpy Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie8d584e2afb189fb74cf96b39590c7c27444ba14
2023-09-07Rework TOSA verification APIGeorgios Pinitas
Change verifier API to consume verification configuration in a JSON format and enable appropriate validation to be performed within the verifier code in the reference model. Also update to latest spec changes for main compliance but not yet including bias support. Signed-off-by: Georgios Pinitas <georgios.pinitas@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I0ceaa1714dd041b00b5b29cd937c8f05e701bc4c
2023-09-06Update TosaVersion compatibility check to match with serialization_libJerry Ge
Serialization_lib change: https://eu-gerrit-1.euhpc.arm.com/c/mltech/mirrors/tosa/serialization_lib/+/542261 Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I59752a3ba7981c356634b911160cedf30fb2599e
2023-09-06fix: Conditionally include `prctl.h` on macosJack Frankland
MacOS does not have the `sys/prctl.h` header. This patch conditionally excludes `prctl.h` on MacOS and usage of its functionality in the same way we currently do for windows. Signed-off-by: Jack Frankland <jack.frankland@arm.com> Change-Id: Ic56ec358552126b3a10827d9c52db388a8acc214
2023-09-05Pass func_config to individual operator APIGrant Watson
Updates the generate_api.py script and associated templates to allow func_config and debug_config to be passed when running individual operators on the API. This will allow us, for example, to set precise_mode and abs_mode when running individual operators. Signed-off-by: Grant Watson <grant.watson@arm.com> Change-Id: Ia3e7ffc146f876daa307558433177c68285843b7
2023-09-01[reference model] Fix dim test generationv0.90a0Tai Ly
In tosa_test_gen.py, change output_shape for dim operators to [] instead of [1] Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I8e7901e02831e9a467bce17e55e9379cb5f1b739
2023-08-30Fix conformance regressions due to type and region changes.Jeremy Johnson
Compress flatbuffer JSON files for conformance to allow while_loop tests to fit in the 30MB file size limit. Do not include new SHAPE type in usable DTypes so that selection of ERROR_IF tests are not changed. Also enable strict json mode for flatbuffers. Change-Id: Id89a9963244c34769a4662f9951adde0aa3334d9 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-08-23[reference model] Code cleanup for SHAPETai Ly
remove unnecessary change of tensor shape array for SHAPE data type Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I8e6ca98448188a1c30f9d29c22b3194d327b17ac
2023-08-21Remove unused struct to avoid warning messageWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Id5820033bb3cc98b5e284237a91b22ffd54a80e3
2023-08-21Update submodule for serialization_libWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: I9504fa05b93417182a76aa74743f779db3036594
2023-08-18Add DIM operator to reference modelWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Iea11ee5d3d98773e9c5e9b827593c05afb41ce3b
2023-08-16Remove `"` from the search pattern `"tosa.*"`TatWai Chong
The double quotation mark is not always appear in the assembly output of operations. Remove it to easier capture general printing pattern. Change-Id: I59301daa07abd905de7259f8ae527a74b1ce5a6c Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2023-08-14Add support for bias broadcastingTai Ly
add support for bias broadcasting for operators: - conv2d - conv3d - depthwise_conv2d - transpose_conv2d - fully_connected could not add framework test for this because tf.nn.bias_add requires bias size to match channel dimension. manually tested reference model evaluation on tosa mlir with bias size of 1 Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I70d29d231a63fc03b10e3006cbd6b16b53cca1f2
2023-08-08Re-write json2numpy testsJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I17227adfca203f353e51ae9297f7e01f8d7edbe9
2023-08-07Update formatting based on clang-format 14Eric Kunze
Signed-off-by: Eric Kunze <eric.kunze@arm.com> Change-Id: I22939e75a08349816a9c103eb7312edcec53855e
2023-08-07Deprecation of parts of TOSA conformanceJeremy Johnson
* Add deprecation notice for conformance_tests repo * Add deprecation notice for framework tests in conformance Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I92b17f4d6d993e47f30d7996a18fc13b934c4cdf
generated by cgit v1.2.1 (git 2.23.0) at 2024-07-07 11:24:48 +0000