Age | Commit message (Collapse) | Author |
|
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Icf04afc7fdae8f506ba4710aaa085d6ea53bb5bf
|
|
FP16 support for all existing operators for compliance:
* DOT_PRODUCT
* ULP
* EXACT
* ABS_ERROR
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I8d25448a793375b53880da3787d8f839767f02cf
|
|
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I750ef5d3c95daca10e7717437ed375a81b434582
|
|
Limit CAST input tensor to maximums of output type to avoid
saturation and infinity.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I33350a4ce0ec828da7d2e7aa8cd3183a89a97431
|
|
POW - there are now 3 test sets to cover random ranges. Also added ROUND
mode to data generator to force integer exponent values.
LOG, EXP, RSQRT, REDUCE_SUM & FULLY_CONNECTED - have had their ranges
reduced for each test.
Fix generate library configuration defaults and checks.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ie5d3bd78f690cc787a2ca4eb9b4bd6808bd9238c
|
|
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Id6229cfaccad866b110630119eb045dbf6453bf5
|
|
Increase exp2 allowed range to account for denormals.
Minor adjustments to verify to match spec updates for pseudo code.
Set ranges of activation function inputs to match spec.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I6fcf665932ac2c9080e284b865da8f7746801f59
|
|
Make test_verif_result_check output to only say result files
are non-compliant as it may not be implmenetation result issues.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I14d10a7ee3aeff7802b6b173280d9f2054760a08
|
|
Fix test set 4 to have more symmetrical data set for AVG_POOL2D.
Increased size of AVG_POOL2D test shapes to meet MIN_DOT_PRODUCTS.
Print out sign of bias magnitude in verify library to help debugging.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Icf139cf5788cbc68f2c537567bdfb4cfe31274df
|
|
Signed-off-by: Tai Ly <tai.ly@arm.com>
Change-Id: I9b87d1ad0f09648d07a6ecfc2075a174a9d04a3d
|
|
Enable use of data ranges for old data gen path as well
as the new generate library path, so that FP16 and BF16 test data
is produced within the correct ranges.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I749870a3112f8c3a75f4d16b8322c813fbf977cd
|
|
Added new ABS_ERROR mode to verify lib and ref model.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ifb78290675833d3df7df91a4d6cef336b02b64a4
|
|
For RECIPROCAL, RSQRT, CEIL, FLOOR, ABS, NEGATE & IDENTITY.
Improved ULP informational output.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I49644573b4c9a30b2b9d6c9624f2a1d46976a378
|
|
Updated shapes to meet MIN_DOT_PRODUCTS.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I82297917c009b3120306f8a9bb965209d109ef8d
|
|
Add extra tests for FP32 REDUCE_SUM that meet MIN_DOT_PRODUCTS.
Plus improved dot product test generation skip information.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ia8198a9500ddddfc86c5bb84230b9a4edf5ffd50
|
|
Added main inference compliance mode data generation and verification.
Improved error reporting in tosa_verif_build_tests.
Fixed pseudo random invalid range check.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ib538b5999444d67704ebc322be3c7becbe5ad206
|
|
Update verify ULP mode to allow fractions (e.g. 0.5).
Update pseudo generator to accept ranges.
Fix up pseudo random distribution based on ranges.
Change-Id: I9168c5f7d37722678c0f1f9e906953c8cec367b1
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Added CONV2D data generation.
Updated verify dot product check to latest specification.
Updated test generator and python datagenerator library to create
const files during test generation.
Add support for compliance test sets to conformance test_select.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I5be3b761a1e3ef259c058e493877cd5a89d5778b
|
|
Added int8, int16, uint8 and uint16.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I6b1174aa7369c34a613b322c4b05bf4fc586ee6e
|
|
Added Pseudo Random number generator in generate library.
Enabled MAX_POOL2D, PAD FP32 tests to use new generator and compliance.
Fixed verify library exact mode to expect reference data as FP64.
Simplified tosa_verif_build_tests internal interfaces for new tests.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Icc0ffa924cf38107c3a212efd452c47a650c9d98
|
|
Improved libraries validation to catch unknown values.
Improved verify output to match generate library.
Refactored generate tests to use less code duplication.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I9c38745fbc8e70f46c19ddae6c62ee248d33b5f1
|
|
Turn off ARGMAX rank 6 tests for v0.60.0
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I535b7887a9db099d39ad943e151f870ebc716791
|
|
Added support for using generate library in tosa_verif_build_tests
and tosa_verif_run_tests tosa tool scripts.
Reduced scope of compliance test creation and verification to
the supported type of FP32.
Fix missing virtual destructor warning in generate_dot_product.h and
add config file for generate library.
Simple pytests included to check python interface.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I6cdad9b00660d6ddc8bd07fdea813937fb48626a
|
|
Add --test-version option to tosa_verif_conformance_generator to
select the version for tests to output.
Add --output-type to allow json, binary or both files to be created
during conformance generation.
Fix passing schema_path to test runner.
Add explicit verify lib path arg to test runner.
Change-Id: I5f1ad137d713fca408a98470ea77bddf8916c5f3
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Updated to conformance generator to not generate tests with results for
compliance tests.
Updated test runner to run compliance mode version (precise & abs mode)
of reference model to create test results to use against SUT results.
Updated reference model to enable abs_mode on correct desc.json flags.
Updated test checker to support compliance checking using verifier lib.
Seperated color printing from test checker.
Change-Id: I7e2fbfc6883916caa5d94d4ece122c48bf45f530
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Make compliance meta-data data-type required for all.
Add data-type checking for all verifier modes.
Add initial enum support for new ROUND compliance mode.
Improve print out information from library.
Use numpy ctypes.data_as to get f16 support compared to ctypes_lib.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ie983ba4ea958a88556f30c09b3ebc19cd9ec96b7
|
|
tosa_verif_conformance_generator
- Move to using ref-model-path instead of ref-model-dir
- Add schema-path and flatc-path
- Add model_files module to locate default places for files
convert2conformance
- Remove default paths
verifier
- Switch to using exact path of verifier library
tosa_verif_run_tests
- Use conformance model_files to locate defaults
Change-Id: Ieca3b164670e2a7dcb047743667cc4e8317daa97
Signed-off-by: Josef Malmström <josef.malmstrom@arm.com>
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Signed-off-by: Won Jeon <won.jeon@arm.com>
Change-Id: Iff77091e4a57f487431ffbf7ac1c89301a153c8b
|
|
Signed-off-by: Jerry Ge <jerry.ge@arm.com>
Change-Id: I03cb878736ccd7e1f5e1f780d7171949a19a9de2
|
|
Build steps to create tosa_reference_verify_lib.so
New python class to handle loading and calling verifier with pytests.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Id42c71c398ed7364742b4a039977412ef3081c4a
|
|
Increase size of tensor dimensions to meet compliance size needs.
Also improve warning output about this size requirement.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I9dc2b8dab05a1bac0083cdca9d99845c287d3ce9
|
|
Incorrectly used "tag" in test.desc instead of "tags"
Incorrectly selected all tests rather than just all positive
tests depending on operator selection option.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ib257bc57accf745beedfa4a3b2ece9539c0972c2
|
|
Add initial support for compliance and lazy data-gen meta data
added to desc.json for MATMUL.
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I00c047814134a96d7c98d890e93b5884e25b8e64
|
|
Rename scripts utlities to tosa_* such as tosa_json2numpy
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ie8d584e2afb189fb74cf96b39590c7c27444ba14
|
|
In tosa_test_gen.py, change output_shape for dim operators to [] instead of [1]
Signed-off-by: Tai Ly <tai.ly@arm.com>
Change-Id: I8e7901e02831e9a467bce17e55e9379cb5f1b739
|
|
Compress flatbuffer JSON files for conformance to allow while_loop
tests to fit in the 30MB file size limit.
Do not include new SHAPE type in usable DTypes so that selection of
ERROR_IF tests are not changed.
Also enable strict json mode for flatbuffers.
Change-Id: Id89a9963244c34769a4662f9951adde0aa3334d9
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
|
|
Signed-off-by: Won Jeon <won.jeon@arm.com>
Change-Id: Iea11ee5d3d98773e9c5e9b827593c05afb41ce3b
|
|
The double quotation mark is not always appear in the assembly output
of operations. Remove it to easier capture general printing pattern.
Change-Id: I59301daa07abd905de7259f8ae527a74b1ce5a6c
Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
|
|
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I17227adfca203f353e51ae9297f7e01f8d7edbe9
|
|
* Add deprecation notice for conformance_tests repo
* Add deprecation notice for framework tests in conformance
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: I92b17f4d6d993e47f30d7996a18fc13b934c4cdf
|
|
Reuse the name deduced from the test run in results.
Also move most string formatting to f strings.
Move path handling to pathlib from os.path
Signed-off-by: Eric Kunze <eric.kunze@arm.com>
Change-Id: I44c7786d4b7af44306e218ff49608df35d4521a7
|
|
* Update Flatbuffers version
* Allows consts to be saved as inputs
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: Ibb9e38d381f8229bfca65b559e35c9ded889c384
|
|
Name of the test can now come from the json description
rather than inferred from the directory. Old method remains to
keep functionality with older tests
Change-Id: I79f872103ae6a101b5cc799af91c933839f28dfb
|
|
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
Change-Id: If30b329905aad9e520e4a9a3ac1dffccd37bb0e9
|
|
Temporarily disable the json to numpy pytest
Otherwise will load as the wrong dtype
Also update the flatbuffer module version to match
tosa_mlir_translator
Change-Id: I40999f631adb1b230bc8a8b891e0955dd67371f0
Signed-off-by: Eric Kunze <eric.kunze@arm.com>
|
|
Signed-off-by: Won Jeon <won.jeon@arm.com>
Change-Id: Ib954db4f8f2fd5008e95d5651a6e84a8f5b7161d
|
|
Signed-off-by: Won Jeon <won.jeon@arm.com>
Change-Id: I6fba6907cef0616c18dc461dbb92d2aceb582f6c
|
|
With the following patch merged, the tosa serialization_lib is
able to read u/int8, u/int16 in their native dtypes:
https://review.mlplatform.org/c/tosa/serialization_lib/+/9871
Signed-off-by: Jerry Ge <jerry.ge@arm.com>
Change-Id: I5e63fb70b055cc5ee33e7dc26237eef9bedeac30
|
|
Signed-off-by: Won Jeon <won.jeon@arm.com>
Change-Id: Iff31b33b818a181371904915d5477a169513aa2e
|
|
Signed-off-by: Won Jeon <won.jeon@arm.com>
Change-Id: Ib42b867287b83a183a0d0fb1f1eb29974f58fae4
|