aboutsummaryrefslogtreecommitdiff
path: root/verif/generator/tosa_test_gen.py
AgeCommit message (Collapse)Author
2024-02-08Main Compliance: RFFT2D supportJeremy Johnson
Correct ref model to produce imaginery values of zero as specification indicates at certain output positions. Fix up precise and abs modes for RFFT2D in ref model to produce correct results and bounds using abs weights. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I33767e4219a260278f7933f28b1799223a95a3cc
2024-02-08Main Compliance testing for CONV3Devacha01
Signed-off-by: evacha01 <evan.chandler@arm.com> Change-Id: Ie05f88db15cd07fd5483ab669329d7048bd3349c
2024-02-07Main Compliance: FFT2D supportJeremy Johnson
Improve access to DOT_PRODUCT generator index and location for debugging. Enable multiple result files for compliance and improve output. Fix up precise and abs modes for FFT2D in ref model to produce correct results and bounds using abs weights. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ide0c9f9f80397e5f1e07ca30a1036d6014b5784d
2024-02-06Main Compliance testing for TRANSPOSE, REVERSE, and CONSTevacha01
Signed-off-by: evacha01 <evan.chandler@arm.com> Change-Id: I95b931c032ce16c56ee05caab4dd26ea89557b3c
2024-02-01[testing] Switch to using shape type for start and size of sliceTatWai Chong
The type of both start and size of slice are tosa shape type since 0.90. Update the type of these arguments and its usage in testing. Change-Id: Iff582982ad55d64a792183c6d46c9266510f924c Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2024-01-30[ref model] Change PadOp's padding to ShapeTai Ly
Changed to use pad input of PadOp for testing. The pad input is now a tensor of tosa.shape type. moved padding error checking from checkTensorAttributes to eval modified pad's PadOutputShapeMismatch test generation to avoid generating output shapes with dimensions <= 0 Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I437c86d9a012903458a648667f6693db67b97d76
2024-01-30Fix up shape operator test errorsJeremy Johnson
Update serialization_lib to store SHAPE as INT64. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie589cd6670dc79b77df981c81cd7c27b982f20fa
2024-01-30Main Compliance: TRANSPOSE_CONV2D supportJeremy Johnson
Update data generator for main compliance values. Add test generation support. Fixed test set by including large 65k tests that were missing. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I8668c774e01c17e5d999aadf99c317e2dd893857
2024-01-30Main Compliance: DEPTHWISE_CONV2D supportJeremy Johnson
Added DEPTHWISE_CONV2D data generation. Updated test generation for FP16 and FP32. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I0471d0a1e4e279a27233f4d285082906ceea1bff
2024-01-29Main Compliance testing for SLICEevacha01
Signed-off-by: evacha01 <evan.chandler@arm.com> Change-Id: Ib9d0abe0bd8826ca5ac051a5fe6ba18e1ba727f1
2024-01-25Save Int8/UInt8 reference outputs to native dtypesJerry Ge
* Int8/UInt8 reference outputs were previously saved to INT32 * Save those in their native dtypes and updated other affected code Signed-off-by: Jerry Ge <jerry.ge@arm.com> Georgios Pinitas <georgios.pinitas@arm.com> Change-Id: Id65fc8773150d3c56bc2c72789a6a0d3c78cd363
2024-01-24Add conformance testing for shape operatorsWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Ie80570146601c470a3be7c04a9d6e1016a7c547c
2024-01-23Update RESHAPE and TILE conformance testingWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Iaf59472cb32e03a92bad87ae9dba1b7548f20268
2024-01-18[reference model] Add shape operatorsTai Ly
- fixed up reshape conformance tests to use shape input instead of attribute - fixed up tile conformance tests to use shape input instead of attribute - fixed output and output rank of dim op - allow rank 0 and rank 1 tensors for tosa.shape values (for shape = {}) - added initialization of rank 0 const_shape tensors (for shape = {}) - Update conformance tests to use new rescale attributes Signed-off-by: Tai Ly <tai.ly@arm.com> Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: I6cce0d2a9ab066fe20a2abf9d2cfde3eb3d8c18b
2024-01-18Main Compliance testing for TILEJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I7f7449c652844097fec6226bcc32ca765b40bc04
2024-01-17Main Compliance testing for CONCATJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If08c560eac171124641dac66d7de9d5053db72df
2024-01-17Main Compliance testing for SELECTJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I7276f2db39e67314c950e972cc1a97b7796dcd18
2024-01-08Main Compliance: REDUCE_PRODUCT supportJeremy Johnson
Update and fix REDUCE_PRODUCT compliance verify lib support. Added compliance test generation with data range to not cause infs. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I3b3004c6caa80d97e330a6393f435f5270b56e21
2024-01-08Main Conformance: Re-adjust TANH compliance checkJeremy Johnson
Add lower bound to ABS ERROR checks to allow for cancellation of small values in error bounds checking. Re-adjust the error bounds multiplier to match the specification. Fix up naming of verify library info structs. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I3e178c3d7d59fef9c3696178646b23ed2a3ffc61
2023-12-14Remove inferred dimension from RESHAPEJeremy Johnson
Test generation changed to only produce static reshape tests Reference model changed to produce ERROR_IF on inferred shapes Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I92c92a40e7c0e457961bc654630040dff79a750b
2023-12-14Main Compliance testing for SCATTER and GATHERJeremy Johnson
Added indices shuffling and random INT32 support to generate lib with testing of these new random generator modes Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I058d8b092470228075e8fe69c2ededa639163003
2023-12-11Enforce no output rewrite REQUIRE in SCATTERJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I3555e7216d403d436bf6e39d4b16bb000645c4bb
2023-12-04Change TANH, SIGMOID to ABS_ERROR complianceJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Icf04afc7fdae8f506ba4710aaa085d6ea53bb5bf
2023-12-04Main Compliance testing support for RESHAPEJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I750ef5d3c95daca10e7717437ed375a81b434582
2023-11-30Main Compliance testing support for CASTJeremy Johnson
Limit CAST input tensor to maximums of output type to avoid saturation and infinity. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I33350a4ce0ec828da7d2e7aa8cd3183a89a97431
2023-11-30Adjust random data ranges for Main Compliance to avoid FP inf and nanJeremy Johnson
POW - there are now 3 test sets to cover random ranges. Also added ROUND mode to data generator to force integer exponent values. LOG, EXP, RSQRT, REDUCE_SUM & FULLY_CONNECTED - have had their ranges reduced for each test. Fix generate library configuration defaults and checks. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie5d3bd78f690cc787a2ca4eb9b4bd6808bd9238c
2023-11-23Main Compliance testing support for COMPARISON opsJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id6229cfaccad866b110630119eb045dbf6453bf5
2023-11-23Main Compliance testing support for LOG and ACTIVATIONsJeremy Johnson
Increase exp2 allowed range to account for denormals. Minor adjustments to verify to match spec updates for pseudo code. Set ranges of activation function inputs to match spec. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I6fcf665932ac2c9080e284b865da8f7746801f59
2023-11-23Main Compliance testing support for AVG_POOL2DJeremy Johnson
Fix test set 4 to have more symmetrical data set for AVG_POOL2D. Increased size of AVG_POOL2D test shapes to meet MIN_DOT_PRODUCTS. Print out sign of bias magnitude in verify library to help debugging. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Icf139cf5788cbc68f2c537567bdfb4cfe31274df
2023-11-16[reference_model] add local_bound attribute to test_genTai Ly
Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I9b87d1ad0f09648d07a6ecfc2075a174a9d04a3d
2023-11-16Fix FP16, BF16 data ranges for conformance testsJeremy Johnson
Enable use of data ranges for old data gen path as well as the new generate library path, so that FP16 and BF16 test data is produced within the correct ranges. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I749870a3112f8c3a75f4d16b8322c813fbf977cd
2023-11-16Main Compliance testing support for EXP & POWJeremy Johnson
Added new ABS_ERROR mode to verify lib and ref model. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ifb78290675833d3df7df91a4d6cef336b02b64a4
2023-11-16Main Compliance testing for simple UNARY opsJeremy Johnson
For RECIPROCAL, RSQRT, CEIL, FLOOR, ABS, NEGATE & IDENTITY. Improved ULP informational output. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I49644573b4c9a30b2b9d6c9624f2a1d46976a378
2023-11-10Main Compliance testing for FULLY_CONNECTEDJeremy Johnson
Updated shapes to meet MIN_DOT_PRODUCTS. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I82297917c009b3120306f8a9bb965209d109ef8d
2023-11-10Main Compliance testing support for ARGMAX, REDUCE_SUM/MAX/MINJeremy Johnson
Add extra tests for FP32 REDUCE_SUM that meet MIN_DOT_PRODUCTS. Plus improved dot product test generation skip information. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ia8198a9500ddddfc86c5bb84230b9a4edf5ffd50
2023-11-10Main Compliance testing support for ADD, SUB, MINIMUM and MAXIMUMJeremy Johnson
Added main inference compliance mode data generation and verification. Improved error reporting in tosa_verif_build_tests. Fixed pseudo random invalid range check. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ib538b5999444d67704ebc322be3c7becbe5ad206
2023-11-02Main compliance testing support for MULJeremy Johnson
Update verify ULP mode to allow fractions (e.g. 0.5). Update pseudo generator to accept ranges. Fix up pseudo random distribution based on ranges. Change-Id: I9168c5f7d37722678c0f1f9e906953c8cec367b1 Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com>
2023-11-02Compliance mode testing for CONV2DJeremy Johnson
Added CONV2D data generation. Updated verify dot product check to latest specification. Updated test generator and python datagenerator library to create const files during test generation. Add support for compliance test sets to conformance test_select. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I5be3b761a1e3ef259c058e493877cd5a89d5778b
2023-10-26Compliance testing support for MAX_POOL2D & PADJeremy Johnson
Added Pseudo Random number generator in generate library. Enabled MAX_POOL2D, PAD FP32 tests to use new generator and compliance. Fixed verify library exact mode to expect reference data as FP64. Simplified tosa_verif_build_tests internal interfaces for new tests. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Icc0ffa924cf38107c3a212efd452c47a650c9d98
2023-10-16Data generator library python interface addedJeremy Johnson
Added support for using generate library in tosa_verif_build_tests and tosa_verif_run_tests tosa tool scripts. Reduced scope of compliance test creation and verification to the supported type of FP32. Fix missing virtual destructor warning in generate_dot_product.h and add config file for generate library. Simple pytests included to check python interface. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I6cdad9b00660d6ddc8bd07fdea813937fb48626a
2023-10-02Update verifier library data-type supportJeremy Johnson
Make compliance meta-data data-type required for all. Add data-type checking for all verifier modes. Add initial enum support for new ROUND compliance mode. Improve print out information from library. Use numpy ctypes.data_as to get f16 support compared to ctypes_lib. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie983ba4ea958a88556f30c09b3ebc19cd9ec96b7
2023-09-07Initial lazy data-gen and compliance test build supportJeremy Johnson
Add initial support for compliance and lazy data-gen meta data added to desc.json for MATMUL. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I00c047814134a96d7c98d890e93b5884e25b8e64
2023-09-01[reference model] Fix dim test generationv0.90a0Tai Ly
In tosa_test_gen.py, change output_shape for dim operators to [] instead of [1] Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I8e7901e02831e9a467bce17e55e9379cb5f1b739
2023-08-18Add DIM operator to reference modelWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Iea11ee5d3d98773e9c5e9b827593c05afb41ce3b
2023-07-31Update serialization library submoduleJeremy Johnson
* Update Flatbuffers version * Allows consts to be saved as inputs Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ibb9e38d381f8229bfca65b559e35c9ded889c384
2023-06-29Add support for ERF operator to reference modelWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Ib42b867287b83a183a0d0fb1f1eb29974f58fae4
2023-06-15Add ERROR_IF to incorrect broadcast shapesJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I7460ad9eed3ed5c7cec6e855a0303753ed28eb1c
2023-05-17Add support for one dimension of size -1 in ReshapeOpJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I0ef7607f4266296a1204c5cccdb5be36f345b5ba
2023-05-10Refactor ref_model rank checking and add level check to argmaxJerry Ge
Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Iad035b31d5e5e83040068e6311501490765bfff7
2023-04-268K levels: Tensor op tests kernel/stride at 8192 maximumsJeremy Johnson
Operators updated: AVG_POOL2D, MAX_POOL2D, CONV2D, CONV3D, DEPTHWISE_CONV2D & TRANSPOSE_CONV2D tosa_verif_build_tests argument --level-8k-sizes used to allow kernel/stride maximum boundary testing Fixed bugs in height/width validator function meaning some esixting avg_pool2d float tests need regening. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I7aeab82d3bd3c49d02d54708f2c9d995cd3cf2df
generated by cgit v1.2.1 (git 2.23.0) at 2024-06-20 11:33:22 +0000