aboutsummaryrefslogtreecommitdiff
path: root/verif
AgeCommit message (Collapse)Author
2024-03-01Testing support for MUL with shift as inputJeremy Johnson
Always create the shift as a tensor for all types in testing. In the reference model, set the shift operand to be available for all types, but only read in the shift tensor for i32. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Signed-off-by: TatWai Chong <tatwai.chong@arm.com> Change-Id: Ia267cbf8b63ca0a9c97b38e8fb4db83eeb8c0538
2024-02-27[reference_model] tosa.fb name changesTai Ly
This patch adjusts reference model for attribute name changes in tosa.fb schema, and for obsoleted slice/tile/reshape attributes also updated examples due to the breaking tosa flatbuffers changes Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I105eb99a4c35f289c5078aed0a7f9cbb6dfe9123
2024-02-26[ref_model] Change resize attrs to inputsTai Ly
This patch implements changes needed for resize op's scale/offset/border changing from attributes to inputs Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I20db0687fad40711f3ded644af51096292dd05b3
2024-02-22Save Int16/UINT16 test outputs to native dtypesJerry Ge
* Int16/UInt16 reference outputs were previously saved to INT32 * Save those in their native dtypes and updated other affected code Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I0c3b7fba096a8cb1ddabef20ad13498b8f46d36f
2024-02-21Add support for FP8 to reference modelWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: I99b70f94aff2ccd4af64875697e124eb60bc5b08
2024-02-12Update test builder internal interfacesJeremy Johnson
Move remaining ops from using testArgs to argsDict. All tvg/build_fcn function interfaces updated. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie886fd931bd74608bda621363100bf8bfd7385e6
2024-02-08Improve Avg_Pool2D ErrorIf TestingJeremy Johnson
* Add test for invalid accumulator dtype Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I506e2047623372670b82db6e9c0010fa89802851
2024-02-08Fix REDUCE ops to support 8k MAX_RANKJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I365da379f599e5eca1841e6d299b4005a7b0f082
2024-02-08Main Compliance: Add RESIZE supportJeremy Johnson
Add RELATIVE verify mode for RESIZE. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I4fe352579507211dae7a048bf080c24426ce42a2
2024-02-08Main Compliance: RFFT2D supportJeremy Johnson
Correct ref model to produce imaginery values of zero as specification indicates at certain output positions. Fix up precise and abs modes for RFFT2D in ref model to produce correct results and bounds using abs weights. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I33767e4219a260278f7933f28b1799223a95a3cc
2024-02-08Main Compliance testing for CONV3Devacha01
Signed-off-by: evacha01 <evan.chandler@arm.com> Change-Id: Ie05f88db15cd07fd5483ab669329d7048bd3349c
2024-02-07Main Compliance: FFT2D supportJeremy Johnson
Improve access to DOT_PRODUCT generator index and location for debugging. Enable multiple result files for compliance and improve output. Fix up precise and abs modes for FFT2D in ref model to produce correct results and bounds using abs weights. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ide0c9f9f80397e5f1e07ca30a1036d6014b5784d
2024-02-06Main Compliance testing for TRANSPOSE, REVERSE, and CONSTevacha01
Signed-off-by: evacha01 <evan.chandler@arm.com> Change-Id: I95b931c032ce16c56ee05caab4dd26ea89557b3c
2024-02-01[testing] Switch to using shape type for start and size of sliceTatWai Chong
The type of both start and size of slice are tosa shape type since 0.90. Update the type of these arguments and its usage in testing. Change-Id: Iff582982ad55d64a792183c6d46c9266510f924c Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2024-01-30[ref model] Change PadOp's padding to ShapeTai Ly
Changed to use pad input of PadOp for testing. The pad input is now a tensor of tosa.shape type. moved padding error checking from checkTensorAttributes to eval modified pad's PadOutputShapeMismatch test generation to avoid generating output shapes with dimensions <= 0 Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I437c86d9a012903458a648667f6693db67b97d76
2024-01-30Fix up shape operator test errorsJeremy Johnson
Update serialization_lib to store SHAPE as INT64. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie589cd6670dc79b77df981c81cd7c27b982f20fa
2024-01-30Main Compliance: TRANSPOSE_CONV2D supportJeremy Johnson
Update data generator for main compliance values. Add test generation support. Fixed test set by including large 65k tests that were missing. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I8668c774e01c17e5d999aadf99c317e2dd893857
2024-01-30Main Compliance: DEPTHWISE_CONV2D supportJeremy Johnson
Added DEPTHWISE_CONV2D data generation. Updated test generation for FP16 and FP32. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I0471d0a1e4e279a27233f4d285082906ceea1bff
2024-01-29Main Compliance testing for SLICEevacha01
Signed-off-by: evacha01 <evan.chandler@arm.com> Change-Id: Ib9d0abe0bd8826ca5ac051a5fe6ba18e1ba727f1
2024-01-25Save Int8/UInt8 reference outputs to native dtypesJerry Ge
* Int8/UInt8 reference outputs were previously saved to INT32 * Save those in their native dtypes and updated other affected code Signed-off-by: Jerry Ge <jerry.ge@arm.com> Georgios Pinitas <georgios.pinitas@arm.com> Change-Id: Id65fc8773150d3c56bc2c72789a6a0d3c78cd363
2024-01-24Add conformance testing for shape operatorsWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Ie80570146601c470a3be7c04a9d6e1016a7c547c
2024-01-23Add dynamic space_to_batch to the framework testTatWai Chong
Also fix the dimension mask out logic that only set batch dimension to unknown but others won't. Change-Id: I9e1d2c3bb1d24cba1242103aa2c7609ef0c2c0b3 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2024-01-23Shape infer dynamic model to static model prior to execution.TatWai Chong
Dynamic shape model cannot directly run on the refenence model as the concrete size of tensor is unknown therefore the volume of tensor is not able to be allocated. Furthemore, the operators also expect the input model is static-shaped. This change turns dynamic model to static model prior to execution. - Add `ifm_dynamic` field into json description to indicate whether the model has dynamic shape or not. - Add the shape inference pass into the compilation pipeline, firstly legalize the dynamic tf/tfl model to dynamic tosa model with unknown shapes, and then run the shape inference pass with static shapes input argument to resolve unknown dimensions. Change-Id: I5d2ffd452becc562dc30546789705bd01dd7a0b0 Signed-off-by: TatWai Chong <tatwai.chong@arm.com>
2024-01-23Update RESHAPE and TILE conformance testingWon Jeon
Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: Iaf59472cb32e03a92bad87ae9dba1b7548f20268
2024-01-18[reference model] Add shape operatorsTai Ly
- fixed up reshape conformance tests to use shape input instead of attribute - fixed up tile conformance tests to use shape input instead of attribute - fixed output and output rank of dim op - allow rank 0 and rank 1 tensors for tosa.shape values (for shape = {}) - added initialization of rank 0 const_shape tensors (for shape = {}) - Update conformance tests to use new rescale attributes Signed-off-by: Tai Ly <tai.ly@arm.com> Signed-off-by: Won Jeon <won.jeon@arm.com> Change-Id: I6cce0d2a9ab066fe20a2abf9d2cfde3eb3d8c18b
2024-01-18Main Compliance testing for TILEJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I7f7449c652844097fec6226bcc32ca765b40bc04
2024-01-17Main Compliance testing for CONCATJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: If08c560eac171124641dac66d7de9d5053db72df
2024-01-17Main Compliance testing for SELECTJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I7276f2db39e67314c950e972cc1a97b7796dcd18
2024-01-08Main Conformance: Add CONV2D f32 test with zero paddingJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I466c1a8cb7c4cb3a8f6f18cf9885e7eb5e9819dd
2024-01-08Main Compliance: REDUCE_PRODUCT supportJeremy Johnson
Update and fix REDUCE_PRODUCT compliance verify lib support. Added compliance test generation with data range to not cause infs. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I3b3004c6caa80d97e330a6393f435f5270b56e21
2024-01-08Main Conformance: Re-adjust TANH compliance checkJeremy Johnson
Add lower bound to ABS ERROR checks to allow for cancellation of small values in error bounds checking. Re-adjust the error bounds multiplier to match the specification. Fix up naming of verify library info structs. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I3e178c3d7d59fef9c3696178646b23ed2a3ffc61
2024-01-05Fix dynamic_input_shape referenced before assignment errorJerry Ge
- Found the error for framework test generator when it's not generating dynamic_shaped tests Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: I54bf1422026f7f962d3f6ed0be9bd2e4dade14c5
2023-12-15Add basic framework test cases for dynamic shapesJerry Ge
- Added a basic infrastructure for allowing generate network with dynamic_shapes - Added tests cases for - batch_to_space - depth_to_space, space_to_depth - linear Signed-off-by: Jerry Ge <jerry.ge@arm.com> Change-Id: Ie3f13231a74485df64b852f554cfe65e995f0d03
2023-12-14Remove inferred dimension from RESHAPEJeremy Johnson
Test generation changed to only produce static reshape tests Reference model changed to produce ERROR_IF on inferred shapes Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I92c92a40e7c0e457961bc654630040dff79a750b
2023-12-14Main Compliance testing for SCATTER and GATHERJeremy Johnson
Added indices shuffling and random INT32 support to generate lib with testing of these new random generator modes Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I058d8b092470228075e8fe69c2ededa639163003
2023-12-11Enforce no output rewrite REQUIRE in SCATTERJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I3555e7216d403d436bf6e39d4b16bb000645c4bb
2023-12-11Rectify coverage loss due to FP16 complianceJeremy Johnson
Operator coverage restored/improved: * REDUCE_SUM * AVG_POOL2D * FULLY_CONNECTED Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie622b2f97b5c34971fdde5b20df3d616171ac11f
2023-12-04Path fixes for ref model and librariesJeremy Johnson
Support relative desc.json paths in refmodel of just "desc.json". Catch when generate and verify library paths are None and provide help in tosa_verif_check_result. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie52ac399ea002e5fcdcc1eec3d9df6153a778e88
2023-12-04Change TANH, SIGMOID to ABS_ERROR complianceJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Icf04afc7fdae8f506ba4710aaa085d6ea53bb5bf
2023-12-04Main Compliance FP16 support - generate and verify.Jeremy Johnson
FP16 support for all existing operators for compliance: * DOT_PRODUCT * ULP * EXACT * ABS_ERROR Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I8d25448a793375b53880da3787d8f839767f02cf
2023-12-04Main Compliance testing support for RESHAPEJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I750ef5d3c95daca10e7717437ed375a81b434582
2023-11-30Main Compliance testing support for CASTJeremy Johnson
Limit CAST input tensor to maximums of output type to avoid saturation and infinity. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I33350a4ce0ec828da7d2e7aa8cd3183a89a97431
2023-11-30Adjust random data ranges for Main Compliance to avoid FP inf and nanJeremy Johnson
POW - there are now 3 test sets to cover random ranges. Also added ROUND mode to data generator to force integer exponent values. LOG, EXP, RSQRT, REDUCE_SUM & FULLY_CONNECTED - have had their ranges reduced for each test. Fix generate library configuration defaults and checks. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ie5d3bd78f690cc787a2ca4eb9b4bd6808bd9238c
2023-11-23Main Compliance testing support for COMPARISON opsJeremy Johnson
Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Id6229cfaccad866b110630119eb045dbf6453bf5
2023-11-23Main Compliance testing support for LOG and ACTIVATIONsJeremy Johnson
Increase exp2 allowed range to account for denormals. Minor adjustments to verify to match spec updates for pseudo code. Set ranges of activation function inputs to match spec. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I6fcf665932ac2c9080e284b865da8f7746801f59
2023-11-23Verify library fix to only accept FP64 ref dataJeremy Johnson
Make test_verif_result_check output to only say result files are non-compliant as it may not be implmenetation result issues. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I14d10a7ee3aeff7802b6b173280d9f2054760a08
2023-11-23Main Compliance testing support for AVG_POOL2DJeremy Johnson
Fix test set 4 to have more symmetrical data set for AVG_POOL2D. Increased size of AVG_POOL2D test shapes to meet MIN_DOT_PRODUCTS. Print out sign of bias magnitude in verify library to help debugging. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Icf139cf5788cbc68f2c537567bdfb4cfe31274df
2023-11-16[reference_model] add local_bound attribute to test_genTai Ly
Signed-off-by: Tai Ly <tai.ly@arm.com> Change-Id: I9b87d1ad0f09648d07a6ecfc2075a174a9d04a3d
2023-11-16Fix FP16, BF16 data ranges for conformance testsJeremy Johnson
Enable use of data ranges for old data gen path as well as the new generate library path, so that FP16 and BF16 test data is produced within the correct ranges. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: I749870a3112f8c3a75f4d16b8322c813fbf977cd
2023-11-16Main Compliance testing support for EXP & POWJeremy Johnson
Added new ABS_ERROR mode to verify lib and ref model. Signed-off-by: Jeremy Johnson <jeremy.johnson@arm.com> Change-Id: Ifb78290675833d3df7df91a4d6cef336b02b64a4
generated by cgit v1.2.1 (git 2.23.0) at 2024-06-10 23:09:04 +0000