aboutsummaryrefslogtreecommitdiff
path: root/src/armnn
AgeCommit message (Collapse)Author
2023-02-13IVGCVSW-7155: Fix Slot replacement during UpdateSubgraphViewSlotPointersFrancis Murtagh
* Only update boundary slots on actual subgraphview * Previously all slots from replacement subgraph added even if internal Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Signed-off-by: Matthew Bentham <matthew.bentham@arm.com> Change-Id: Ic9ef9fc41ad248838d1c019dd0368378c3119648
2023-01-25Fix incorrect copyright noticesMatthew Sloyan
* Updated headers that were missing a copyright notice. * Reverted years that were incorrectly updated. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I65842f1e9b9fd5654563edd5090133cb3c89fecc
2023-01-23IVGCVSW-7453 Comparison does not Calculate its shape properlyMike Kelly
* Fixed issue where ComparisonLayer wasn't calculating its output shape correctly. Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I37fe437b598bde694e519d6792182924bd0197cd
2023-01-20IVGCVSW-7013 Removing the check for constant layer in FoldPadTeresa Charlin
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Iaa3cff46217117aefdb92f281e9da2b0315f3af9
2023-01-18Github #700: Fix order of optimizations so dequantization works with foldingFrancis Murtagh
* Folding of pad into conv2d expected a Constant layer not Dequantisation * Fusing Dequantisation with Constant to a Constant ensures that. * Group Constant layer optimizations together where possible. * Add unit test. Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Change-Id: Id0393313bf097595f2f13738b7513e427116ea4a
2023-01-18IVGCVSW-7405 Improving error handling around creating directories.Colm Donelan
The -F execute network option creates a directory to print intermediate tensors but minor problems caused serious failures. This attempts to clean up the error handling. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: Ia44c008919b1bee299b43a672235b1fcc25bf1bd
2023-01-13Do the same for both ShapeInferenceMethod in Elementwise inferOutputShapeTeresa Charlin
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Iefe255ed27e1f69d99dc0cae9cfa1d6d1a51720e
2023-01-12IVGCVSW-7244 ConcatLayer overlapping views on TFLiteParserMike Kelly
* Added ability to calculate dynamic tensors and propagate them through the model so that when those tensors are later used as inputs they have the right shapes. * Added InferOutputShapes to DetectionPostProcessLayer. * Added InferOutputShapes to MeanLayer. * Added InferOutputShapes to RankLayer. * Added InferOutputShapes to ReduceLayer. * Fixed typos in TfLiteParser. Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I880c0716938ef278f5dbf01a8a73a5cc99ce5ded
2023-01-12IVGCVSW-7418 Allow working copy SubgraphView to get Original SlotsFrancis Murtagh
* API to remove need for workaround so backend users can get slots * OutputSlots outside the SubgraphView needed to obtain TensorInfo * Fix a few Copyright headers * Add shared_ptr back to original subgraph view using std::enable_shared_from_this Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Change-Id: I033a00d6fc4020619d406ac06a156b7e380a426a
2023-01-09IVGCVSW-7418: Can't call SubstituteSubgraph on working copy of subgraph in ↵Matthew Bentham
Optimize * Add unit test for WorkingCopy of SubgraphView with Inputs and Outputs * Added check to ensure InputSlot is connected before trying to disconnect Signed-off-by: Matthew Bentham <matthew.bentham@arm.com> Change-Id: I261d55e38c94687a9de64cdee726a7c7442ed537
2023-01-06IVGCVSW-7031 Generate static execute networkRyan OShea
* Build ExecNet lib dependencies as object libs except libarmnn * Disable PIPE when building static ExecNet * Remove multiple definition from AsyncExecutionCallback * Disable DynamicBackend for ExecNet Static build * Disable inference tests for TfLiteParser and ONNX during static ExecNet * Remove Tensorflow Parser if condition * Add Disable thread macro to InferenceModel * Don't compile dynamic backend symbols in Runtime.cpp for Baremetal and Exenet Static Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: If41c063eab5f05b3df0a6e064924a36a177f116a
2023-01-05Deregistering custom allocators in RuntimeImpl destructor.Colm Donelan
Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: Ib148daf81700d9038d6e91a35c9c17e770c26e84
2023-01-05GitHub #543 Problem Parsing Mixed-Precision ModelMike Kelly
* Fixed bug when converting Constants with Per-Axis Quantization Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: Ifbea23e60483746ec987da491dae96e74cb33af4
2023-01-04IVGCVSW-7211 Fix float16 operators being wrongly unsupported with ↵Cathal Corbett
android-nn-driver. * Not a concern with the delegate/parser as tflite builtin operators have little float16 support * Will also fix float16 workloads running on the support_library. Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: Iec2033dbc8ece2140b188de1f193c344a68b9c36
2022-12-23Remove Network.cpp debug message merged in "Optimize the calling of ↵Cathal Corbett
IsLayerSupported()." Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I7fa4110a1b545c1fcd79e387d53ae7d6cd381f31
2022-12-21Add profile data for explicit copy of input and outputsMatthew Bentham
Signed-off-by: Matthew Bentham <matthew.bentham@arm.com> Change-Id: I54d4a017e5f2d0bbeb742b6a33121e9ca5327f47
2022-12-12IVGCVSW-7209 Remove deprecated code due to be removed in 23.02Mike Kelly
* Removed weights and bias from Convolution, DepthwiseConv & FullyConnected layers * Removed the weight and bias ConstTensorHandles from the QueueDescriptors * Updated Workloads to take tensors from WorkloadInfo rather than the QueueDescriptors * Removed unused RedirectMembersToConstantInputs optimization and tests. Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I9ffcdc4a1c0dff725539dd69fc435b700bd98a56
2022-12-12Optimize the calling of IsLayerSupported().Cathal Corbett
* Done as part of 22.11/23.02 innovation days. * IsLayerSupported() is called in model prepare (delegate, android-nn-driver and shim/support_library) and again in ArmNN once model otimization is performed. * From calling IsLayerSupported() the first time, we should know that the layers are supported and what backend they are supported on. * Solution is to set the BackendId of the IConnectableLayer when IsLayerSupported() is called the first time, * In the Optimize() function we then check if the backend is set. If so, we do not call IsLayerSupported() again. * In the case a layer that is supported gets optimized, then the BackendId of that layer get set to "Unknown" for the new optimized layer and IsLayerSupported() will get called on the newly optimized layer. * Includes bug fix IVGCVSW-7213 for Android Mean FP16 CpuAcc tests. Also related to bug IVGCVSW-7211. Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I7a7820d0cdb079ffb5a3a2e0c44e252f652df53b
2022-12-12Updates following execution of Includewhatyouuse on armnn/include.Colm Donelan
This tool forces explicit includes of all dependencies and highlights unused dependencies. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I92e449245246452a0227cbd13f9c082e2088bf8c
2022-12-07IVGCVSW-6853 Rewrite BuildArmComputePermutationVector()Teresa Charlin
* Some pemutation vectors were not converted correctly. * Add Transpose end to end test. * Comments added with an example to clarify the differences betweeen Transpose and Permute Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I6c0954ca6ce00ef5f2a6f3625abe6f4fd27b5cdf
2022-12-07Fix some memory overruns / undefined behaviour in ShapeInferenceTestsMatthew Bentham
In several cases the address of a single float value on the stack was passed as a pointer to the constructor of ScopedTensor (which needs a backing-store of size equal to GetNumBytes()). Replace by using a std::vector to explicitly create and initialize the right number of elements. Signed-off-by: Matthew Bentham <matthew.bentham@arm.com> Change-Id: I8a1f4bf169bd89983f2d68047173ec901a21e1fb
2022-12-06Print BatchMatMul and Gather Descriptors on dot graphTeresa Charlin
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I01de14cd46fe614dfcb11b2b4f9323f32e01ee9d
2022-11-22IVGCVSW-7080 Remove deprecated code due to be removed in 23.02Keith Davis
* Extended deprecation time of SubgraphView interface to 23.08 Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: Ic0a729ea31402f0b39724da47212ae5cc04465c4
2022-11-16IVGCVSW-7214 Disable BF16-Turbo-Mode and remove conversion layersRyan OShea
- Remove Bf16ToFp32 Conversion Layer - Remove Fp32ToBf16 Conversion Layer - Remove B16 Conversion tests * Throw exception if m_ReduceFp32ToBf16 optimzer option is set to true * Provide comments to enable fast math in order to use bf16 * Update docs to inform users to enable fast math for bf16 Execute Network Changes * Require bf16_turbo_mode to also have fast_math_enabled set to true - Remove setting m_ReduceFp32ToBf16 optimizer option Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: Ibaa6da9d29c96a1ce32ff5196b0847fde9f04a1c
2022-10-28IVGCVSW-7296 REDUCE_PROD tests fail when using Tf 2.10Teresa Charlin
* In TF what ArmNN calls quantized data types can be non-quantized as well. * This patch creates 2 models: * ArmNN: model where int8 and uint8 will always be quantized, but scale can be 1 and offset 0 * TFLite: model where int8 and uint8 can be quantized and non-quantized Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Id960f2f30988f2bbec88cb4e0c52c189ac957bae
2022-10-19MLCE-545 INT8 TFLite model execution abnormalKeith Davis
* Bug fix where files were being overwritten at each debug layer Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: I609fdc82afcee925824efb02183c7dbc942fced0
2022-10-19MLCE-545 INT8 TFLite model execution abnormalKeith Davis
* Add functionality to print output tensors to file in tempdir * UnitTests Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: Idfb4c186544187db1fecdfca11c662540f645439
2022-10-14IVGCVSW-7267 Make the AllowExpandedDims option workJim Flynn
Signed-off-by: Jim Flynn <jim.flynn@arm.com> Change-Id: I3573078206272c3a72a2b3acf8781ab458ea6c90
2022-10-11IVGCVSW-7222 Fix incorrect kernel measurements in profiling outputKevin May
* Some CL kernels are not run after the first inference and this breaks the profiler which is expecting a measurement for every kernel each run * Add a function HasKernelMeasurements() to ascertain if the Event is returning kernel measurements and if so insert 0.0 values for any missing kernel measurements. * Fix ExecuteNetwork to only print a json object after all inferences have completed Signed-off-by: Kevin May <kevin.may@arm.com> Change-Id: I99f2bb0db847f5a52ab4c5705b072155c6b6f333
2022-10-04MLCE-545 INT8 TFLite model execution abnormalKeith Davis
* Fix for Debug mode in ExNet does not work with ConstTensorsAsInputs * Remove unnecessary assertion with ambiguous message in LoadedNetwork Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: I9cd5d1f811dbbc89072d1190c510bf1b22e3069c
2022-09-28IVGCVSW-7209 Delay one release the removal of weights and biasTeresa Charlin
* This affects only to the layers (not workloads) Conv, DWConv and FC Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I66a91ed1a78bc0464e00423c7fc7c28c91d199ce
2022-09-15Make SubgraphViewSelector give deterministic resultsRob Hughes
The subgraphs produced by SubgraphViewSelector were not produced in a deterministic order, as the order was determined by the pointer values of some objects, which are not guaranteed to be the same for each execution. This patch adds a post-processing sorting step based on the GUIDs of the layers and the slot indices so that the results will be the same for each execution. This makes debugging the optimised graph much easier as subsequent stages can also be deterministic. It also simplifies some unit tests. Change-Id: I64f552706b7fb1bf82c19d85a448e054277917bc Signed-off-by: Rob Hughes <robert.hughes@arm.com>
2022-09-07IVGCVSW-7209 Remove deprecated code due to be removed in 22.11Teresa Charlin
* Files deleted when Stabilizing the API Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I0ae73ee36968fa880761c10358bfa827be5fe054
2022-09-06IVGCVSW-7155 SubgraphView::SubstituteSubgraph IOutputSlots incorrectly ↵Cathal Corbett
overridden Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: If594e291951a5f9ed1957a19a971c498f6e7843f
2022-09-06IVGCVSW-7006 Remove deprecated code due to be removed in 22.08Teresa Charlin
* AddConv and AddDWConv with weights and bias * ResizeBilinearDescriptor * b,blacklist option in accuracy tool !android-nn-driver:8172 Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ibbc04fd18be7f938b11590bf67cd7af103cb4d99
2022-09-05IVGCVSW-6497: BatchMatMul TfLite ParserSamuel Yap
* Added armnnTfLiteParser for BatchMatMul * Added unit testing for parser * Updated CMakeLists Signed-off-by: Samuel Yap <samuel.yap@arm.com> Change-Id: If6842aaf7cf08f688093b714e2ecea6e8cd87161
2022-08-30IVGCVSW-7105: BatchMatMul Optional Parameter SupportSamuel Yap
* Added transpose parameters to pre-transpose each input tensor's slices * Added adjoint parameters to pre-adjoint each input tensor's slices * Small refactoring (BatchMatMulDescriptor static helpers and BatchMatMulImpl constructor) * Updated input validation and output shape inference for parameters * Additional layer unit tests for parameters added * Versionings incremented Signed-off-by: Samuel Yap <samuel.yap@arm.com> Change-Id: Ibe5242a8a5bf604c13de0dc65844fd6c421cc667
2022-08-29IVGCVSW-7106 Additional fix models with multiple input and output tensors.Colm Donelan
* The previous fix for IVGCVSW-7106 introduced a problem around operators with multiple inputs and outputs: addSeparator was being applied to all tensors in the list not just the last one. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I0325d9abcb7fb512f834c61686c698bbfc29a3be
2022-08-29IVGCVSW-6954 'Arm NN SL Improvements'Sadik Armagan
* Move the Conv2D and DepthwiseConv2D validation to Optimization level when the weights and tensors are as constant inputs * Take into account offset and scales values when doing INT8 to FP32 dequantization Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: I1f81f15640395ac041923b10dbe9151159715117
2022-08-29IVGCVSW-7106 Incorrect Json format for some networks.Colm Donelan
* ProfilingDetails assumed that every workload description included both tensors and parameters. This is not always the case. * Modify ProfilingDetails::AddDetailsToString to check the next element to be printed before deciding to add a separator and new line. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I2577b0e8a149d0a172ee12975e18b78238d8256e
2022-08-29Bug Fix for refactor of the ExecuteNetwork for strategy in Precompiled layer.Teresa Charlin
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ib91b734d4add47e23ad00f76e53f1873ff617831
2022-08-05IVGCVSW-7149 FoldPadIntoQuantizedAvgPoolCpuRefTest test failing while ↵Cathal Corbett
running Arm NN Unittest Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I567452000287babad345e61ea85ea84f362f48e0
2022-08-05IVGCVSW-7147 Bug Fix for refactor of the ExecuteNetwork for strategy in ↵Teresa Charlin
ConvertLayers. * ConvertBf16ToFp32Layer * ConvertFp16ToFp32Layer * ConvertFp32ToBf16Layer * ConvertFp32ToFp16Layer Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I5e763519a12f017dc14b09ea191fdb3b7398c0d7
2022-08-05GitHub #667: Neon fold padding into average pool 2D quantization bug fix.Cathal Corbett
* Originated from a GitHub issue: https://github.com/ARM-software/armnn/issues/667 * Initially, Arm NN supports the pool 2D operation because there is no padding on the pool2d. Neon failure occurs when padding is followed by average pool 2D due to folding optimization. * Here we prevent the folding optimization from happening for the above special case and add it in as a backend specific optimization. Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: Ia0fd90c3a6b4b9d29c81106f154617d2e893e26b
2022-08-05Bug Fix for refactor of the ExecuteNetwork for Strategy in MemCopyLayerTeresa Charlin
* Correcting some typos Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Icb21dc4828e51afa38816bd454926fc41e9e82cb
2022-07-28Revert "Revert "IVGCVSW-6650 Refactor ExecuteNetwork""Teresa Charlin
This reverts commit 1a7f033768acb27da11503bd29abb468d2e77f9e. List of fixes to be able to add this code again: * "emplacing_back" the vector inputTensors into the vector m_InputTensorsVec outside the for loop * GetIOInfo() uses IOptimizedNetwork instead of INetwork, where the infered shapes are not saved * Add missing data type Signed32 to SetupInputsAndOutputs() * PrintOutputTensors() prints the actual output without dequantizing * Add profilingDetailsMethod as input in networkProperties in ArmNNExecutor constructor * Fix typos Change-Id: I91de166f87228282db3efa27431fe91458834442 Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ic6634d48892d11e5f146cdf285e1e333e93e9937 Signed-off-by: Francis Murtagh <francis.murtagh@arm.com>
2022-07-27IVGCVSW-7109: Add Batch MatMul front end support - ReferenceSamuel Yap
* Descriptors added for BatchMatMul * Layer definition added * Input validation added (will likely change when opt. param support comes in) * Ref workload implementation for BatchMatMul added (will also change with opt. param support) * Ref layer tests made for BatchMatMul * CMake and other build files updated Signed-off-by: Samuel Yap <samuel.yap@arm.com> Change-Id: Ic885301da543ee0fbe7922b85e7f9658c4efc617
2022-07-27IVGCVSW-6978: RedirectMembersToConstantInputs does not work with ↵Francis Murtagh
Fp32NetworkToBf16Converter * Fuse FP32ToBF16Layers with Constant Layer so Conv2d/FullyConnected can have their weights redirected. * If BF16 Unsupported in Conv2d || FullyConnected revert fused Constant Layer to FP32 Change-Id: If523c708a822659d64597d9ae39cca1c2f84b76f Signed-off-by: Francis Murtagh <francis.murtagh@arm.com>
2022-07-27IVGCVSW-6896 Fix pre-import when using sync execute.Colm Donelan
* Refactor backend capability checks in LoadedNetwork. * ImportInputs should check the number of tensors does not exceed the number of inputs. * In EnqueueWorkload the check for for the count of input tensors was ignoring pre-imported inputs. * Added checks to verify ImportInputs/ImportOutputs worked as expected in EndToEndTestImpl. * Improve documentation on ImportInputs/ImportOutputs in IRuntime.hpp. * Disabled import tests in CL and Neon EndToEndTests that cannot work. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: Iae4b2644a1c9f01ee72bce1afb211661cc9ae2e3
2022-07-27IVGCVSW-6620 Update the async api to use ExecutionDataMatthew Sloyan
* ExecutionData holds a void* which can be assigned to data required for execution in a backend. WorkingMemDescriptors are used in the Ref backend which hold TensorHandles for inputs and outputs. * Updated ExecuteAsync functions to take ExecutionData. * Added CreateExecutionData and UpdateExectutionData to IBackendInternal. * Streamlined experimental IWorkingMemHandle API by removing map related function and unused m_workingMemDescriptorMap from WorkingMemHandle. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I54b0aab12872011743a141eb42dae200227769af
generated by cgit v1.2.1 (git 2.23.0) at 2024-06-29 11:22:21 +0000