aboutsummaryrefslogtreecommitdiff
path: root/delegate
AgeCommit message (Collapse)Author
2023-04-27IVGCVSW-7591 IVGCVSW-7592 Add Gather and GatherNd to Opaque DelegateKevin May
Signed-off-by: Kevin May <kevin.may@arm.com> Change-Id: Id2b6a4f70b1cb50e5f7f7ab4e30487b3816c9ad4
2023-04-26IVGCVSW-7575 Implement opaque delegate for ArgMinMax operatorJohn Mcloughlin
* Added Opaque ArgMinMax and associated test cases Signed-off-by: John Mcloughlin <john.mcloughlin@arm.com> Change-Id: I098b94cc35707370a0bbc7456bfdd48bb47432f0
2023-04-26IVGCVSW-7577, IVGCVSW-7578 Implement BatchToSpaceNd and SpaceToBatchNd in ↵Kevin May
Opaque Delegate Signed-off-by: Kevin May <kevin.may@arm.com> Change-Id: I38304abce1a417bb69aced2a5b38e976ea0cbbc0
2023-04-26IVGCVSW-7579 IVGCVSW-7581 IVGCVSW-7583 Implement Comparison, Concat and Mean ↵Matthew Sloyan
in Opaque Delegate * Removed input slot check from Connect function as number of TFLite and Arm NN inputs can differ. * Moved SetupConcatViewOrigin function to DelegateUtils.hpp * Simplified validation checks in VistConvolution functions as IsValid and IsDynamic were already being called. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I858dbe4b643f9d350d9c38ea255ce5effbda4612
2023-04-24IVGCVSW-7584 Implement Conv2d and DepthwiseConv2d operators for Opaque DelegateMatthew Sloyan
* Added VisitConvolution2d and VisitDepthwiseConv2d functions to Convolution2d.hpp. * Enabled Convolution2d and DepthwiseConv2d tests. * Moved TransposeConvolution2d tests to separate file. * Added Opaque Delegate shared functions. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Ica10c9469fc830f512edad1ad79884f90ae511d0
2023-04-19GitHub #640 Add support for CEIL operatorTeresa Charlin
* Reference workload * TfLite Delegate * TfLite Parser * Serializer and Deserializer * Changed fallback tests in delegate to use COS instead of CEIL Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I36e0dbff33694182d1dba0c95d463506428e2f04
2023-04-13IVGCVSW-7564 Implement Cast operator for Opaque DelegateRyan OShea
* Adds VisitCast function to Redefine.hpp * Enables Cast Test for OpaqueUnitTests * Various Fixes to the opaque delegate to allow operator to run Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I43d42eea5c987d6aed8a0f909a6bf583fddcc94e
2023-04-12IVGCVSW-7197 Implement Pimpl Idiom for OptimizerOptionsJohn Mcloughlin
Signed-off-by: John Mcloughlin <john.mcloughlin@arm.com> Change-Id: Id4bdc31e3e6f18ccaef232c29a2d2825c915b21c
2023-04-07IVGCVSW-7563 Implement DelegateTestInterpreter for opaque delegateMatthew Sloyan
* Added opaque delegate DelegateTestInterpreter implementation * Moved classic specific tests to ArmnnClassicDelegateTest.cpp * Moved opaque specific tests to ArmnnOpaqueDelegateTest.cpp * Removed ArmnnDelegateTest.cpp * Moved TfLiteStableDelegate implementation to armnn_delegate.cpp Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Ifc92b6fb38dc370f3fb88a4daca56d457e74bc2e
2023-04-06IVGCVSW-7197 Implement Pimpl Idiom for Delegate OptionsJohn McLoughlin
* ABI break on the delegate interface. Bumping the version number. Signed-off-by: John Mcloughlin <john.mcloughlin@arm.com> Change-Id: I6ef3bc1ea240ef08b67bb3cb9d363a5bbbbdd906
2023-04-05IVGCVSW-7559 Implement DoPrepare with registrationMatthew Sloyan
* Added ArmnnOpaqueDelegate::IdentifyOperatorsToDelegate implementation. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I0b65847358d339a15fc3f729f89deb9b86da0c66
2023-04-05IVGCVSW-7562 Implement DelegateTestInterpreter for classic delegateMatthew Sloyan
* Updated all tests to use new DelegateTestInterpreter. * Fixed some unit tests where the shape was incorrect. * Add file identifier to FlatBuffersBuilder, as it is required for validation when creating the model using new API. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I1c4f5464367b35d4528571fa94d14bfaef18fb4d
2023-04-05IVGCVSW-7560 Implement Armnn Subgraph in Opaque DelegateRyan OShea
* Implement Armnn Subgraph methods in opaque armnn_delegate.cpp * Temporarily remove opaque delegate from DelegateUnitTests Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I43557eb0b96a22c896890593a3ed9ca9744994bb
2023-04-03IVGCVSW-7558 Implement Delegate UtilsMatthew Sloyan
* Implement OpaqueDelegateUtils.hpp using new accessors. * Moved classic delegate utils to ClassicDelegateUtils.hpp. * DelegateUtils.hpp now contains common utils. * Removed unused ConnectConstant function. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I0fa611c82f5c06b0b7e0c37bfc343e09fb6a96c9
2023-03-31IVGCVSW-7556 Introduce Opaque Delegate APIFrancis Murtagh
* Also added cmake for the new layers to reduce merge conflicts. Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Change-Id: Ieb59aa2b7e2a18c57c9357b8d5b5cd63d8211c85
2023-03-28IVGCVSW-7555 Restructure DelegateTeresa Charlin
* New folders created: * common is for common code where TfLite API is not used * classic is for existing delegate implementations * opaque is for new opaque delegate implementation, * tests is for shared between existing Delegate and Opaque Delegate which have test utils to work which delegate to use. * Existing delegate is built to libarmnnDelegate.so and opaque delegate is built as libarmnnOpaqueDelegate.so * Opaque structure is introduced but no API is added yet. * CmakeList.txt and delegate/CMakeList.txt have been modified and 2 new CmakeList.txt added * Rename BUILD_ARMNN_TFLITE_DELEGATE as BUILD_CLASSIC_DELEGATE * Rename BUILD_ARMNN_TFLITE_OPAQUE_DELEGATE as BUILD_OPAQUE_DELEGATE Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ib682b9ad0ac8d8acdc4ec6d9099bb0008a9fe8ed
2023-03-22IVGCVSW-7526 Upgrade ArmNN to Tensorflow 2.12Ryan OShea
When creating a flatbuffers model, we need to provide an empty buffer 0 that is reserved by tensorflow. When creating empty buffers for inputs and outputs we can not pass in an empty vector, or tflite will assume that we know how many bytes to allocate in advance. Instead we need to only pass in the builder. * Update libraries in FindTfLite.cmake * Add nullptr to delegate struct for OpaqueDelegateBuilder * Fix issue in unit tests where Flatbuffers model was not being parsed by tflite * Tensorflow 2.12 now includes C++ 17 features. Update our cmake build to require a compiler to support these features. * Change minimum cmake in Arm NN to 3.7 as that's the minimum for the delegate build. Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Signed-off-by: Narumol Prangnawarat <narumol.prangnawarat@arm.com> Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I7d15b196b8c59b1914f8fc1c4c2f8960630c069c
2023-03-14IVGCVSW-3808 Add ElementwiseBinaryLayerMike Kelly
!android-nn-driver:9329 * Added ElementwiseBinaryLayer that can represent all ElementwiseBinary operations including Add, Div, Sub, Maximum, Mul and Minimum. * Updated Delegate to use ElementwiseBinaryLayer instead of the Add, Div, Sub, Maximum, Mul and Minimum layers. * Updated Deserializer to use ElementwiseBinaryLayer instead of the Add, Div, Sub, Maximum, Mul and Minimum layers. * Updated OnnxParser to use ElementwiseBinaryLayer instead of the Add layer. * Updated TfLiteParser to use ElementwiseBinaryLayer instead of the Add, Div, Sub, Maximum, Mul and Minimum layers. * Updated CL and Neon tests to use ElementwiseBinaryLayer. * Updated CL and Neon Backend Specific Optimizations to accept ElementBinaryLayers as well as Add, Div, Mul, Sub, Maximum and Minimum layers. Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I7cbb96b60eb01f0e2b57b0541016d48a08b86c75
2023-02-28Add constant version of IConnectableLayer::GetConstantTensorsByRefMatthew Bentham
This makes it easier to use, particularly in backends where it is common to pass around const pointers to IConnectableLayer. The non-constant version is rewritten to use the constant version. Signed-off-by: Matthew Bentham <matthew.bentham@arm.com> Change-Id: Id3a8384447e93c213299a85ade9a667df5960534
2023-02-21MLCE-753 Expand Tensorshape for relevent layers before verifying supportRyan OShea
Previously we were adding a reshape layer to "broadcast" tensors for elementwise operations. This broadcast was happening too late and was really just an expand dims. This was breaking the constant attributes of tensors and layer support of certain backends. * Remove addition of reshape layer when expanding dimensions * Replace broadcast function with expand dims to equal rank function * Fix some error status checks in various layers * Add new TensorUtil function that expands dims to a defined rank * Add unit tests to new TensorUtil function Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I31aca47c98075fef4f86864a15470f5faa55ab8d
2023-02-06IVGCVSW-7476 Remove ProfilingOptions from DelegateOptions.Colm Donelan
ProfilingOptions is not used in DelegateOptions. Instead the parameters are passed in through the RuntimeOptions. This is done in ExecuteNetwork and TfliteExecutor. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: Iaab3d4ef277c47e1ff82a51ba2648f5f51ec3e2c
2023-02-03IVGCVSW-7501 Allow constant tensors as inputs for input data in the delegateRyan OShea
In the TLCT tests we were failing many tests because they used constant tensors as data input for the layers. We had the functionality in place but we didnt have it spread across the visit functions. * Check if inputs are constant tensors and attempt to assign them to input slot of layers. * Add missing checks to some functions that return a kTfLiteStatus so we can see if they fail * Clean up CreateConstTensor function Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I8610b770aea56932a98f91c961d59b3de47c2ab5
2023-01-30IVGCVSW-7475 Fix uncaught warnings treated as errors in delegate release buildRyan OShea
* Add initialization of activation type in delegate layers with fused activations * Remove unused variable activationOutputInfo on layers with fused activations Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: Ieb2ba00516a159639871c391069faf162cbb666d
2023-01-27IVGCVSW-7450 Fix delegate fallback when fused activation is unsupportedRyan OShea
In layers that support fused activations, we check for activation layer support after we already create the base layer. This breaks the fallback as we already added the base layer to the graph. * Creates ValidateFusedActivation shared function * Moves Activation validation higher in the VisitFunction Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I239af360923f695fc374ddeaeefa24c062eaf9e8
2023-01-27IVGCVSW-7441 Checking for constant input tensors before populating.Colm Donelan
* When the tfLiteExecutor attempts to populate the input tensors it did not check whether the tensor was constant. This was causing segmentation faults. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I80a4cc788de4ffe08afb2df9185d04fcb8b27c3a
2023-01-24IVGCVSW-7277 Fixed issues with FullyConnected on certain TFLite modelsMike Kelly
* TFLite Parser: * Fixed issue in ParseReshape where the targetShape wasn't always calculated correctly * Fixed issue in ParseFullyConnected where the wrong name was used for the ReshapeLayer * Added an ExpandDims to the FullyConnected to ensure that we reshape the output correctly * TFLite Delegate: * Added an ExpandDims to the FullyConnected to ensure that we reshape the output correctly Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I129dfcb8543f8a3a297c0589c841be20ef3b6407
2023-01-24IVGCVSW-7455 Workaround to allow CLBatchMatMul to parse some 4D modelsMike Kelly
* Added ability to reduce dimension sizes when calling BuildArmComputeTensorInfo or BuildArmComputeTensorShapes, this will attempt to remove leading 1s in order to squeeze the number of dimensions but retain the size. * Changed ClBatchMatMulWorkload to attempt to squeeze the number of dimensions to 3 as the CL Gemm Kernel can only support up to 3 dimensions. Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I6b3d0886c5b97fdb686838fc3dc292833ddc4643
2023-01-24IVGCVSW-7297 When creating multiple Executors only the lastMike Kelly
one works fine * All ArmNNExecutors now share a single IRuntime. * All armnn_delegates now share a single IRuntime. * Increased delegate major version. Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I95cbdc32655ec0beb476dbb2d60f1a0209df8f04
2023-01-13Update license from commit:Ryan OShea
Fix delegate fallback during VisitNode 05b6a3e5946a3f58b6f5b0caface9153a1c2b364 Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: Iba2c70fd49ce8d7c49fb6deb7183b42db2cc68c7
2023-01-12Fix delegate fallback during VisitNodeRyan OShea
During VisitNode throwing an ArmNN exception incorrectly terminates the process instead of handing over to tflite * Catches ArmNN exceptions during VisitNode Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I6c71be11e9b73694747b27fe9febab8d9669b4d4
2023-01-09IVGCVSW-6493 Add GpuAcc Batch MatMul workload Fp32Teresa Charlin
* GpuAcc only supports up to 3D, so no 4D test have been added Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ie926cd45c350be624cbdc6cb27c89d2d3f60884b
2023-01-06IVGCVSW-7031 Generate static execute networkRyan OShea
* Build ExecNet lib dependencies as object libs except libarmnn * Disable PIPE when building static ExecNet * Remove multiple definition from AsyncExecutionCallback * Disable DynamicBackend for ExecNet Static build * Disable inference tests for TfLiteParser and ONNX during static ExecNet * Remove Tensorflow Parser if condition * Add Disable thread macro to InferenceModel * Don't compile dynamic backend symbols in Runtime.cpp for Baremetal and Exenet Static Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: If41c063eab5f05b3df0a6e064924a36a177f116a
2023-01-05Add backend name to delegate test names that did not have it.Teresa Charlin
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: If9531f74d0b503c36f1b72ef51eee7e9f65559f2
2022-12-20IVGCVSW-7409 GPU backend options not being passed from the delegate.Colm Donelan
Two problems here: * First the Delegate was using the parameter options after the execution of std::move on it. * In ExecuteNetworkParams 3 GPU backend options were instead being set as optimizer options. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I61c7fad8a5819a0a4aec0243899019a342c5cc5f
2022-12-12Optimize the calling of IsLayerSupported().Cathal Corbett
* Done as part of 22.11/23.02 innovation days. * IsLayerSupported() is called in model prepare (delegate, android-nn-driver and shim/support_library) and again in ArmNN once model otimization is performed. * From calling IsLayerSupported() the first time, we should know that the layers are supported and what backend they are supported on. * Solution is to set the BackendId of the IConnectableLayer when IsLayerSupported() is called the first time, * In the Optimize() function we then check if the backend is set. If so, we do not call IsLayerSupported() again. * In the case a layer that is supported gets optimized, then the BackendId of that layer get set to "Unknown" for the new optimized layer and IsLayerSupported() will get called on the newly optimized layer. * Includes bug fix IVGCVSW-7213 for Android Mean FP16 CpuAcc tests. Also related to bug IVGCVSW-7211. Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I7a7820d0cdb079ffb5a3a2e0c44e252f652df53b
2022-11-22IVGCVSW-6980 Delegate support for slice operatorCathal Corbett
Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I90d800160b070e25d999b5102a7ce6d3e0ed6a81
2022-11-16IVGCVSW-7214 Disable BF16-Turbo-Mode and remove conversion layersRyan OShea
- Remove Bf16ToFp32 Conversion Layer - Remove Fp32ToBf16 Conversion Layer - Remove B16 Conversion tests * Throw exception if m_ReduceFp32ToBf16 optimzer option is set to true * Provide comments to enable fast math in order to use bf16 * Update docs to inform users to enable fast math for bf16 Execute Network Changes * Require bf16_turbo_mode to also have fast_math_enabled set to true - Remove setting m_ReduceFp32ToBf16 optimizer option Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: Ibaa6da9d29c96a1ce32ff5196b0847fde9f04a1c
2022-11-15Minor error formatting fixes.Colm Donelan
Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I17823fb8b6bbabc4da327187167ce9582ee29b32
2022-10-28IVGCVSW-7296 REDUCE_PROD tests fail when using Tf 2.10Teresa Charlin
* In TF what ArmNN calls quantized data types can be non-quantized as well. * This patch creates 2 models: * ArmNN: model where int8 and uint8 will always be quantized, but scale can be 1 and offset 0 * TFLite: model where int8 and uint8 can be quantized and non-quantized Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Id960f2f30988f2bbec88cb4e0c52c189ac957bae
2022-10-28IVGCVSW-6494 Add CpuAcc Batch MatMul Workload Fp32Teresa Charlin
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I2def6995f81d33e68f1ea45d8d19a1e6294049b1
2022-10-23IVGCVSW-7126 Modify odrer of Tf 2.10 librariesColm Donelan
* Modify the order of Tf absl libraries in delegate build. * Disable failing delegate unit tests. * Add -Wno-comment to delegate ExecuteNetwork build. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I9e4696e032a5e26df8ba578b980456cbac4d0ab0
2022-10-21IVGCVSW-7126 Update Arm NN to Tensorflow 2.10Colm Donelan
* Modify get_tensorflow.sh to point to the 2.10 Tensorflow tag. * Modify FindTfLite.cmake to handle the break up of libruy.a * Modify armnn_delegate.cpp to add registration_external field. * Prevent the serializer/deserializer from using a system installed version of flatc. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I688f33f387924ba29bf400f60e56f73b2329fdc1
2022-09-22IVGCVSW-6498 Add Support for Batch MatMul to TfLite DelegateRyan OShea
* Creates delegate/src/BatchMatMul.hpp * Add VisitBatchMatMul function * Add BatchMatMul to switch in armnn_delegate * Creates delegate/src/test/BatchMatMulTest.cpp * Creates delegate/src/test/BatchMatMulTestHelper.hpp * Add Int8 and Fp32 unit tests on ref backend * Add BatchMatMul to delegate supported ops Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I50e61314cf063f986c8a0f7d508847a96953735e
2022-08-29IVGCVSW-6779 Add build-tool Docs and update docsJames Conroy
* Added Arm NN Build Tool README.md and license. * Added Dockerfile diagram image and license. * Updated existing guides: added note to the cross compile guide to use build-tool, removed content from the Delegate BuildGuideNative.md. * Added Arm NN logo image to homepage README.md and added license. Signed-off-by: James Conroy <james.conroy@arm.com> Change-Id: I9306ad24165cf3b0203c2a1852ec3afd718e9eab
2022-08-29IVGCVSW-6603 'Add a no fallback mode to the TfLite Delegate'Sadik Armagan
* Added disable-tflite-runtime-fallback option to armnn_delegate * Updated armnn_delegate version Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: I449b16404d3ffe98e6dac52a43e7c25225addd73
2022-08-29Fixing documents for 22.08Nikhil Raj
* Fixed broken links * Updated link to bazel guide * Fixed versioning issues in packaging doc * Updated paths to binary files Signed-off-by: Nikhil Raj <nikhil.raj@arm.com> Change-Id: I6120bb2dd56d55a994cd12d99bed054ce4d7c058
2022-08-29IVGCVSW-7154 'Constant Tensors As Inputs'Sadik Armagan
* Fixed the issues about converting Conv2D and DepthwiseConv2d input issues * Read 1D input tensors that do not have shape specified Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: I12f3d1c57a2afedac42c6e7b31e4b1fc689abeca
2022-08-05Fix '#if defined(ARMNN_POST_TFLITE_2_3)' in TfLiteParser Conv3D test.Cathal Corbett
Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I5c68b81a67fc2b5a33cf62753351440564bb868e
2022-07-28GitHub #650: DelegateQuickStartGuide.md errors fix.Cathal Corbett
Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: If24cad1d5d403e195d7adc539afb83cc5df134d1
2022-07-27Documentation refactor and updates.Cathal Corbett
* Includes updating ContributorGuide.md with information section 'Contributing code to Arm NN'. * Renaming of ContributorGuide.md to CONTRIBUTING.md. Also renamed associated licence file and links in other pages that call the md file. Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: Ic3336d78b64ae41af07ea9fd03631bb161613c4b
generated by cgit v1.2.1 (git 2.23.0) at 2024-07-04 13:04:51 +0000