aboutsummaryrefslogtreecommitdiff
path: root/delegate
AgeCommit message (Collapse)Author
2023-05-08IVGCVSW-7307 Add CpuAcc Batch MatMul WorkloadTeresa Charlin
* Call dedicated MatMul kernel in ACL * Add int8 tests * Add int8 to documentation * Force tensors to be dynamic (nonConst) as per request of ACL Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I992ae9aae1174214607bf29305f21cdeaf3fdc1b
2023-05-08BugFix: Calculate explicit padding for Delegate Transpose Convolution using ↵Matthew Sloyan
output size * Added fix to Classic and Opaque Delegate which now matches the TfLiteParser. * Removed uses of single parameter IsValid function in classic Convolution visit functions. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I5be91ec25c11354a9b8472d0a429e71e02df6f9c
2023-05-05MLCE-1050 Error handing Slice operatorsMike Kelly
* If the dimension Size[n] in a Slice is -1 then it should be treated as "InputShape[n] - Begin[n]" but the Delegate simply cast the Size to uint and treated it as 4294967295. * Added the layer name that includes the node index to the Slice to aid debugging. Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I45fa88b24982c3c97f48d0dc05cf7d9bb6db4074
2023-05-05IVGCVSW-7423 Add ArmnnDelegatePluginNarumol Prangnawarat
Signed-off-by: Narumol Prangnawarat <narumol.prangnawarat@arm.com> Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: Ie02021ac56a512598760e4c6d05ef1a80f4aec8d
2023-05-05IVGCVSW-7618 Implement UnidirectionalSequenceLstm operator for Opaque DelegateMatthew Sloyan
* Intermediate tensors aren't accessible through the new Opaque interface yet, so we have to cast to TfLiteNode for now. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Ifd91131e5d5ff6cc057b80729fea9afa68ed240b
2023-05-04IVGCVSW-7605 IVGCVSW-7604 Implement Squeeze and ExpandDims operators for ↵Matthew Sloyan
Classic and Opaque Delegate * Implemented unsupported operators in Classic Delegate. * Added unit tests. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Ib39eeea53c114b15943e8dc2e796ce64c40cb3a5
2023-05-04IVGCVSW-7612 Implement Split OperatorsDavid Monahan
* Added implementation for Split / SplitV in Opaque Delegate Signed-off-by: David Monahan <david.monahan@arm.com> Change-Id: I1ad2925aece7d4e0567e63ecb768348152a70a7a
2023-05-02IVGCVSW-7603 Implement Reshape operators for Opaque DelegateMatthew Sloyan
* Moved CreateOutputTensorShape function to common DelegateUtils.hpp Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I3d8a9834ecd6b7cda170cce958677a0dde62824a
2023-04-28IVGCVSW-7610 IVGCVSW-7616 Slice and StridedSlice for opaque delegateTeresa Charlin
* Change alignment in Shape.hpp Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ibc21cf5c56a1ba2daa7507a5d5c2b7311756f17f
2023-04-28IVGCVSW-7606 IVGCVSW-7607 Add Resize and Reduce to Opaque DelegateJohn Mcloughlin
* Added 2 opaque delegate operators and associated test cases * Removed IsDynamicTensor check from BatchMatMul as covered by IsValid. Signed-off-by: John Mcloughlin <john.mcloughlin@arm.com> Change-Id: If7c58cb23ae5c5b8a9451dddfd7b6dfcbf248d4c
2023-04-28IVGCVSW-7611 IVGCVSW-7614 IVGCVSW-7615 IVGCVSW-7617 Softmax, SpaceToDepth, ↵Teresa Charlin
DepthToSpace and Tranpose for opaque delegate Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ie0c608f94a76956e9be75f555824cef865cab395
2023-04-28IVGCVSW-7602 IVGCVSW-7602 Implement opaque delegate for Quantize + ↵Francis Murtagh
Dequantize operator Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Change-Id: I318cb25f526dfe0f7aa6afcf77971afd8d437209
2023-04-28IVGCVSW-7596 IVGCVSW-7619 IVGCVSW-7597 Pack, Unpack and Pad for opaque delegateTeresa Charlin
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I25415793497f0ee08d880539e265b133875a20f7
2023-04-28IVGCVSW-7588 Implement ElementWiseBinary Op for Opaque DelegateDavid Monahan
* Added visit functions for ElementwiseBinary Ops * Moved MultiLayerFacade.hpp to common directory and updated both delegates to use it Signed-off-by: David Monahan <david.monahan@arm.com> Change-Id: I84b8bd74d15a194895e63da47c29be994531a889
2023-04-28IVGCVSW-7608 IVGCVSW-7594 IVGCVSW-7598 IVGCVSW-7599 Implement Floor,Matthew Sloyan
Lstm, Pooling2d and Pooling3d operators for Opaque Delegate Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Ic9af1c50589285ab359661699d32a889cd267cd9
2023-04-27IVGCVSW-7576 IVGCVSW-7609 Add BatchMatMul and Shape to Opaque DelegateJohn Mcloughlin
Signed-off-by: John Mcloughlin <john.mcloughlin@arm.com> Change-Id: Id4b02b951ed81c69171f6af2d0d327175c6e5d3c
2023-04-27IVGCVSW-7584 IVGCVSW-7585 Implement Conv3d and TransposeConv2d operators for ↵Francis Murtagh
opaque delegate * Add check for TF version so tests work with opaque and classic Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Change-Id: I3a6699150afabfc6200e1cd4d264a1f7c65c5f60
2023-04-27IVGCVSW-7589 IVGCVSW-7595 IVGCVSW-7593 ElementwiseUnary, Normalization and ↵Teresa Charlin
LogicalBinary operators for opaque delegate * Report the operator as part of the layer name for: - LogicalBinary, - ElementwiseUnary - Comparison - Activation * Fixing indentation in Gather.hpp * Removing not needed includes in Gather, GatherNd and Comparison * Correct end of namespace comment in Comparison * Correct log from TfLiteArmnnDelegate to TfLiteArmnnOpaqueDelegate Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ia0d497709309e912d31eb4b6db0fef9e79b7a3af
2023-04-27IVGCVSW-7574 IVGCVSW-7590 IVGCVSW-7600 Implement Activation, FullyConnected ↵Matthew Sloyan
and Prelu operators for Opaque Delegate * Added missing headers to opaque/CMakeLists.txt (Control and Comparison) * Cleaned up Control.hpp headers. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I442edb9c467b515b130fbaf02879f0802006255f
2023-04-27IVGCVSW-7591 IVGCVSW-7592 Add Gather and GatherNd to Opaque DelegateKevin May
Signed-off-by: Kevin May <kevin.may@arm.com> Change-Id: Id2b6a4f70b1cb50e5f7f7ab4e30487b3816c9ad4
2023-04-26IVGCVSW-7575 Implement opaque delegate for ArgMinMax operatorJohn Mcloughlin
* Added Opaque ArgMinMax and associated test cases Signed-off-by: John Mcloughlin <john.mcloughlin@arm.com> Change-Id: I098b94cc35707370a0bbc7456bfdd48bb47432f0
2023-04-26IVGCVSW-7577, IVGCVSW-7578 Implement BatchToSpaceNd and SpaceToBatchNd in ↵Kevin May
Opaque Delegate Signed-off-by: Kevin May <kevin.may@arm.com> Change-Id: I38304abce1a417bb69aced2a5b38e976ea0cbbc0
2023-04-26IVGCVSW-7579 IVGCVSW-7581 IVGCVSW-7583 Implement Comparison, Concat and Mean ↵Matthew Sloyan
in Opaque Delegate * Removed input slot check from Connect function as number of TFLite and Arm NN inputs can differ. * Moved SetupConcatViewOrigin function to DelegateUtils.hpp * Simplified validation checks in VistConvolution functions as IsValid and IsDynamic were already being called. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I858dbe4b643f9d350d9c38ea255ce5effbda4612
2023-04-24IVGCVSW-7584 Implement Conv2d and DepthwiseConv2d operators for Opaque DelegateMatthew Sloyan
* Added VisitConvolution2d and VisitDepthwiseConv2d functions to Convolution2d.hpp. * Enabled Convolution2d and DepthwiseConv2d tests. * Moved TransposeConvolution2d tests to separate file. * Added Opaque Delegate shared functions. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Ica10c9469fc830f512edad1ad79884f90ae511d0
2023-04-19GitHub #640 Add support for CEIL operatorTeresa Charlin
* Reference workload * TfLite Delegate * TfLite Parser * Serializer and Deserializer * Changed fallback tests in delegate to use COS instead of CEIL Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I36e0dbff33694182d1dba0c95d463506428e2f04
2023-04-13IVGCVSW-7564 Implement Cast operator for Opaque DelegateRyan OShea
* Adds VisitCast function to Redefine.hpp * Enables Cast Test for OpaqueUnitTests * Various Fixes to the opaque delegate to allow operator to run Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I43d42eea5c987d6aed8a0f909a6bf583fddcc94e
2023-04-12IVGCVSW-7197 Implement Pimpl Idiom for OptimizerOptionsJohn Mcloughlin
Signed-off-by: John Mcloughlin <john.mcloughlin@arm.com> Change-Id: Id4bdc31e3e6f18ccaef232c29a2d2825c915b21c
2023-04-07IVGCVSW-7563 Implement DelegateTestInterpreter for opaque delegateMatthew Sloyan
* Added opaque delegate DelegateTestInterpreter implementation * Moved classic specific tests to ArmnnClassicDelegateTest.cpp * Moved opaque specific tests to ArmnnOpaqueDelegateTest.cpp * Removed ArmnnDelegateTest.cpp * Moved TfLiteStableDelegate implementation to armnn_delegate.cpp Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Ifc92b6fb38dc370f3fb88a4daca56d457e74bc2e
2023-04-06IVGCVSW-7197 Implement Pimpl Idiom for Delegate OptionsJohn McLoughlin
* ABI break on the delegate interface. Bumping the version number. Signed-off-by: John Mcloughlin <john.mcloughlin@arm.com> Change-Id: I6ef3bc1ea240ef08b67bb3cb9d363a5bbbbdd906
2023-04-05IVGCVSW-7559 Implement DoPrepare with registrationMatthew Sloyan
* Added ArmnnOpaqueDelegate::IdentifyOperatorsToDelegate implementation. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I0b65847358d339a15fc3f729f89deb9b86da0c66
2023-04-05IVGCVSW-7562 Implement DelegateTestInterpreter for classic delegateMatthew Sloyan
* Updated all tests to use new DelegateTestInterpreter. * Fixed some unit tests where the shape was incorrect. * Add file identifier to FlatBuffersBuilder, as it is required for validation when creating the model using new API. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I1c4f5464367b35d4528571fa94d14bfaef18fb4d
2023-04-05IVGCVSW-7560 Implement Armnn Subgraph in Opaque DelegateRyan OShea
* Implement Armnn Subgraph methods in opaque armnn_delegate.cpp * Temporarily remove opaque delegate from DelegateUnitTests Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I43557eb0b96a22c896890593a3ed9ca9744994bb
2023-04-03IVGCVSW-7558 Implement Delegate UtilsMatthew Sloyan
* Implement OpaqueDelegateUtils.hpp using new accessors. * Moved classic delegate utils to ClassicDelegateUtils.hpp. * DelegateUtils.hpp now contains common utils. * Removed unused ConnectConstant function. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I0fa611c82f5c06b0b7e0c37bfc343e09fb6a96c9
2023-03-31IVGCVSW-7556 Introduce Opaque Delegate APIFrancis Murtagh
* Also added cmake for the new layers to reduce merge conflicts. Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Change-Id: Ieb59aa2b7e2a18c57c9357b8d5b5cd63d8211c85
2023-03-28IVGCVSW-7555 Restructure DelegateTeresa Charlin
* New folders created: * common is for common code where TfLite API is not used * classic is for existing delegate implementations * opaque is for new opaque delegate implementation, * tests is for shared between existing Delegate and Opaque Delegate which have test utils to work which delegate to use. * Existing delegate is built to libarmnnDelegate.so and opaque delegate is built as libarmnnOpaqueDelegate.so * Opaque structure is introduced but no API is added yet. * CmakeList.txt and delegate/CMakeList.txt have been modified and 2 new CmakeList.txt added * Rename BUILD_ARMNN_TFLITE_DELEGATE as BUILD_CLASSIC_DELEGATE * Rename BUILD_ARMNN_TFLITE_OPAQUE_DELEGATE as BUILD_OPAQUE_DELEGATE Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ib682b9ad0ac8d8acdc4ec6d9099bb0008a9fe8ed
2023-03-22IVGCVSW-7526 Upgrade ArmNN to Tensorflow 2.12Ryan OShea
When creating a flatbuffers model, we need to provide an empty buffer 0 that is reserved by tensorflow. When creating empty buffers for inputs and outputs we can not pass in an empty vector, or tflite will assume that we know how many bytes to allocate in advance. Instead we need to only pass in the builder. * Update libraries in FindTfLite.cmake * Add nullptr to delegate struct for OpaqueDelegateBuilder * Fix issue in unit tests where Flatbuffers model was not being parsed by tflite * Tensorflow 2.12 now includes C++ 17 features. Update our cmake build to require a compiler to support these features. * Change minimum cmake in Arm NN to 3.7 as that's the minimum for the delegate build. Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Signed-off-by: Narumol Prangnawarat <narumol.prangnawarat@arm.com> Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I7d15b196b8c59b1914f8fc1c4c2f8960630c069c
2023-03-14IVGCVSW-3808 Add ElementwiseBinaryLayerMike Kelly
!android-nn-driver:9329 * Added ElementwiseBinaryLayer that can represent all ElementwiseBinary operations including Add, Div, Sub, Maximum, Mul and Minimum. * Updated Delegate to use ElementwiseBinaryLayer instead of the Add, Div, Sub, Maximum, Mul and Minimum layers. * Updated Deserializer to use ElementwiseBinaryLayer instead of the Add, Div, Sub, Maximum, Mul and Minimum layers. * Updated OnnxParser to use ElementwiseBinaryLayer instead of the Add layer. * Updated TfLiteParser to use ElementwiseBinaryLayer instead of the Add, Div, Sub, Maximum, Mul and Minimum layers. * Updated CL and Neon tests to use ElementwiseBinaryLayer. * Updated CL and Neon Backend Specific Optimizations to accept ElementBinaryLayers as well as Add, Div, Mul, Sub, Maximum and Minimum layers. Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I7cbb96b60eb01f0e2b57b0541016d48a08b86c75
2023-02-28Add constant version of IConnectableLayer::GetConstantTensorsByRefMatthew Bentham
This makes it easier to use, particularly in backends where it is common to pass around const pointers to IConnectableLayer. The non-constant version is rewritten to use the constant version. Signed-off-by: Matthew Bentham <matthew.bentham@arm.com> Change-Id: Id3a8384447e93c213299a85ade9a667df5960534
2023-02-21MLCE-753 Expand Tensorshape for relevent layers before verifying supportRyan OShea
Previously we were adding a reshape layer to "broadcast" tensors for elementwise operations. This broadcast was happening too late and was really just an expand dims. This was breaking the constant attributes of tensors and layer support of certain backends. * Remove addition of reshape layer when expanding dimensions * Replace broadcast function with expand dims to equal rank function * Fix some error status checks in various layers * Add new TensorUtil function that expands dims to a defined rank * Add unit tests to new TensorUtil function Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I31aca47c98075fef4f86864a15470f5faa55ab8d
2023-02-06IVGCVSW-7476 Remove ProfilingOptions from DelegateOptions.Colm Donelan
ProfilingOptions is not used in DelegateOptions. Instead the parameters are passed in through the RuntimeOptions. This is done in ExecuteNetwork and TfliteExecutor. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: Iaab3d4ef277c47e1ff82a51ba2648f5f51ec3e2c
2023-02-03IVGCVSW-7501 Allow constant tensors as inputs for input data in the delegateRyan OShea
In the TLCT tests we were failing many tests because they used constant tensors as data input for the layers. We had the functionality in place but we didnt have it spread across the visit functions. * Check if inputs are constant tensors and attempt to assign them to input slot of layers. * Add missing checks to some functions that return a kTfLiteStatus so we can see if they fail * Clean up CreateConstTensor function Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I8610b770aea56932a98f91c961d59b3de47c2ab5
2023-01-30IVGCVSW-7475 Fix uncaught warnings treated as errors in delegate release buildRyan OShea
* Add initialization of activation type in delegate layers with fused activations * Remove unused variable activationOutputInfo on layers with fused activations Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: Ieb2ba00516a159639871c391069faf162cbb666d
2023-01-27IVGCVSW-7450 Fix delegate fallback when fused activation is unsupportedRyan OShea
In layers that support fused activations, we check for activation layer support after we already create the base layer. This breaks the fallback as we already added the base layer to the graph. * Creates ValidateFusedActivation shared function * Moves Activation validation higher in the VisitFunction Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I239af360923f695fc374ddeaeefa24c062eaf9e8
2023-01-27IVGCVSW-7441 Checking for constant input tensors before populating.Colm Donelan
* When the tfLiteExecutor attempts to populate the input tensors it did not check whether the tensor was constant. This was causing segmentation faults. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I80a4cc788de4ffe08afb2df9185d04fcb8b27c3a
2023-01-24IVGCVSW-7277 Fixed issues with FullyConnected on certain TFLite modelsMike Kelly
* TFLite Parser: * Fixed issue in ParseReshape where the targetShape wasn't always calculated correctly * Fixed issue in ParseFullyConnected where the wrong name was used for the ReshapeLayer * Added an ExpandDims to the FullyConnected to ensure that we reshape the output correctly * TFLite Delegate: * Added an ExpandDims to the FullyConnected to ensure that we reshape the output correctly Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I129dfcb8543f8a3a297c0589c841be20ef3b6407
2023-01-24IVGCVSW-7455 Workaround to allow CLBatchMatMul to parse some 4D modelsMike Kelly
* Added ability to reduce dimension sizes when calling BuildArmComputeTensorInfo or BuildArmComputeTensorShapes, this will attempt to remove leading 1s in order to squeeze the number of dimensions but retain the size. * Changed ClBatchMatMulWorkload to attempt to squeeze the number of dimensions to 3 as the CL Gemm Kernel can only support up to 3 dimensions. Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I6b3d0886c5b97fdb686838fc3dc292833ddc4643
2023-01-24IVGCVSW-7297 When creating multiple Executors only the lastMike Kelly
one works fine * All ArmNNExecutors now share a single IRuntime. * All armnn_delegates now share a single IRuntime. * Increased delegate major version. Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I95cbdc32655ec0beb476dbb2d60f1a0209df8f04
2023-01-13Update license from commit:Ryan OShea
Fix delegate fallback during VisitNode 05b6a3e5946a3f58b6f5b0caface9153a1c2b364 Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: Iba2c70fd49ce8d7c49fb6deb7183b42db2cc68c7
2023-01-12Fix delegate fallback during VisitNodeRyan OShea
During VisitNode throwing an ArmNN exception incorrectly terminates the process instead of handing over to tflite * Catches ArmNN exceptions during VisitNode Signed-off-by: Ryan OShea <ryan.oshea3@arm.com> Change-Id: I6c71be11e9b73694747b27fe9febab8d9669b4d4
2023-01-09IVGCVSW-6493 Add GpuAcc Batch MatMul workload Fp32Teresa Charlin
* GpuAcc only supports up to 3D, so no 4D test have been added Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ie926cd45c350be624cbdc6cb27c89d2d3f60884b
generated by cgit v1.2.1 (git 2.23.0) at 2024-06-10 02:35:32 +0000