aboutsummaryrefslogtreecommitdiff
path: root/src/backends/reference
AgeCommit message (Collapse)Author
2021-11-08IVGCVSW-6527 Support the new memory API in loaded networkFinn Williams
* enable external memory management for neon and ref backends * change m_TensorMemoryVector to hold shared pointers * change input layer backend Id to match backend id of connected layer Signed-off-by: Finn Williams <finn.williams@arm.com> Change-Id: I2216a724028312eb101b290df3f224177826b1a0
2021-10-27IVGCVSW-6469 Add MirrorPad FrontEnd and Ref SupportMatthew Sloyan
* Added PaddingMode enum to PaddingDescriptor to enable Symmetric and Reflect padding. * Added Symmetric and Reflect Ref implementation. * Added Serializer & Deserializer support. * Added unit tests. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I4bed907b31742b32ccefe5e8ca39a6f1e5bd9dee
2021-10-20Add ConstTensorsAsInput support for Conv3dMatthew Sloyan
* Constant weights and biases are now stored as Constant layers. * Updated Serializer, Deserializer and unit tests to reflect this. * Updated TfLiteParser. * Updated Ref backend to handle constant weights and bias as inputs rather than reading from member variables. * Added Conv3d EndToEnd test. * Added NCDHW DataLayout and unit tests. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I10cdd354ca5f1c748730f92ffdb36bf810f83c8e
2021-10-19IVGCVSW-6310 Update ICustomAllocator and add GetDefaultAllocator function to ↵Francis Murtagh
backends Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Change-Id: If2879e8a82692285e7fa3f4d09abd608ca28de12
2021-10-14IVGCVSW-6416 AddFullyConnected API crashes without connected weights/bias layersCathal Corbett
* Created method in Graph.cpp/hpp ConstructErrorMessageForUnconnectedInputs() to verify weights and bias are set for FullyConnected layers. * Above method called in Graph.cpp InferTensorInfos() to print a more descriptive message when the weights or bias for a FullyConnectedLayer is not set. * Added try-catch in TestUtils.cpp Connect() to ensure input slot is available when connecting two layers. This ensures we catch the case where bias is is not enabled and we try and set a bias layer. * Added unit tests to check for LayerValidationError when weights or bias is not set on a FullyConnectedLayer. * Added unit test to check for LayerValidationError when bias is not enabled and we try and connect bias to FullyConnected Layer. * Seperated FullyConnected EndToEnd unit test method into two methods. First, performs tests on EndToEnd examples asserting output value. Second, performs tests on error catching. * Added comments to created methods. Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I5207d8c5ebacfff598556742ccd4f53eef7dee0c
2021-10-08IVGCVSW-6313 Support pre-importing outputsFinn Williams
* Add ClearImportedInputs/Outputs function to IRuntime * Add UnImport function to ITensorHandle * Remove mutex from IWorkingMemHandle Change-Id: I34c9b6e1618755e10f3b4597afa1d9a9ea97e5fe Signed-off-by: Finn Williams <finwil01@e127804.cambridge.arm.com>
2021-10-08IVGCVSW-6417: Catch AddFullyConnected API error when weights TensorInfo ↵Cathal Corbett
isn't set * Updated code in Graph.cpp InferTensorInfos() to be more descriptive. * Added method VerifyConstantLayerSetTensorInfo() in Graph.cpp/hpp to error when ConstantLayer TensorInfo is not set. * Updated Optimize() in Network.cpp to call VerifyConstantLayerSetTensorInfo(). * Added unit test with ConstantLayer TensorInfo not set to catch error in VerifyConstantLayerSetTensorInfo(). * Added comments around method VerifyConstantLayerSetTensorInfo(). Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I366596243f7c5823676222e2d0cce1335bc8c325
2021-10-06IVGCVSW-6300 'IMemoryOptimizerStrategy Add strategy library and add support ↵Sadik Armagan
in BackendRegistry' * Updated IRuntime interface for providing custom memory optimizer strategy. * Enabled selecting existing memory optimizer strategy by using BackendOptions * Added MemoryOptimizerStrategyLibrary that sets one of the existing memory optimizer strategies selected by user Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: I037f8ac8efa79c0f71bd63e379101e3ad92d80c9
2021-10-02IVGCVSW-5985 Remove deprecated codeJan Eilers
* Removes deprecated AddLayer, IsLayerSupported functions * Marks the whole LayerVisitor class as deprecated not just the constructor. This required to wrap all Accept functions in a no deprecate macro because the LayerVisitor is used as a parameter in there * Removes usage of deprecated LayerVisitor and replaces it with ExecuteStrategy. This required a few structural changes in the unit tests * Adds a default implementation for IStrategy called StrategyBase * Changes pyarmnn to use non deprecated constructor for INetworkProperties and adds related unit test * Marks usage of deprecated code in pyarmnn as deprecated. This required to extend INetworkProperties to allow backwards compatibility * Removes deprecated functions from CpuAcc, GpuAcc and Ref backends Note: This patch breaks compatibility with backends that are not updated in this patch !android-nn-driver:6325 Signed-off-by: Jan Eilers <jan.eilers@arm.com> Change-Id: Id13b6f37a74d26eadeda2da1dc92915e725ed5a5
2021-10-01IVGCVSW-6163 Add Conv3d FrontEnd and Ref ImplementationMatthew Sloyan
* Added front-end * Added Reference workload * Added Serializer & Deserializer support * Added unit tests * Added NDHWC DataLayout Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Iec4d39e7433b5334d52fa44cf8efc6bcd39319d8
2021-09-30IVGCVSW-6293 Add Unit test int8 Channel ShuffleTeresa Charlin
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Icf69b4c707014c5ae76425ad326bd2826381f305
2021-09-29IVGCVSW-3716 Add EndToEnd Layer test for Channel Shuffle WorkloadTeresa Charlin
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I43d052d020f90c13688901929cd22c715471ef4f
2021-09-24IVGCVSW-3705 Add Channel Shuffle Front end and Ref ImplementationSimon Obute
* Add front end * Add reference workload * Add unit tests * Add Serializer and Deserializer * Update ArmNN Versioning Signed-off-by: Simon Obute <simon.obute@arm.com> Change-Id: I9ac1f953af3974382eac8e8d62d794d2344e8f47
2021-09-14IVGCVSW-6339 'IMemoryOptimizerStrategy Add backend capabilities'Sadik Armagan
* Added ExternallyManagedMemory capability, set false for backends * Added MultiAxisPacking capability, set false for backends Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: I96a2b66ed069a82552dae33f6e8ebe067bbd6c44
2021-09-03IVGCVSW-6262 Add support for Reduce ProdTeresa Charlin
* Tflite parser * Tflite delegate * Serializer * Deserializer * Ref, CpuAcc and GpuAcc workloads Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I601a9ee1680b372c7955d9a628857d08c3cfd377
2021-08-31MLCE-530 Add support of int8 weight for UnidirectionalSequenceLstmNarumol Prangnawarat
to Ref backend and armnn delegate Signed-off-by: Narumol Prangnawarat <narumol.prangnawarat@arm.com> Change-Id: I203d0029c12221228ffe229acda3c90594394e9b
2021-08-26IVGCVSW-6311 Add the pre-import backend capabilityFinn Williams
* Update acl and ref backend capabilities * Add check for AsyncExecution capability in LoadedNetwork * Add new exception for capabilities Signed-off-by: Finn Williams <Finn.Williams@arm.com> Change-Id: I2f053c845a64f336e96d452793639c6e0d3eaa89
2021-08-10MLCE-530 Add support for UnidirectionalSequenceLstm to RefWorkloadexperimental/daves_custom_allocator_dmabufNarumol Prangnawarat
* Add implementation of IsUnidirectionalSequenceLstmSupported to RefLayerSupport * Add RefUnidirectionalSequenceLstmWorkload * Refactor Lstm to be able to use for Lstm and SequenceLstm * Unit tests Signed-off-by: Narumol Prangnawarat <narumol.prangnawarat@arm.com> Change-Id: Ibc066d213213a11b955dfefbe518de643298ba0c
2021-08-10IVGCVSW-6292 Allow profiling details to be switched off during profilingKeith Davis
* Add switch for network details during profiling Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: I8bd49fd58f0e0255598106e9ab36806ee78391d6
2021-08-06IVGCVSW-6119 ConstTensorsAsInput: FullyConnectedMatthew Sloyan
* Constant weights and biases are now stored as Constant layers. * Updated Serializer, Deserializer and unit tests to reflect this. * Updated TfLiteDelegate, TfLiteParser and OnnxParser. * Updated Schema with IsConstant and ConstantTensorsAsInputs. * Updated Ref backend to handle constant weights and bias as inputs rather than reading from member variables. * Added dynamic or constant input EndToEnd tests. !android-nn-driver:5959 Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Ibf3cf437df1100e4b322b0d303c575c6339f9696
2021-08-05IVGCVSW-5980 Add Descriptor, TensorInfo and Convolution algorithm to JSONKeith Davis
* Add GUID as field to layer details and profiling events * Add Optional GUID param to existing tests * Improve Details macro to be inline function * Fix some formatting Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: I66f192a90a7642b3ee8e7dda0d3f428cce002581
2021-08-04IVGCVSW-5980 JSON profiling outputKeith Davis
* Add new ProfilingDetails class to construct operator details string * Add new macro which helps append layer details to ostream * Add ProfilingEnabled to NetworkProperties so that profiling can be realised when loading the network * Add further optional info to WorkloadInfo specific to convolutions * Generalise some JsonPrinter functions into JsonUtils for reusability * Remove explicit enabling of profiling within InferenceModel as it is done when loading network * Add ProfilingDetails macros to ConvolutionWorkloads for validation Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: Ie84bc7dc667e72e6bcb635544f9ead7af1765690
2021-07-29Fix use of non-standard datatypesRob Hughes
Change-Id: I03ef0571b24ad2bddfabf3bc5b11b614e7d7c8b4 Signed-off-by: Rob Hughes <robert.hughes@arm.com>
2021-07-16Fix binding of reference to null pointer in RefFullyConnectedWorkloadMatthew Bentham
As the bias decoder is optional, pass it as a pointer to the FullyConnected function instead of by reference. Signed-off-by: Matthew Bentham <matthew.bentham@arm.com> Change-Id: I0bd2e601015e7fa1ae21b50f6c3dd7eac4176554
2021-07-15Fix some compile warnings & errorsRob Hughes
* Fix type of literals * Add explicit casts * #include missing headers * Replace use of non-standard u_int8_t * Remove name of unused variables * Fix DLL export macro - this was broken when some of the exporting headers were moved to a different library. Change-Id: Ie569c8df41a077b46c608798f39526352e2aeb3a Signed-off-by: Rob Hughes <robert.hughes@arm.com>
2021-06-21Remove remaining boost testsMatthew Sloyan
* Updated unit tests to use doctest. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I5231225cd0830f16cc0ef20e9ef27e7b92feab19
2021-06-16MLCE-510 Add CpuRef Shape Operator to ArmNNKeith Davis
* Add front end * Add reference workload * Serialization/Deserialization * Add unit tests * Update ArmNN Versioning Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: I6fcb1fa341d6f08dea4003b13544e6e9f53fefd3
2021-06-16IVGCVSW-6088 Add Sin and Log to ElementWiseUnaryTeresa Charlin
* Ref workload * Cl workload * Neon workload * Serializer * Deserializer * Remove boost include from TensorTest.cpp Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I498548169cc77609c55cf3105f1de5a7429772cf
2021-06-16IVGCVSW-5826 Change weights layout for depthwise to [1,H,W,I*M]Jan Eilers
* This change is necessary because tflite uses a [1,H,W,I*M] format and uses the I*M dimension for per axis quantization. Our previous layout [M,I,H,W] can't handle the correlating quantization scales. * Updates Onnx-, TfLiteParser and TfliteDelegate * Updates the CpuRef, CpuAcc and GpuAcc backends * Adjusts unit tests * Adds test to ensure models with old layout can still be read and executed * Adds conversion function to previous layout [1,H,W,I*M] --> [M,I,H,W] which can be used by backend developers !android-nn-driver:5553 Signed-off-by: Jan Eilers <jan.eilers@arm.com> Change-Id: Ifef23368b8c3702cf315a5838d214f7dc13c0152
2021-06-11IVGCVSW-5963 'Move unit tests to new framework'Sadik Armagan
* Used doctest in ArmNN unit tests Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: Ia9cf5fc72775878885c5f864abf2c56b3a935f1a
2021-06-09IVGCVSW-5855 Refactor the reporting of capabilities from backendsFinn Williams
Signed-off-by: Finn Williams <Finn.Williams@arm.com> Change-Id: I05fc331a8e91bdcb6b8a2f32cfb555060fc5d797
2021-06-03Remove duplicated check for Dequantize input type not supportedTeresa Charlin
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ie05412091c25dc67f9d5d842aa1b63350fba9ee7
2021-06-02IVGCVSW-5962 Remove boost::multi_arraySadik Armagan
* Replaced all instances of boost::multi_array with flat vectors. * Updated LayerTestResult struct with new member variables. * Updated CompareTensor function to compare flat vectors and the shape. * Removed MakeTensor function from TensorHelpers.hpp. * Removed GetTensorShapeAsArray function from LayerTestResult.hpp. * Removed boost::array usage. * Removed boost::extents usages. * Removed boost::random usages. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: Iccde9d6640b534940292ff048fb80c00b38c4743
2021-06-01IVGCVSW-5833 Move the ProfilingGuid out of Types.hpp to its own header in ↵Nikhil Raj
profiling common !android-nn-driver:5691 Signed-off-by: Nikhil Raj <nikhil.raj@arm.com> Change-Id: Ib71af0831e324ac6bd27b1a36f4a6ec1a703b14a
2021-05-26IVGCVSW-6009 Integrate threadpool into ExNetKevin May
* Remove concurrent flag from ExecuteNetwork as it is possible to deduce if SimultaneousIterations > 1 * Add void RunAsync() * Refactor some unit tests Change-Id: I7021d4821b0e460470908294cbd9462850e8b361 Signed-off-by: Keith Davis <keith.davis@arm.com> Signed-off-by: Kevin May <kevin.may@arm.com>
2021-05-19IVGCVSW-5964 Removing some obsolete Boost references.Colm Donelan
Signed-off-by: Colm Donelan <Colm.Donelan@arm.com> Change-Id: Ic0a2e62e3808caf89ee429ec184d947a04340248
2021-05-06IVGCVSW-5813 Add Async Queue to IRuntimeKeith Davis
Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: Icc0d131c8ee2e9748e2f14762a75962b39c10f9d
2021-05-06IVGCVSW-5815 Generalise ConstCpuTensorHandleJames Conroy
* Generalises ConstCpuTensorHandle and inherited classes by removing 'Cpu' from aliases. * New renamed classes: ConstTensorHandle, TensorHandle, ScopedTensorHandle, PassthroughTensorHandle, ConstPassthroughTensorHandle. Signed-off-by: James Conroy <james.conroy@arm.com> Change-Id: I1824e0e134202735fb77051f20a7252f161dfe16
2021-05-05IVGCVSW-5418 ExecuteNetwork test for MobileBERTmathad01
* Fix bug in stack layer which causes mobileBERT model to seg fault when running in concurrent mode. Signed-off-by: mathad01 <matthew.haddon@arm.com> Change-Id: Ib2b73ffaa5b70147b837b7c764064e530baba637
2021-04-29IVGCVSW-5775 'Add Async Support to ExecuteNetwork'Sadik Armagan
* Enabled async mode with '-n, concurrent' and 'simultaneous-iterations' in ExecuteNetwork * Number of input files provided should be equal to number of input files provided multiply by number of simultaneous iterations divided by comma !armnn:5443 Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: Ibeb318010430bf4ae61a02b18b1bf88f3657774c
2021-04-28IVGCVSW-5843 Separate memory managers for WorkingMemHandlesFinn Williams
* Add inter layer memory management to WorkingMemHandle * Change Const layers to be executed once in loadedNetworkConstruction and share tensorHandle between all WorkingMemHandles * Fix various reference workloads pointing to memory in the queueDescriptor Signed-off-by: Finn Williams <Finn.Williams@arm.com> Change-Id: I69d4b3c5c84d2f5abe4540c3e624ab4f00d88226
2021-04-28IVGCVSW-5416 'Add android-nn-driver support for CASTSadik Armagan
Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: I02da912e5e4ca650b367ca40fe3f5ca5baa61cbb
2021-04-20IVGCVSW-5816 Constant memory accessFrancis Murtagh
* Add new class ManagedConstTensorHandle to Unmap when out of scope * Integrate into existing layers that have constants * Add unit tests Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Change-Id: I0a05e14e438804b37e9862e76b5ca329483f6b45
2021-04-14IVGCVSW-5787 Add/Update Execute() implementations in RefActivationWorkloadFinn Williams
* Added multithreaded StridedSliceEndToEndTest Signed-off-by: Finn Williams <Finn.Williams@arm.com> Change-Id: I4579db7b5959e0a22256f1bda00238c22e611dec
2021-04-12IVGCVSW-5410 Add front-end support for CASTmathad01
IVGCVSW-5415 Add TfLiteParser support for CAST * Added front end support for CAST, including support in the Reference workload, Serialization, Deserializtion, Unit tests, and TfLiteParser. Signed-off-by: mathad01 <matthew.haddon@arm.com> Change-Id: Iaf670ca5912a21ed6bc84f7f83a68b42154846bb
2021-04-09MLCE-328 Serializer/Deserializer does not support Signed64Mike Kelly
* Added support for Signed64 to flatbuffer's schema & updated source tree * Added support for Signed64 to TFLite Delegate * Added support for Signed64 to Serializer * Added support for Signed64 to Deserializer * Added unit test for ArgMinMax to Deserializer * Deprecated m_Output_Type from the ArgMinMaxDescriptor: the output type is solely determined by the DataType of the output Tensor * Fixed issue where RefArgMinMaxWorkload could output data using the wrong DataType * Added Signed64 to RefLayerSupport::IsArgMinMaxSupported as a supported type Signed-off-by: Mike Kelly <mike.kelly@arm.com> Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Ib622c052a1f8aa3e658262f8bde5a6881a8cbe10
2021-04-07Fix graph copy memory spikeFinn Williams
* Change layer storage of ConstTensors to std::shared_ptr<ConstCpuTensorHandle> * Change clone to share ConstTensor rather than copy * Remove uses of non-const GetTensor() call * Reduce scope of non-optimized network in ExeNet, so memory can be released after use Signed-off-by: Finn Williams <Finn.Williams@arm.com> Change-Id: Ibb2c7309d12411d21405bd6024c76bcdf5404545
2021-04-02IVGCVSW-5783 'Add AsyncExecution Capability'Sadik Armagan
* Added AsyncExecution to the BackendCapability enum class. * Logged a warning if backends do not support AsyncExecution capability if AsyncNetwork is created. Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: I49f8467297f4b6b8e414cb6a3638a7d3f1bb886a
2021-03-29IVGCVSW-5790 Merge async prototypeMike Kelly
* Added thread safe execution mechanism for armnn * Removed duplicate function bool Compare(T a, T b, float tolerance) * Added StridedSliceAsyncEndToEndTest * Fixed memory leak Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I2d367fc77ee7c01b8953138543e76af5e691211f
2021-03-25IVGCVSW-5736 and IVGCVSW-5743 'NonConstWeights: Update front-end and ↵Sadik Armagan
TfLiteDelegate support for FullyConnected Operator' * Added front-end support for non-const weights for FULLY_CONNECTED operator * Added FULLY_CONNECTED end-to-end test * Updated FULLY_CONNECTED operator support in TfLite Arm NN Delegate for non-const weights * Updated the version numbers Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: Iffa5b9aa9297aca4c02d923cce4636c88ac21faa
generated by cgit v1.2.1 (git 2.23.0) at 2024-07-18 04:07:26 +0000