aboutsummaryrefslogtreecommitdiff
path: root/src/armnn/Network.cpp
AgeCommit message (Collapse)Author
2022-07-28Revert "Revert "IVGCVSW-6650 Refactor ExecuteNetwork""Teresa Charlin
This reverts commit 1a7f033768acb27da11503bd29abb468d2e77f9e. List of fixes to be able to add this code again: * "emplacing_back" the vector inputTensors into the vector m_InputTensorsVec outside the for loop * GetIOInfo() uses IOptimizedNetwork instead of INetwork, where the infered shapes are not saved * Add missing data type Signed32 to SetupInputsAndOutputs() * PrintOutputTensors() prints the actual output without dequantizing * Add profilingDetailsMethod as input in networkProperties in ArmNNExecutor constructor * Fix typos Change-Id: I91de166f87228282db3efa27431fe91458834442 Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: Ic6634d48892d11e5f146cdf285e1e333e93e9937 Signed-off-by: Francis Murtagh <francis.murtagh@arm.com>
2022-07-22IVGCVSW-7109: Add Batch MatMul front end support - ReferenceSamuel Yap
* Descriptors added for BatchMatMul * Layer definition added * Input validation added (will likely change when opt. param support comes in) * Ref workload implementation for BatchMatMul added (will also change with opt. param support) * Ref layer tests made for BatchMatMul * CMake and other build files updated Signed-off-by: Samuel Yap <samuel.yap@arm.com> Change-Id: Ic885301da543ee0fbe7922b85e7f9658c4efc617
2022-07-22IVGCVSW-6978: RedirectMembersToConstantInputs does not work with ↵Francis Murtagh
Fp32NetworkToBf16Converter * Fuse FP32ToBF16Layers with Constant Layer so Conv2d/FullyConnected can have their weights redirected. * If BF16 Unsupported in Conv2d || FullyConnected revert fused Constant Layer to FP32 Change-Id: If523c708a822659d64597d9ae39cca1c2f84b76f Signed-off-by: Francis Murtagh <francis.murtagh@arm.com>
2022-06-27IVGCVSW-6981 Remove deprecated code 22.05 [Post Release]Nikhil Raj
Signed-off-by: Nikhil Raj <nikhil.raj@arm.com> Change-Id: I9ccaefbe28ea572e9e2b4a2168574804667f7460
2022-06-22Revert "Revert "IVGCVSW-6873 Import inputs but don't export outputs fails.""Francis Murtagh
This reverts commit a0f8b15d4ddb5075f380003ff31b271d389d3b66. Reason for revert: <Test ClDmaBufInternalTests review > Change-Id: Ibc4a77fa008643849da7330391942e4c87b941e2
2022-06-21Revert "IVGCVSW-6873 Import inputs but don't export outputs fails."James Conroy
This reverts commit 03bf98a8bc51ad20eef4b9ca5fbf6ce15e063721. Reason for revert: Caused failures in tests located in internal repo. Change-Id: If35cb0ede349b270e4e7827324382e09455d8cfa
2022-06-20IVGCVSW-6873 Import inputs but don't export outputs fails.Colm Donelan
Only one bool is used to indicate whether inputs should be imported. However, its possible for the user to want to import inputs but not export outputs. In addition it's possible for a user to enabled import during optimize but then pass a memory source that does not require import. * Add m_ExportEnabled to INetwork.hpp. * Modify Network::dNetwork to consider both m_ImportEnabled and m_ExportEnabled. * Add ValidateSourcesMatchOptimizedNetwork to LoadedNetwork to validate import options between optimize and network load. * Update the TfLite delegate consider exportEnabled flag in the optimizer. !armnn-internal-tests:425350 Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I776eab81595898e43f91ab40306962eae61329f4
2022-05-23IVGCVSW-6123 ConstTensorsAsInputs: Conv2dKeith Davis
* Use new INetwork::AddConvolution2dLayer instead of deprecated version * Remove duplicated test in SerlializerTests * Fix some cosmetics Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: I3407815bfdc1cdc01ca0a667b8e4d80d8621783f
2022-05-18IVGCVSW-6147 ConstTensorsAsInput: Optimizer - FusePermuteIntoConstLayerCathal Corbett
* No trailing permute layer after a constant layer * Unit test for optimization Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I0d098f5af41d2c55df7cef1ccfb848093320ddc1
2022-05-18IVGCVSW-6455 Support Const + Dequantize layer and optimize it.Teresa Charlin
* Support Float16 as input to Dequantize layer * Add Optimization to substitute Const+Dequantize layers with Const layer Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I58bb7e3871ca480c7b6fca93c4efb2de84e09e64 Signed-off-by: David <david.monahan@arm.com>
2022-05-18IVGCVSW-6929 Support for models with implicit expandedMike Kelly
dimensions * Added allow-expanded-dims to TFLite parser and ArmNN delegate * If true ArmNN will disregard dimensions with a size of 1 when validating tensor shapes. Tensor sizes must still match. * This allows us to support models where tensors have expanded dimensions (i.e. extra dimensions with a size of 1). * Fixed bug in Network where it assumed that only the first option could be ShapeInferenceMethod. * Fixed bug where m_ShapeInferenceMethod was lost when copying or moving Graphs. * Changed Delegate to pass "infer-output-shape", "allow-expanded-dims" and other BackendOptions through to the Network during construction. Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: Ibe7c5ae6597796fc9164cb07bd372bd7f8f8cacf
2022-05-17IVGCVSW-6126 ConstTensorsAsInput: Conv2d - BackendsCathal Corbett
!android-nn-driver:7477 Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: Ibf633ccccc385bd980934ff829407d21981323ef
2022-05-16IVGCVSW-6124 ConstTensorsAsInput: Conv2d - FrontEndKeith Davis
* Update Front-end and Tools. * Updated Serializer, Deserializer and unit tests to reflect this. * Updated TfLiteDelegate, TfLiteParser and OnnxParser. * Updated Ref. * Fixed resulting Neon / CL tests * Unified optimizers for conv2d ops * Optimizer Fix - Fp32ToBf16 * Partial implementation for ACL backends to fix VTS failures !android-nn-driver:7477 Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: I5fb18877f7ee32643e15a9818945356274bb401b
2022-05-12IVGCVSW-6940 ConstTensorsAsInput: DepthwiseConvolution2d - Complete ACLCathal Corbett
* Added backend specific optimization & test for CpuAcc and GpuAcc: PermuteDepthwiseConv2dWeights Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I600476b2e9c557a39818a574c1091c9d650b21b1
2022-05-06Wrong variable 'm_ProjectionWeights' is used to check if the pointer is ↵Cathal Corbett
valid from IVGCVSW-4449 * Github issue: https://github.com/ARM-software/armnn/issues/639 Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: If93f2cae101814652664c671417946b94a9f749c
2022-05-05IVGCVSW-6127 ConstTensorsAsInput: DepthwiseConvolution2dCathal Corbett
!android-nn-driver:7418 * Update Front-end and Tools. * Updated Serializer, Deserializer and unit tests to reflect this. * Updated TfLiteDelegate, TfLiteParser and OnnxParser. * Change NNDriver to new API. * Updated Ref. * Neon and Cl backend partially completed (Backend.cpp files). * Added dynamic or constant input EndToEnd tests. * Added ConstantTensorAsInputMemeberVariableRedirect Optimization. Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: Ib18b6c10a093042e165e25237dc04a4c67ba82da
2022-05-03IVGCVSW-6856 Add GATHERNd FrontEnd and Ref ImplementationTeresa Charlin
* Add front end * Add reference workload * Add unit tests * Add EndToEnd test Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com> Change-Id: I4cebd17b18476df86162e2dda3366c10e80bd2f8
2022-04-01IVGCVSW-6732 Tests surrounded in '#if defined(ARMNNREF_ENABLED)' in ↵Cathal Corbett
android-nn-driver do not execute. * Change to src/backends/cl/workloads/ClLstmFloatWorkload.cpp fix LstmTests_GpuAcc tests. * Change to src/backends/cl/workloads/ClConvertFp16ToFp32Workload.hpp & ClConvertFp32ToFp16Workload.hpp fix MeanTests_GpuAcc and Convolution2DTests_1.1 tests. * Added UnitTests to src/backends/cl/test/ClImportTensorHandleTests.cpp to test import on Convert Layers. !android-nn-driver:7264 Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I0c46dc4b9c54eca8771ab12ed0302b6224606957
2022-03-23IVGCVSW-6706 Move headers to profiling/client/includeJim Flynn
!android-nn-driver:7337 Change-Id: Ide401623829cc99fb9b51e9bbce3482ce706a8dd Signed-off-by: Jim Flynn <jim.flynn@arm.com>
2022-03-03IVGCVSW-6811 replace ProfilingService includes with IProfilingServiceJim Flynn
Change-Id: I00521756c8a19d10bfdc98c6ef4204c7f84901c6 Signed-off-by: Jim Flynn <jim.flynn@arm.com>
2022-03-01IVGCVSW-6704 Change the namespace from armnn::profiling to arm::pipeCathal Corbett
* Updated ABI version to 29 due to being the first ABI break in 22.05 !android-nn-driver:7226 Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I9c50007dcd5b5e792757e7bd1213606df5ffec36
2022-02-16IVGCVSW-6399 Remove deprecated code 22.02 (FullyConnected)Francis Murtagh
* Remove deprecated INetwork::AddFullyconnectedLayer() taking weights as argument as they are now taken as separate input layers. * Updated test that was using the deprecated function. * Remove calls in pyarmnn Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Change-Id: Ibc52ac7fa57afd9033eb226bbc24a09e88a7d361
2022-02-03IVGCVSW-6634 SubgraphView: Add method of returning a GetSubgraphWorkingCopyFrancis Murtagh
* Add pointer to SubgraphView allowing it to store a working copy implementation of its own representation of graph. * Make SubgraphView a friend of Graph to allow access to layers. * Add constructor to SubgraphView taking SubgraphViewWorkingCopyPtr * Rewrite Graph::SubstituteSubgraph for use on SubgraphView * Add GetWorkingCopy() method * Add tests for replacement of multiplication with DepthwiseConv2d * Check GetBackendHint() has value before passing to PrecompiledLayer * Add GetOwningIConnectableLayer to IInputSlot to allow traversing from IConnectableLayer->IOutputSlot->IInputSlot->IConnectableLayer Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Change-Id: Iaaef14448d8b73867eaee9d69f4f98d5d1bf171c
2022-01-14IVGCVSW-6633 SubgraphView uses IConnectableLayer rather than Layer in its ↵Francis Murtagh
m_Layers * Added IInputSlot, IOutputSlot and IConnectableLayer to SubgraphView * Deprecated old member functions * Removed deprecated calls in ArmNN * Added GetOwningIConnectableLayer function to IOutputSlot * Updates ArmNN Core Major version for IOutputSlot ABI break * Updated Minor version of TfliteParser, OnnxParser and Delegate Signed-off-by: Francis Murtagh <francis.murtagh@arm.com> Change-Id: I2a8611bfabf5ae09d3602fe6a4bef166e18117b9
2022-01-12Fix INetwork::AddPrecompiledLayer to take the CompiledBlobPtr by value ↵Cathal Corbett
rather than reference. Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I4057d2d6055220bb0d415a934ac09688b699b77c
2021-12-23IVGCVSW-6632 OptimizationViews: has INetwork rather than Graph for holding ↵Cathal Corbett
layers * Deprecate the GetGraph() function in OptimizationViews & remove/fix occurances where OptimizationViews.GetGraph() is called. * OptimizationViews has member INetworkPtr. * OptimizationViews has GetINetwork() method. * Unit test added to OptimizationViewsTests.cpp. Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: Ifc1e53f1c34d786502279631942f0472f401038e
2021-12-15IVGCVSW-6626 Promote backend headers in backendCommon to armnn/backendsColm Donelan
Move the following header files from backendsCommon to armnn/backends. * MemCopyWorkload.hpp * TensorHandle.hpp * Workload.hpp * WorkloadData.hpp * WorkloadFactory.hpp Replace them with forwarding headers and a pragma deprecation message. Resolve the deprecation messages in Arm NN code. Signed-off-by: Colm Donelan <colm.donelan@arm.com> Change-Id: I47f116b30f86e478c9057795bc518c391a8ae514
2021-12-14IVGCVSW-6630 Add new method AddPrecompiledLayer() to INetworkCathal Corbett
* Add new method AddPrecompiledLayer() to INetwork with Comments noting it is for backend users. * Added unit test to SubgraphViewTests.cpp. * Bug fix and code refactor in Graph.cpp specifically around Graph::SubstituteSubgraph(SubgraphView& subgraph, IConnectableLayer* substituteLayer) Change-Id: If2d816e5109e48ce920bf92d8823b39130c23a16 Signed-off-by: Cathal Corbett <cathal.corbett@arm.com>
2021-12-08Add addition timing logging outputJan Eilers
* Adds ExecuteNetwork when building the delegate only * Adds timings to delegate subgraph creation * Adds executions times Signed-off-by: Jan Eilers <jan.eilers@arm.com> Change-Id: Ieff2f67ea8dbb6c2a708f8810e84a20485b7a631
2021-12-08Add ModelOptions logging outputJan Eilers
Signed-off-by: Jan Eilers <jan.eilers@arm.com> Change-Id: I1947924504c1308442b816cff513b2080ed3cc0d
2021-11-17IVGCVSW-6509 Front End + Reference Workload implementationTamás Nyíri
Subtask of story: IVGCVSW-6164 Add a Pooling3d FrontEnd and Ref Implementation * Add front end * Add reference workload * Add corresponding unit tests Change-Id: Icce4146dd0a06a1da46a2def00a82d343e171750 Signed-off-by: Tamas Nyiri <tamas.nyiri@arm.com>
2021-11-08IVGCVSW-6527 Support the new memory API in loaded networkFinn Williams
* enable external memory management for neon and ref backends * change m_TensorMemoryVector to hold shared pointers * change input layer backend Id to match backend id of connected layer Signed-off-by: Finn Williams <finn.williams@arm.com> Change-Id: I2216a724028312eb101b290df3f224177826b1a0
2021-10-28IVGCVSW-5636 'Implement NNAPI caching functions'Sadik Armagan
* Get number of inputs and outputs from optimized network. * Get number of cached files if backend supports caching. Signed-off-by: Sadik Armagan <sadik.armagan@arm.com> Change-Id: Ie02ac123bb7df9b0593a2fe46b5bb564a5994780
2021-10-20Add ConstTensorsAsInput support for Conv3dMatthew Sloyan
* Constant weights and biases are now stored as Constant layers. * Updated Serializer, Deserializer and unit tests to reflect this. * Updated TfLiteParser. * Updated Ref backend to handle constant weights and bias as inputs rather than reading from member variables. * Added Conv3d EndToEnd test. * Added NCDHW DataLayout and unit tests. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I10cdd354ca5f1c748730f92ffdb36bf810f83c8e
2021-10-15Refactor: Profiler moved to GraphDerek Lamberti
* This is to enable later work to instrument the Optimizer. Signed-off-by: Derek Lamberti <derek.lamberti@arm.com> Change-Id: I2cf1fe022e0d100d6d8705adfbb8cab3ffc96a86
2021-10-15Profiling instrumentation throughout the OptimizerDerek Lamberti
Signed-off-by: Derek Lamberti <derek.lamberti@arm.com> Change-Id: Ifcdafc12ca09455af2389bf84e34b6b87bbb0b15
2021-10-08IVGCVSW-6417: Catch AddFullyConnected API error when weights TensorInfo ↵Cathal Corbett
isn't set * Updated code in Graph.cpp InferTensorInfos() to be more descriptive. * Added method VerifyConstantLayerSetTensorInfo() in Graph.cpp/hpp to error when ConstantLayer TensorInfo is not set. * Updated Optimize() in Network.cpp to call VerifyConstantLayerSetTensorInfo(). * Added unit test with ConstantLayer TensorInfo not set to catch error in VerifyConstantLayerSetTensorInfo(). * Added comments around method VerifyConstantLayerSetTensorInfo(). Signed-off-by: Cathal Corbett <cathal.corbett@arm.com> Change-Id: I366596243f7c5823676222e2d0cce1335bc8c325
2021-10-02IVGCVSW-5985 Remove deprecated codeJan Eilers
* Removes deprecated AddLayer, IsLayerSupported functions * Marks the whole LayerVisitor class as deprecated not just the constructor. This required to wrap all Accept functions in a no deprecate macro because the LayerVisitor is used as a parameter in there * Removes usage of deprecated LayerVisitor and replaces it with ExecuteStrategy. This required a few structural changes in the unit tests * Adds a default implementation for IStrategy called StrategyBase * Changes pyarmnn to use non deprecated constructor for INetworkProperties and adds related unit test * Marks usage of deprecated code in pyarmnn as deprecated. This required to extend INetworkProperties to allow backwards compatibility * Removes deprecated functions from CpuAcc, GpuAcc and Ref backends Note: This patch breaks compatibility with backends that are not updated in this patch !android-nn-driver:6325 Signed-off-by: Jan Eilers <jan.eilers@arm.com> Change-Id: Id13b6f37a74d26eadeda2da1dc92915e725ed5a5
2021-10-01IVGCVSW-6163 Add Conv3d FrontEnd and Ref ImplementationMatthew Sloyan
* Added front-end * Added Reference workload * Added Serializer & Deserializer support * Added unit tests * Added NDHWC DataLayout Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Iec4d39e7433b5334d52fa44cf8efc6bcd39319d8
2021-09-24IVGCVSW-3705 Add Channel Shuffle Front end and Ref ImplementationSimon Obute
* Add front end * Add reference workload * Add unit tests * Add Serializer and Deserializer * Update ArmNN Versioning Signed-off-by: Simon Obute <simon.obute@arm.com> Change-Id: I9ac1f953af3974382eac8e8d62d794d2344e8f47
2021-08-20IVGCVSW-6294 Detect and remove extraneous fp16 conversion layersJan Eilers
Signed-off-by: Jan Eilers <jan.eilers@arm.com> Change-Id: I69a6ceda0d3e8d39947df71a2ad0d365ac7767d8 Signed-off-by: Finn Williams <Finn.Williams@arm.com>
2021-08-10IVGCVSW-6289 Separate tensor shape inference and validation callsFinn Williams
* Pass m_shapeInferenceMethod to OptimizerOptions in ExecuteNetwork Signed-off-by: Finn Williams <Finn.Williams@arm.com> Change-Id: I90280fb7629092d3b66e8a3968ca9e35a0df854a
2021-08-09IVGCVSW-6119 ConstTensorsAsInput: FullyConnected Bug FixMatthew Sloyan
* Updated FullyConnected layer member variables when cloning as some backends still require them. * Added SetConstant call when using deprecated AddFullyConnectedLayer method to ensure backwards compatibility. * Added SetConstant to SimpleSample to ensure it runs on all backends. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Ie7b4e4b868f23f8fcf9c41ffd12e2ea9ea53afca
2021-08-06Infer tensor shapes before graph optimizationFinn Williams
Signed-off-by: Finn Williams <Finn.Williams@arm.com> Change-Id: I2a7a9b7b21848e2b3258788f06b01159f95a0d6c
2021-08-06IVGCVSW-6119 ConstTensorsAsInput: FullyConnectedMatthew Sloyan
* Constant weights and biases are now stored as Constant layers. * Updated Serializer, Deserializer and unit tests to reflect this. * Updated TfLiteDelegate, TfLiteParser and OnnxParser. * Updated Schema with IsConstant and ConstantTensorsAsInputs. * Updated Ref backend to handle constant weights and bias as inputs rather than reading from member variables. * Added dynamic or constant input EndToEnd tests. !android-nn-driver:5959 Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: Ibf3cf437df1100e4b322b0d303c575c6339f9696
2021-07-22MLCE-530 Add front end support for UnidirectionalSequenceLstm on ArmNNNarumol Prangnawarat
Signed-off-by: Narumol Prangnawarat <narumol.prangnawarat@arm.com> Change-Id: I57bcbdec3eb0155f41af0fe7d6abf9bac2ec86eb
2021-06-30IVGCVSW-6161 ConstTensorsAsInput: Optimizer - Redirect ConstTensor layer membersMatthew Sloyan
* Optimization that searches for layers with ConstantLayers as inputs. * The layer member variables are then redirected to these ConstantLayers. Signed-off-by: Matthew Sloyan <matthew.sloyan@arm.com> Change-Id: I24a2bf0e8575b808343e0bbe3897b344e94796ad
2021-06-30IVGCVSW-6114 Create multiple LoadedNetworks from one OptimizedNetworkMike Kelly
* Added IOptimizedNetwork constructor that takes another IOptimizedNetwork and a ModelOptions. * Changed PreCompiledLayer to use shared_ptr rather than unique_ptr to store the PreCompiledObject (no interface changes). * Added unit tests to ensure that PreCompiledLayer::Clone() clones the pointer to the PreCompiledObject correctly. Signed-off-by: Mike Kelly <mike.kelly@arm.com> Change-Id: I3ef56055e0d189ffce9e651882d34da16c70a240
2021-06-16MLCE-510 Add CpuRef Shape Operator to ArmNNKeith Davis
* Add front end * Add reference workload * Serialization/Deserialization * Add unit tests * Update ArmNN Versioning Signed-off-by: Keith Davis <keith.davis@arm.com> Change-Id: I6fcb1fa341d6f08dea4003b13544e6e9f53fefd3
2021-06-16IVGCVSW-5835 Move the src/profiling/LabelsAndEventClasses class to ↵Nikhil Raj
profiling/common Signed-off-by: Nikhil Raj <nikhil.raj@arm.com> Change-Id: I83de0fe9c83ec7e698817095ce5cfe3319c36ea6
generated by cgit v1.2.1 (git 2.23.0) at 2024-07-01 15:47:48 +0000