ArmNN
 23.08
Public Member Functions | Static Public Member Functions | List of all members
NeonBackend Class Reference

#include <NeonBackend.hpp>

Inheritance diagram for NeonBackend:
[legend]
Collaboration diagram for NeonBackend:
[legend]

Public Member Functions

 NeonBackend ()=default
 
 ~NeonBackend ()=default
 
const BackendId & GetId () const override
 
IBackendInternal::IMemoryManagerUniquePtr CreateMemoryManager () const override
 
IWorkloadFactoryPtr CreateWorkloadFactory (const IBackendInternal::IMemoryManagerSharedPtr &memoryManager=nullptr) const override
 
IWorkloadFactoryPtr CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry) const override
 
IWorkloadFactoryPtr CreateWorkloadFactory (const IMemoryManagerSharedPtr &memoryManager, const ModelOptions &modelOptions) const override
 
IWorkloadFactoryPtr CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions) const override
 
IBackendInternal::IBackendContextPtr CreateBackendContext (const IRuntime::CreationOptions &) const override
 Create the runtime context of the backend. More...
 
IBackendInternal::IBackendProfilingContextPtr CreateBackendProfilingContext (const IRuntime::CreationOptions &, IBackendProfilingPtr &backendProfiling) override
 Create context specifically used for profiling interaction from backends. More...
 
IBackendInternal::ILayerSupportSharedPtr GetLayerSupport () const override
 
IBackendInternal::ILayerSupportSharedPtr GetLayerSupport (const ModelOptions &modelOptions) const override
 
OptimizationViews OptimizeSubgraphView (const SubgraphView &subgraph, const ModelOptions &modelOptions) const override
 
std::vector< ITensorHandleFactory::FactoryId > GetHandleFactoryPreferences () const override
 (Optional) Returns a vector of supported TensorHandleFactory ids in preference order. More...
 
void RegisterTensorHandleFactories (class TensorHandleFactoryRegistry &registry) override
 (Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
 
IBackendInternal::IBackendSpecificModelContextPtr CreateBackendSpecificModelContext (const ModelOptions &modelOptions) const override
 
BackendCapabilities GetCapabilities () const override
 Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted. More...
 
std::unique_ptr< ICustomAllocator > GetDefaultAllocator () const override
 Returns the default memory allocator for the backend. More...
 
- Public Member Functions inherited from IBackendInternal
 ~IBackendInternal () override=default
 Allow backends created by the factory function to be destroyed through IBackendInternal. More...
 
virtual IWorkloadFactoryPtr CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) const
 
virtual OptimizationViews OptimizeSubgraphView (const SubgraphView &subgraph) const
 
bool SupportsTensorAllocatorAPI () const
 
ITensorHandleFactory::FactoryId GetBackwardCompatibleFavoriteHandleFactory ()
 
virtual void RegisterTensorHandleFactories (class TensorHandleFactoryRegistry &registry, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags)
 (Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
 
virtual bool UseCustomMemoryAllocator (std::shared_ptr< ICustomAllocator > allocator, armnn::Optional< std::string & > errMsg)
 Signals the backend to use a custom memory allocator provided by the user. More...
 
virtual unsigned int GetNumberOfCacheFiles () const
 Returns the number of files cached if backend supports caching. More...
 
virtual ExecutionData CreateExecutionData (WorkingMemDescriptor &workingMemDescriptor) const
 Returns ExecutionData for the backend. More...
 
virtual void UpdateExecutionData (ExecutionData &executionData, WorkingMemDescriptor &workingMemDescriptor) const
 Update the ExecutionData for a layer. More...
 

Static Public Member Functions

static const BackendId & GetIdStatic ()
 
- Static Public Member Functions inherited from IBackendInternal
static constexpr BackendVersion GetApiVersion ()
 Returns the version of the Backend API. More...
 

Additional Inherited Members

- Public Types inherited from IBackendInternal
using IWorkloadFactoryPtr = std::unique_ptr< IWorkloadFactory >
 
using IBackendContextPtr = std::unique_ptr< IBackendContext >
 
using IBackendProfilingContextPtr = std::shared_ptr< arm::pipe::IBackendProfilingContext >
 This is the bridge between backend and backend profiling we'll keep it in the backend namespace. More...
 
using IBackendProfilingPtr = std::unique_ptr< arm::pipe::IBackendProfiling >
 
using ILayerSupportSharedPtr = std::shared_ptr< ILayerSupport >
 
using IBackendSpecificModelContextPtr = std::shared_ptr< IBackendModelContext >
 
using IMemoryManagerUniquePtr = std::unique_ptr< IMemoryManager >
 
using IMemoryManagerSharedPtr = std::shared_ptr< IMemoryManager >
 
- Protected Member Functions inherited from IBackendInternal
 IBackendInternal ()=default
 Creation must be done through a specific backend interface. More...
 
- Protected Member Functions inherited from IBackend
 IBackend ()
 
virtual ~IBackend ()
 

Detailed Description

Definition at line 29 of file NeonBackend.hpp.

Constructor & Destructor Documentation

◆ NeonBackend()

NeonBackend ( )
default

◆ ~NeonBackend()

~NeonBackend ( )
default

Member Function Documentation

◆ CreateBackendContext()

IBackendInternal::IBackendContextPtr CreateBackendContext ( const IRuntime::CreationOptions &  ) const
overridevirtual

Create the runtime context of the backend.

Implementations may return a default-constructed IBackendContextPtr if no context is needed at runtime. Implementations must throw BackendUnavailableException if the backend cannot be used (for example, necessary accelerator hardware is not present). The default implementation always returns a default-constructed pointer.

Reimplemented from IBackendInternal.

Definition at line 107 of file NeonBackend.cpp.

108 {
109  return IBackendContextPtr{};
110 }

◆ CreateBackendProfilingContext()

IBackendInternal::IBackendProfilingContextPtr CreateBackendProfilingContext ( const IRuntime::CreationOptions &  creationOptions,
IBackendProfilingPtr &  backendProfiling 
)
overridevirtual

Create context specifically used for profiling interaction from backends.

Reimplemented from IBackendInternal.

Definition at line 112 of file NeonBackend.cpp.

114 {
115  return IBackendProfilingContextPtr{};
116 }

◆ CreateBackendSpecificModelContext()

IBackendInternal::IBackendSpecificModelContextPtr CreateBackendSpecificModelContext ( const ModelOptions &  modelOptions) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 118 of file NeonBackend.cpp.

120 {
121  return IBackendSpecificModelContextPtr{new NeonBackendModelContext{modelOptions}};
122 }

Referenced by NeonBackend::CreateWorkloadFactory(), and NeonBackend::GetLayerSupport().

◆ CreateMemoryManager()

IBackendInternal::IMemoryManagerUniquePtr CreateMemoryManager ( ) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 50 of file NeonBackend.cpp.

51 {
52  return std::make_unique<NeonMemoryManager>(std::make_unique<arm_compute::Allocator>(),
53  BaseMemoryManager::MemoryAffinity::Offset);
54 }

References BaseMemoryManager::Offset.

◆ CreateWorkloadFactory() [1/4]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry &  tensorHandleFactoryRegistry) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 70 of file NeonBackend.cpp.

72 {
73  auto memoryManager = std::make_shared<NeonMemoryManager>(std::make_unique<arm_compute::Allocator>(),
74  BaseMemoryManager::MemoryAffinity::Offset);
75 
76  tensorHandleFactoryRegistry.RegisterMemoryManager(memoryManager);
77 
78  auto factory = std::make_unique<NeonTensorHandleFactory>(memoryManager);
79  // Register copy and import factory pair
80  tensorHandleFactoryRegistry.RegisterCopyAndImportFactoryPair(factory->GetId(), factory->GetId());
81  // Register the factory
82  tensorHandleFactoryRegistry.RegisterFactory(std::move(factory));
83 
84 
85  return std::make_unique<NeonWorkloadFactory>(
86  PolymorphicPointerDowncast<NeonMemoryManager>(memoryManager));
87 }

References BaseMemoryManager::Offset, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

◆ CreateWorkloadFactory() [2/4]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry &  tensorHandleFactoryRegistry,
const ModelOptions &  modelOptions 
) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 89 of file NeonBackend.cpp.

91 {
92  auto memoryManager = std::make_shared<NeonMemoryManager>(std::make_unique<arm_compute::Allocator>(),
93  BaseMemoryManager::MemoryAffinity::Offset);
94 
95  tensorHandleFactoryRegistry.RegisterMemoryManager(memoryManager);
96 
97  auto factory = std::make_unique<NeonTensorHandleFactory>(memoryManager);
98  // Register copy and import factory pair
99  tensorHandleFactoryRegistry.RegisterCopyAndImportFactoryPair(factory->GetId(), factory->GetId());
100  // Register the factory
101  tensorHandleFactoryRegistry.RegisterFactory(std::move(factory));
102 
103  return std::make_unique<NeonWorkloadFactory>(
104  PolymorphicPointerDowncast<NeonMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
105 }

References NeonBackend::CreateBackendSpecificModelContext(), BaseMemoryManager::Offset, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

◆ CreateWorkloadFactory() [3/4]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IBackendInternal::IMemoryManagerSharedPtr &  memoryManager = nullptr) const
overridevirtual

Implements IBackendInternal.

Definition at line 56 of file NeonBackend.cpp.

58 {
59  return std::make_unique<NeonWorkloadFactory>(
60  PolymorphicPointerDowncast<NeonMemoryManager>(memoryManager));
61 }

◆ CreateWorkloadFactory() [4/4]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IMemoryManagerSharedPtr &  memoryManager,
const ModelOptions &  modelOptions 
) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 63 of file NeonBackend.cpp.

65 {
66  return std::make_unique<NeonWorkloadFactory>(
67  PolymorphicPointerDowncast<NeonMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
68 }

References NeonBackend::CreateBackendSpecificModelContext().

◆ GetCapabilities()

BackendCapabilities GetCapabilities ( ) const
inlineoverridevirtual

Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted.

Reimplemented from IBackendInternal.

Definition at line 68 of file NeonBackend.hpp.

69  {
70  return cpuAccCapabilities;
71  };

References armnn::cpuAccCapabilities.

◆ GetDefaultAllocator()

std::unique_ptr< ICustomAllocator > GetDefaultAllocator ( ) const
overridevirtual

Returns the default memory allocator for the backend.

Returns
- Returns unique pointer to the Default Allocator of the Backend

Reimplemented from IBackendInternal.

Definition at line 559 of file NeonBackend.cpp.

560 {
561  return std::make_unique<DefaultAllocator>();
562 }

◆ GetHandleFactoryPreferences()

std::vector< ITensorHandleFactory::FactoryId > GetHandleFactoryPreferences ( ) const
overridevirtual

(Optional) Returns a vector of supported TensorHandleFactory ids in preference order.

Reimplemented from IBackendInternal.

Definition at line 540 of file NeonBackend.cpp.

541 {
542  return std::vector<ITensorHandleFactory::FactoryId>() = { NeonTensorHandleFactory::GetIdStatic() };
543 }

References NeonTensorHandleFactory::GetIdStatic().

◆ GetId()

const BackendId& GetId ( ) const
inlineoverridevirtual

Implements IBackend.

Definition at line 36 of file NeonBackend.hpp.

36 { return GetIdStatic(); }

References NeonBackend::GetIdStatic().

◆ GetIdStatic()

const BackendId & GetIdStatic ( )
static

Definition at line 44 of file NeonBackend.cpp.

45 {
46  static const BackendId s_Id{NeonBackendId()};
47  return s_Id;
48 }

References armnn::NeonBackendId().

Referenced by NeonBackend::GetId().

◆ GetLayerSupport() [1/2]

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( ) const
overridevirtual

Implements IBackendInternal.

Definition at line 124 of file NeonBackend.cpp.

125 {
126  static ILayerSupportSharedPtr layerSupport
127  {
128  new NeonLayerSupport(IBackendInternal::IBackendSpecificModelContextPtr{})
129  };
130  return layerSupport;
131 }

◆ GetLayerSupport() [2/2]

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( const ModelOptions &  modelOptions) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 133 of file NeonBackend.cpp.

134 {
135  static ILayerSupportSharedPtr layerSupport
136  {
137  new NeonLayerSupport(CreateBackendSpecificModelContext(modelOptions))
138  };
139  return layerSupport;
140 }

References NeonBackend::CreateBackendSpecificModelContext().

◆ OptimizeSubgraphView()

OptimizationViews OptimizeSubgraphView ( const SubgraphView &  subgraph,
const ModelOptions &  modelOptions 
) const
overridevirtual

Reimplemented from IBackendInternal.

Definition at line 142 of file NeonBackend.cpp.

144 {
145  OptimizationViews optimizationViews(modelOptions);
146 
147  auto it = subgraph.end();
148  std::map<LayerGuid, Layer*> untouched;
149 
150  while (it != subgraph.begin())
151  {
152  --it;
153  Layer& base = *(PolymorphicDowncast<Layer*>(*it));
154  untouched.insert({base.GetGuid(), &base});
155  }
156 
157  it = subgraph.end();
158  while (it != subgraph.begin())
159  {
160  --it;
161  Layer& base = *(PolymorphicDowncast<Layer*>(*it));
162 
163  // Fuse activation into previous layer if supported by backend
164  if ((base.GetType() == LayerType::DepthwiseConvolution2d || base.GetType() == LayerType::Convolution2d
165  || base.GetType() == LayerType::BatchNormalization || base.GetType() == LayerType::FullyConnected
166  || base.GetType() == LayerType::Addition || base.GetType() == LayerType::Multiplication
167  || base.GetType() == LayerType::Subtraction || base.GetType() == LayerType::Division)
168  && (base.GetAdditionalInformation<ActivationDescriptor>() == nullptr))
169  {
170  for (auto output = base.BeginOutputSlots(); output != base.EndOutputSlots(); ++output)
171  {
172  if (output->GetNumConnections() == 1)
173  {
174  for (auto&& childInput : output->GetConnections())
175  {
176  if ((childInput->GetOwningLayer().GetType() == LayerType::Activation) &&
177  (checkDataTypeInputandOutput(childInput->GetOwningLayer())))
178  {
179  Layer& child = childInput->GetOwningLayer();
180 
181  auto* activationLayer = PolymorphicDowncast<ActivationLayer*>(&child);
182 
183  const std::string name = std::string("fused-") + child.GetName() + std::string("-into-") +
184  base.GetName();
185 
186  // Get params from activation layer
187  ActivationDescriptor activationDesc = activationLayer->GetParameters();
188 
189  if (base.GetType() == LayerType::Convolution2d)
190  {
191  Convolution2dLayer* baseLayer = PolymorphicDowncast<Convolution2dLayer*>(&base);
192 
193  Optional<TensorInfo> biases;
194 
195  if (baseLayer->GetParameters().m_BiasEnabled)
196  {
197  biases = baseLayer->GetInputSlot(2).GetConnectedOutputSlot()->GetTensorInfo();
198  }
199 
200  arm_compute::Status status = NeonConvolution2dWorkloadValidate(
201  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
202  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
203  baseLayer->GetParameters(),
204  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
205  biases,
206  false,
207  &activationDesc);
208 
209  if (status)
210  {
211  FuseConvolution2dLayer<Convolution2dLayer>(optimizationViews,
212  baseLayer,
213  activationLayer,
214  activationDesc,
215  name);
216  untouched.erase(baseLayer->GetGuid());
217  untouched.erase(activationLayer->GetGuid());
218  }
219  }
220  else if (base.GetType() == LayerType::DepthwiseConvolution2d)
221  {
222  DepthwiseConvolution2dLayer* baseLayer =
223  PolymorphicDowncast<DepthwiseConvolution2dLayer*>(&base);
224 
225  Optional<TensorInfo> biases;
226 
227  if (baseLayer->GetParameters().m_BiasEnabled)
228  {
229  biases = baseLayer->GetInputSlot(2).GetConnectedOutputSlot()->GetTensorInfo();
230  }
231 
232  arm_compute::Status status = NeonDepthwiseConvolutionWorkloadValidate(
233  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
234  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
235  baseLayer->GetParameters(),
236  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
237  biases,
238  &activationDesc);
239 
240  if (status)
241  {
242  FuseDepthwiseConvolution2dLayer<DepthwiseConvolution2dLayer>(optimizationViews,
243  baseLayer,
244  activationLayer,
245  activationDesc,
246  name);
247  untouched.erase(baseLayer->GetGuid());
248  untouched.erase(activationLayer->GetGuid());
249  }
250  }
251  else if (base.GetType() == LayerType::FullyConnected)
252  {
253  FullyConnectedLayer* baseLayer = PolymorphicDowncast<FullyConnectedLayer*>(&base);
254  FullyConnectedDescriptor descriptor = baseLayer->GetParameters();
255 
256  // As bias is optional only try to get TensorInfo from input if bias is enabled.
257  Optional<TensorInfo> biases;
258  if (descriptor.m_BiasEnabled)
259  {
260  biases = baseLayer->GetInputSlot(2).GetConnectedOutputSlot()->GetTensorInfo();
261  }
262 
263  arm_compute::Status status = NeonFullyConnectedWorkloadValidate(
264  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
265  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
266  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
267  biases,
268  baseLayer->GetParameters(),
269  &activationDesc);
270 
271  if (status)
272  {
273  FuseFullyConnectedLayer<FullyConnectedLayer>(optimizationViews,
274  baseLayer,
275  activationLayer,
276  activationDesc,
277  name);
278  untouched.erase(baseLayer->GetGuid());
279  untouched.erase(activationLayer->GetGuid());
280  }
281  }
282  else if (base.GetType() == LayerType::BatchNormalization)
283  {
284  BatchNormalizationLayer* baseLayer =
285  PolymorphicDowncast<BatchNormalizationLayer*>(&base);
286 
287  arm_compute::Status status = NeonBatchNormalizationValidate(
288  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
289  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
290  baseLayer->m_Mean->GetTensorInfo(),
291  baseLayer->m_Variance->GetTensorInfo(),
292  baseLayer->m_Beta->GetTensorInfo(),
293  baseLayer->m_Gamma->GetTensorInfo(),
294  baseLayer->GetParameters(),
295  &activationDesc);
296 
297  if (status)
298  {
299  BatchNormalizationLayer* replacementLayer =
300  FuseBatchNormalizationLayer<BatchNormalizationLayer>(optimizationViews,
301  baseLayer,
302  activationLayer,
303  activationDesc,
304  name);
305 
306  replacementLayer->m_Beta = std::move(baseLayer->m_Beta);
307  replacementLayer->m_Gamma = std::move(baseLayer->m_Gamma);
308  replacementLayer->m_Mean = std::move(baseLayer->m_Mean);
309  replacementLayer->m_Variance = std::move(baseLayer->m_Variance);
310  untouched.erase(baseLayer->GetGuid());
311  untouched.erase(activationLayer->GetGuid());
312  }
313  }
314  else if (base.GetType() == LayerType::Addition)
315  {
316  AdditionLayer* baseLayer = PolymorphicDowncast<AdditionLayer*>(&base);
317 
318  arm_compute::Status status = NeonAdditionWorkloadValidate(
319  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
320  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
321  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
322  &activationDesc);
323 
324  if (status)
325  {
326  FuseAdditionLayer<AdditionLayer>(optimizationViews,
327  baseLayer,
328  activationLayer,
329  activationDesc,
330  name);
331  untouched.erase(baseLayer->GetGuid());
332  untouched.erase(activationLayer->GetGuid());
333  }
334  }
335  else if (base.GetType() == LayerType::Division)
336  {
337  DivisionLayer* baseLayer = PolymorphicDowncast<DivisionLayer*>(&base);
338 
339  arm_compute::Status status = NeonDivisionWorkloadValidate(
340  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
341  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
342  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
343  &activationDesc);
344 
345  if (status)
346  {
347  FuseDivisionLayer<DivisionLayer>(optimizationViews,
348  baseLayer,
349  activationLayer,
350  activationDesc,
351  name);
352  untouched.erase(baseLayer->GetGuid());
353  untouched.erase(activationLayer->GetGuid());
354  }
355  }
356  else if (base.GetType() == LayerType::Multiplication)
357  {
358  MultiplicationLayer* baseLayer = PolymorphicDowncast<MultiplicationLayer*>(&base);
359 
360  arm_compute::Status status = NeonMultiplicationWorkloadValidate(
361  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
362  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
363  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
364  &activationDesc);
365 
366  if (status)
367  {
368  FuseMultiplicationLayer<MultiplicationLayer>(optimizationViews,
369  baseLayer,
370  activationLayer,
371  activationDesc,
372  name);
373  untouched.erase(baseLayer->GetGuid());
374  untouched.erase(activationLayer->GetGuid());
375  }
376  }
377  else if (base.GetType() == LayerType::Subtraction)
378  {
379  SubtractionLayer* baseLayer = PolymorphicDowncast<SubtractionLayer*>(&base);
380 
381  arm_compute::Status status = NeonSubtractionWorkloadValidate(
382  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
383  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
384  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
385  &activationDesc);
386 
387  if (status)
388  {
389  FuseSubtractionLayer<SubtractionLayer>(optimizationViews,
390  baseLayer,
391  activationLayer,
392  activationDesc,
393  name);
394  untouched.erase(baseLayer->GetGuid());
395  untouched.erase(activationLayer->GetGuid());
396  }
397  }
398  else if (base.GetType() == LayerType::ElementwiseBinary)
399  {
400  ElementwiseBinaryLayer* baseLayer = PolymorphicDowncast<ElementwiseBinaryLayer*>(&base);
401 
402  if (baseLayer->GetParameters().m_Operation == BinaryOperation::Add)
403  {
404  arm_compute::Status status = NeonAdditionWorkloadValidate(
405  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
406  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
407  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
408  &activationDesc);
409 
410  if (status)
411  {
412  FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
413  baseLayer,
414  activationLayer,
415  activationDesc,
416  BinaryOperation::Add,
417  name);
418  untouched.erase(baseLayer->GetGuid());
419  untouched.erase(activationLayer->GetGuid());
420  }
421  }
422  else if (baseLayer->GetParameters().m_Operation == BinaryOperation::Div)
423  {
424  arm_compute::Status status = NeonDivisionWorkloadValidate(
425  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
426  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
427  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
428  &activationDesc);
429 
430  if (status)
431  {
432  FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
433  baseLayer,
434  activationLayer,
435  activationDesc,
436  BinaryOperation::Div,
437  name);
438  untouched.erase(baseLayer->GetGuid());
439  untouched.erase(activationLayer->GetGuid());
440  }
441  }
442  else if (baseLayer->GetParameters().m_Operation == BinaryOperation::Mul)
443  {
444  arm_compute::Status status = NeonMultiplicationWorkloadValidate(
445  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
446  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
447  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
448  &activationDesc);
449 
450  if (status)
451  {
452  FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
453  baseLayer,
454  activationLayer,
455  activationDesc,
456  BinaryOperation::Mul,
457  name);
458  untouched.erase(baseLayer->GetGuid());
459  untouched.erase(activationLayer->GetGuid());
460  }
461  }
462  else if (baseLayer->GetParameters().m_Operation == BinaryOperation::Sub)
463  {
464  arm_compute::Status status = NeonSubtractionWorkloadValidate(
465  baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
466  baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
467  activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
468  &activationDesc);
469 
470  if (status)
471  {
472  FuseElementwiseBinaryLayer<ElementwiseBinaryLayer>(optimizationViews,
473  baseLayer,
474  activationLayer,
475  activationDesc,
476  BinaryOperation::Sub,
477  name);
478  untouched.erase(baseLayer->GetGuid());
479  untouched.erase(activationLayer->GetGuid());
480  }
481  }
482  // No fusion available for other BinaryOperations
483  }
484  }
485  }
486  }
487  }
488  }
489 
490  // Separate reduce layer with multiple axes into multiple reduce layers with 1 axis.
491  if (base.GetType() == LayerType::Reduce)
492  {
493  ReduceLayer* baseLayer = PolymorphicDowncast<ReduceLayer*>(&base);
494  ReduceDescriptor reduceDescriptor = baseLayer->GetParameters();
495 
496  if (!reduceDescriptor.m_vAxis.empty() && reduceDescriptor.m_vAxis.size() > 1)
497  {
498  // Add new layers to the graph and connect them.
499  std::vector<IConnectableLayer*> layers = ChainReduceLayers<ReduceLayer>(optimizationViews,
500  baseLayer,
501  reduceDescriptor);
502 
503  // Replace existing baselayer with new subgraph.
504  ReplaceLayers<ReduceLayer>(optimizationViews, baseLayer, layers);
505  untouched.erase(baseLayer->GetGuid());
506  }
507  }
508 
509  // Remove Reshape where possible
510  if (base.GetType() == LayerType::Reshape)
511  {
512  ReshapeLayer* baseLayer = PolymorphicDowncast<ReshapeLayer*>(&base);
513 
514  // Cannot remove a Reshape if it's connected to any layer that has an NCHW layout
515  if (ConnectedToLayerWithNCHW(baseLayer))
516  {
517  continue;
518  }
519  // Cannot remove a Reshape if it's connected to a SplitterLayer
520  if (ConnectedToLayerType(baseLayer, LayerType::Splitter))
521  {
522  continue;
523  }
524  RemoveReshapeLayer(baseLayer, untouched, optimizationViews);
525  }
526  }
527 
528  if (optimizationViews.GetSubstitutions().empty() && optimizationViews.GetDeletedSubgraphs().empty())
529  {
530  optimizationViews.AddUntouchedSubgraph(SubgraphView(subgraph));
531  }
532  else
533  {
534  ReportUntouchedLayers(optimizationViews, untouched);
535  }
536 
537  return optimizationViews;
538 }

References armnn::Activation, armnn::Add, armnn::Addition, OptimizationViews::AddUntouchedSubgraph(), armnn::BatchNormalization, SubgraphView::begin(), Layer::BeginOutputSlots(), armnn::ConnectedToLayerType(), armnn::ConnectedToLayerWithNCHW(), armnn::Convolution2d, armnn::DepthwiseConvolution2d, armnn::Div, armnn::Division, armnn::ElementwiseBinary, SubgraphView::end(), Layer::EndOutputSlots(), armnn::FullyConnected, Layer::GetAdditionalInformation(), InputSlot::GetConnectedOutputSlot(), OptimizationViews::GetDeletedSubgraphs(), Layer::GetGuid(), Layer::GetInputSlot(), Layer::GetName(), LayerWithParameters< Parameters >::GetParameters(), OptimizationViews::GetSubstitutions(), OutputSlot::GetTensorInfo(), Layer::GetType(), BatchNormalizationLayer::m_Beta, FullyConnectedDescriptor::m_BiasEnabled, Convolution2dDescriptor::m_BiasEnabled, DepthwiseConvolution2dDescriptor::m_BiasEnabled, BatchNormalizationLayer::m_Gamma, BatchNormalizationLayer::m_Mean, ElementwiseBinaryDescriptor::m_Operation, BatchNormalizationLayer::m_Variance, ReduceDescriptor::m_vAxis, armnn::Mul, armnn::Multiplication, armnn::NeonAdditionWorkloadValidate(), armnn::NeonBatchNormalizationValidate(), armnn::NeonConvolution2dWorkloadValidate(), armnn::NeonDepthwiseConvolutionWorkloadValidate(), armnn::NeonDivisionWorkloadValidate(), armnn::NeonFullyConnectedWorkloadValidate(), armnn::NeonMultiplicationWorkloadValidate(), armnn::NeonSubtractionWorkloadValidate(), armnn::Reduce, armnn::RemoveReshapeLayer(), armnn::ReportUntouchedLayers(), armnn::Reshape, armnn::Splitter, armnn::Sub, and armnn::Subtraction.

◆ RegisterTensorHandleFactories()

void RegisterTensorHandleFactories ( class TensorHandleFactoryRegistry &  )
overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor() IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 545 of file NeonBackend.cpp.

546 {
547  auto memoryManager = std::make_shared<NeonMemoryManager>(std::make_unique<arm_compute::Allocator>(),
548  BaseMemoryManager::MemoryAffinity::Offset);
549 
550  registry.RegisterMemoryManager(memoryManager);
551 
552  auto factory = std::make_unique<NeonTensorHandleFactory>(memoryManager);
553  // Register copy and import factory pair
554  registry.RegisterCopyAndImportFactoryPair(factory->GetId(), factory->GetId());
555  // Register the factory
556  registry.RegisterFactory(std::move(factory));
557 }

References BaseMemoryManager::Offset, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

The documentation for this class was generated from the following files:
armnn::NeonFullyConnectedWorkloadValidate
arm_compute::Status NeonFullyConnectedWorkloadValidate(const TensorInfo &input, const TensorInfo &output, const TensorInfo &weights, const Optional< TensorInfo > &biases, const FullyConnectedDescriptor &descriptor, const ActivationDescriptor *activationDescriptor)
Definition: NeonFullyConnectedWorkload.cpp:24
armnn::BaseMemoryManager::MemoryAffinity::Offset
@ Offset
armnn::BinaryOperation::Mul
@ Mul
armnn::BinaryOperation::Add
@ Add
armnn::NeonAdditionWorkloadValidate
arm_compute::Status NeonAdditionWorkloadValidate(const TensorInfo &input0, const TensorInfo &input1, const TensorInfo &output, const ActivationDescriptor *activationDescriptor)
Definition: NeonAdditionWorkload.cpp:20
armnn::LayerType::Splitter
@ Splitter
armnn::LayerType::BatchNormalization
@ BatchNormalization
armnn::NeonMultiplicationWorkloadValidate
arm_compute::Status NeonMultiplicationWorkloadValidate(const TensorInfo &input0, const TensorInfo &input1, const TensorInfo &output, const ActivationDescriptor *activationDescriptor)
Definition: NeonMultiplicationWorkload.cpp:19
armnn::NeonSubtractionWorkloadValidate
arm_compute::Status NeonSubtractionWorkloadValidate(const TensorInfo &input0, const TensorInfo &input1, const TensorInfo &output, const ActivationDescriptor *activationDescriptor)
Definition: NeonSubtractionWorkload.cpp:22
armnn::ConnectedToLayerType
bool ConnectedToLayerType(Layer *baseLayer, LayerType layerType, unsigned int dimSize=0)
Checks the Layer's Connections to see if it's connected to a Layer with the provided layerType.
Definition: SubgraphUtils.hpp:224
armnn::BinaryOperation::Sub
@ Sub
armnn::NeonTensorHandleFactory::GetIdStatic
static const FactoryId & GetIdStatic()
Definition: NeonTensorHandleFactory.cpp:89
armnn::LayerType::Reduce
@ Reduce
armnn::IBackendInternal::IBackendContextPtr
std::unique_ptr< IBackendContext > IBackendContextPtr
Definition: IBackendInternal.hpp:90
armnn::LayerType::ElementwiseBinary
@ ElementwiseBinary
armnn::cpuAccCapabilities
const BackendCapabilities cpuAccCapabilities("CpuAcc", { {"NonConstWeights", true}, {"AsyncExecution", false}, {"ProtectedContentAllocation", false}, {"ConstantTensorsAsInputs", true}, {"PreImportIOTensors", false}, {"ExternallyManagedMemory", true}, {"MultiAxisPacking", false}, {"SingleAxisPacking", true}, {"HasFp16", arm_compute::CPUInfo::get().has_fp16()} })
armnn::NeonDivisionWorkloadValidate
arm_compute::Status NeonDivisionWorkloadValidate(const TensorInfo &input0, const TensorInfo &input1, const TensorInfo &output, const ActivationDescriptor *activationDescriptor)
Definition: NeonDivisionWorkload.cpp:18
armnn::LayerType::Subtraction
@ Subtraction
armnn::RemoveReshapeLayer
void RemoveReshapeLayer(ReshapeLayer *baseLayer, std::map< LayerGuid, Layer * > &untouched, OptimizationViews &optimizationViews)
Definition: SubgraphUtils.hpp:246
armnn::LayerType::Multiplication
@ Multiplication
armnn::LayerType::Addition
@ Addition
armnn::NeonDepthwiseConvolutionWorkloadValidate
arm_compute::Status NeonDepthwiseConvolutionWorkloadValidate(const TensorInfo &input, const TensorInfo &output, const DepthwiseConvolution2dDescriptor &descriptor, const TensorInfo &weights, const Optional< TensorInfo > &biases, const ActivationDescriptor *activationDescriptor)
Definition: NeonDepthwiseConvolutionWorkload.cpp:29
armnn::NeonBackend::GetIdStatic
static const BackendId & GetIdStatic()
Definition: NeonBackend.cpp:44
armnn::LayerType::Division
@ Division
armnn::IBackendInternal::IBackendProfilingContextPtr
std::shared_ptr< arm::pipe::IBackendProfilingContext > IBackendProfilingContextPtr
This is the bridge between backend and backend profiling we'll keep it in the backend namespace.
Definition: IBackendInternal.hpp:92
armnn::LayerType::FullyConnected
@ FullyConnected
armnn::NeonBackend::CreateBackendSpecificModelContext
IBackendInternal::IBackendSpecificModelContextPtr CreateBackendSpecificModelContext(const ModelOptions &modelOptions) const override
Definition: NeonBackend.cpp:118
armnn::LayerType::DepthwiseConvolution2d
@ DepthwiseConvolution2d
armnn::Status
Status
Definition: Types.hpp:42
armnn::LayerType::Reshape
@ Reshape
armnn::NeonConvolution2dWorkloadValidate
arm_compute::Status NeonConvolution2dWorkloadValidate(const TensorInfo &input, const TensorInfo &output, const Convolution2dDescriptor &descriptor, const TensorInfo &weights, const Optional< TensorInfo > &biases, bool isFastMathEnabled, const ActivationDescriptor *activationDescriptor)
Definition: NeonConvolution2dWorkload.cpp:24
armnn::NeonBatchNormalizationValidate
arm_compute::Status NeonBatchNormalizationValidate(const TensorInfo &input, const TensorInfo &output, const TensorInfo &mean, const TensorInfo &var, const TensorInfo &beta, const TensorInfo &gamma, const BatchNormalizationDescriptor &descriptor, const ActivationDescriptor *activationDescriptor)
Definition: NeonBatchNormalizationWorkload.cpp:24
armnn::IBackendInternal::ILayerSupportSharedPtr
std::shared_ptr< ILayerSupport > ILayerSupportSharedPtr
Definition: IBackendInternal.hpp:94
armnn::ReportUntouchedLayers
void ReportUntouchedLayers(OptimizationViews &optimizationViews, std::map< LayerGuid, Layer * > untouched)
Definition: SubgraphUtils.hpp:173
armnn::ConnectedToLayerWithNCHW
bool ConnectedToLayerWithNCHW(Layer *baseLayer)
Checks if the Layer is connected to any Layer that has an NCHW layout.
Definition: SubgraphUtils.hpp:203
armnn::BinaryOperation::Div
@ Div
armnn::LayerType::Convolution2d
@ Convolution2d
armnn::NeonBackendId
constexpr const char * NeonBackendId()
Definition: NeonBackendId.hpp:10
armnn::LayerType::Activation
@ Activation
armnn::IBackendInternal::IBackendSpecificModelContextPtr
std::shared_ptr< IBackendModelContext > IBackendSpecificModelContextPtr
Definition: IBackendInternal.hpp:96