ClBackend Class Reference

#include <ClBackend.hpp>

Inheritance diagram for ClBackend:

Classes
class	ClBackendCustomAllocatorMemoryRegion
class	ClBackendCustomAllocatorWrapper

Public Member Functions
	ClBackend ()
	ClBackend (std::shared_ptr< ICustomAllocator > allocator)
	~ClBackend ()=default
const BackendId &	GetId () const override
IBackendInternal::IMemoryManagerUniquePtr	CreateMemoryManager () const override
IBackendInternal::IWorkloadFactoryPtr	CreateWorkloadFactory (const IBackendInternal::IMemoryManagerSharedPtr &memoryManager=nullptr) const override
IBackendInternal::IWorkloadFactoryPtr	CreateWorkloadFactory (TensorHandleFactoryRegistry &registry) const override
IWorkloadFactoryPtr	CreateWorkloadFactory (const IMemoryManagerSharedPtr &memoryManager, const ModelOptions &modelOptions) const override
IWorkloadFactoryPtr	CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions) const override
IWorkloadFactoryPtr	CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) const override
std::vector< ITensorHandleFactory::FactoryId >	GetHandleFactoryPreferences () const override
	(Optional) Returns a vector of supported TensorHandleFactory ids in preference order. More...
void	RegisterTensorHandleFactories (TensorHandleFactoryRegistry &registry) override
	(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor()/IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
void	RegisterTensorHandleFactories (TensorHandleFactoryRegistry &registry, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) override
	(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor()/IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
IBackendInternal::IBackendContextPtr	CreateBackendContext (const IRuntime::CreationOptions &) const override
	Create the runtime context of the backend. More...
IBackendInternal::IBackendProfilingContextPtr	CreateBackendProfilingContext (const IRuntime::CreationOptions &, IBackendProfilingPtr &backendProfiling) override
	Create context specifically used for profiling interaction from backends. More...
IBackendInternal::ILayerSupportSharedPtr	GetLayerSupport () const override
IBackendInternal::ILayerSupportSharedPtr	GetLayerSupport (const ModelOptions &modelOptions) const override
OptimizationViews	OptimizeSubgraphView (const SubgraphView &subgraph, const ModelOptions &modelOptions) const override
IBackendInternal::IBackendSpecificModelContextPtr	CreateBackendSpecificModelContext (const ModelOptions &modelOptions) const override
std::unique_ptr< ICustomAllocator >	GetDefaultAllocator () const override
	Returns the default memory allocator for the backend. More...
BackendCapabilities	GetCapabilities () const override
	Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted. More...
virtual bool	UseCustomMemoryAllocator (std::shared_ptr< ICustomAllocator > allocator, armnn::Optional< std::string &> errMsg) override
	Signals the backend to use a custom memory allocator provided by the user. More...
virtual unsigned int	GetNumberOfCacheFiles () const override
	Returns the number of files cached if backend supports caching. More...
Public Member Functions inherited from IBackendInternal
	~IBackendInternal () override=default
	Allow backends created by the factory function to be destroyed through IBackendInternal. More...
virtual OptimizationViews	OptimizeSubgraphView (const SubgraphView &subgraph) const
bool	SupportsTensorAllocatorAPI () const
ITensorHandleFactory::FactoryId	GetBackwardCompatibleFavoriteHandleFactory ()
virtual bool	HasCapability (BackendCapability) const
	Returns true if backend support the capability false otherwise. More...

Static Public Member Functions
static const BackendId &	GetIdStatic ()
Static Public Member Functions inherited from IBackendInternal
static constexpr BackendVersion	GetApiVersion ()
	Returns the version of the Backend API. More...

Public Attributes
std::shared_ptr< ClBackendCustomAllocatorWrapper >	m_CustomAllocator
bool	m_UsingCustomAllocator = false

Additional Inherited Members
Public Types inherited from IBackendInternal
using	IWorkloadFactoryPtr = std::unique_ptr< IWorkloadFactory >
using	IBackendContextPtr = std::unique_ptr< IBackendContext >
using	IBackendProfilingContextPtr = std::shared_ptr< armnn::profiling::IBackendProfilingContext >
	This is the bridge between backend and backend profiling we'll keep it in the backend namespace. More...
using	IBackendProfilingPtr = std::unique_ptr< armnn::profiling::IBackendProfiling >
using	ILayerSupportSharedPtr = std::shared_ptr< ILayerSupport >
using	IBackendSpecificModelContextPtr = std::shared_ptr< IBackendModelContext >
using	IMemoryManagerUniquePtr = std::unique_ptr< IMemoryManager >
using	IMemoryManagerSharedPtr = std::shared_ptr< IMemoryManager >
Protected Member Functions inherited from IBackendInternal
	IBackendInternal ()=default
	Creation must be done through a specific backend interface. More...
Protected Member Functions inherited from IBackend
	IBackend ()
virtual	~IBackend ()

Detailed Description

Definition at line 37 of file ClBackend.hpp.

Constructor & Destructor Documentation

ClBackend() [1/2]

ClBackend ( )

inline

Definition at line 40 of file ClBackend.hpp.

: m_CustomAllocator(nullptr) {};

ClBackend() [2/2]

ClBackend ( std::shared_ptr< ICustomAllocator >  allocator )

inline

Definition at line 41 of file ClBackend.hpp.

References ClBackend::GetIdStatic(), ClBackend::UseCustomMemoryAllocator(), and ClBackend::~ClBackend().

    {
        std::string err;
        UseCustomMemoryAllocator(allocator, err);
    }

~ClBackend()

~ClBackend ( )

default

Referenced by ClBackend::ClBackend().

Member Function Documentation

CreateBackendContext()

IBackendInternal::IBackendContextPtr CreateBackendContext ( const IRuntime::CreationOptions &  ) const

overridevirtual

Create the runtime context of the backend.

Implementations may return a default-constructed IBackendContextPtr if no context is needed at runtime. Implementations must throw BackendUnavailableException if the backend cannot be used (for example, necessary accelerator hardware is not present). The default implementation always returns a default-constructed pointer.

Reimplemented from IBackendInternal.

Definition at line 236 of file ClBackend.cpp.

Referenced by ClBackend::GetId().

{
    return IBackendContextPtr{new ClBackendContext{options}};
}

CreateBackendProfilingContext()

IBackendInternal::IBackendProfilingContextPtr CreateBackendProfilingContext ( const IRuntime::CreationOptions &  creationOptions, IBackendProfilingPtr &  backendProfiling  )

overridevirtual

Create context specifically used for profiling interaction from backends.

Reimplemented from IBackendInternal.

Definition at line 241 of file ClBackend.cpp.

Referenced by ClBackend::GetId().

{
    return IBackendProfilingContextPtr{};
}

CreateBackendSpecificModelContext()

IBackendInternal::IBackendSpecificModelContextPtr CreateBackendSpecificModelContext ( const ModelOptions &  modelOptions ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 247 of file ClBackend.cpp.

Referenced by ClBackend::CreateWorkloadFactory(), ClBackend::GetId(), ClBackend::GetLayerSupport(), and ClBackend::OptimizeSubgraphView().

{
    return IBackendSpecificModelContextPtr{new ClBackendModelContext{modelOptions}};
}

CreateMemoryManager()

IBackendInternal::IMemoryManagerUniquePtr CreateMemoryManager ( ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 51 of file ClBackend.cpp.

References ClBackend::m_CustomAllocator, and ClBackend::m_UsingCustomAllocator.

Referenced by ClBackend::GetId().

{
    if (m_UsingCustomAllocator)
    {
        return std::make_unique<ClMemoryManager>(m_CustomAllocator);
    }
    return std::make_unique<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
}

CreateWorkloadFactory() [1/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IBackendInternal::IMemoryManagerSharedPtr &  memoryManager = nullptr ) const

overridevirtual

Implements IBackendInternal.

Definition at line 60 of file ClBackend.cpp.

Referenced by ClBackend::GetId().

{
    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager));
}

CreateWorkloadFactory() [2/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( TensorHandleFactoryRegistry &  registry ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 74 of file ClBackend.cpp.

References ClBackend::m_CustomAllocator, ClBackend::m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

{
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }

    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<ClTensorHandleFactory>(memoryManager);
    std::unique_ptr<ITensorHandleFactory> importFactory = std::make_unique<ClImportTensorHandleFactory>(
        static_cast<MemorySourceFlags>(MemorySource::Malloc), static_cast<MemorySourceFlags>(MemorySource::Malloc));

    registry.RegisterCopyAndImportFactoryPair(factory->GetId(), importFactory->GetId());
    registry.RegisterCopyAndImportFactoryPair(importFactory->GetId(), factory->GetId());

    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
    registry.RegisterFactory(std::move(importFactory));

    return std::make_unique<ClWorkloadFactory>(
            PolymorphicPointerDowncast<ClMemoryManager>(memoryManager));
}

CreateWorkloadFactory() [3/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IMemoryManagerSharedPtr &  memoryManager, const ModelOptions &  modelOptions  ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 67 of file ClBackend.cpp.

References ClBackend::CreateBackendSpecificModelContext().

{
    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
}

CreateWorkloadFactory() [4/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry &  tensorHandleFactoryRegistry, const ModelOptions &  modelOptions  ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 102 of file ClBackend.cpp.

References ClBackend::CreateBackendSpecificModelContext(), ClBackend::m_CustomAllocator, ClBackend::m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

{
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }

    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<ClTensorHandleFactory>(memoryManager);
    std::unique_ptr<ITensorHandleFactory> importFactory = std::make_unique<ClImportTensorHandleFactory>(
        static_cast<MemorySourceFlags>(MemorySource::Malloc), static_cast<MemorySourceFlags>(MemorySource::Malloc));

    registry.RegisterCopyAndImportFactoryPair(factory->GetId(), importFactory->GetId());
    registry.RegisterCopyAndImportFactoryPair(importFactory->GetId(), factory->GetId());

    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
    registry.RegisterFactory(std::move(importFactory));

    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
}

CreateWorkloadFactory() [5/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry &  tensorHandleFactoryRegistry, const ModelOptions &  modelOptions, MemorySourceFlags  inputFlags, MemorySourceFlags  outputFlags  ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 130 of file ClBackend.cpp.

References ClBackend::CreateBackendSpecificModelContext(), ClBackend::m_CustomAllocator, ClBackend::m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), TensorHandleFactoryRegistry::RegisterMemoryManager(), and armnn::Undefined.

{
    // To allow force import if inputFlags/outputFlags are Undefined, set it as Malloc
    if (inputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        inputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
    if (outputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        outputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }

    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<ClTensorHandleFactory>(memoryManager);
    std::unique_ptr<ITensorHandleFactory> importFactory = std::make_unique<ClImportTensorHandleFactory>(
            inputFlags, outputFlags);

    registry.RegisterCopyAndImportFactoryPair(factory->GetId(), importFactory->GetId());
    registry.RegisterCopyAndImportFactoryPair(importFactory->GetId(), factory->GetId());

    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
    registry.RegisterFactory(std::move(importFactory));

    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
}

GetCapabilities()

BackendCapabilities GetCapabilities ( ) const

inlineoverridevirtual

Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted.

Reimplemented from IBackendInternal.

Definition at line 93 of file ClBackend.hpp.

References armnn::gpuAccCapabilities.

    {
        return gpuAccCapabilities;
    };

GetDefaultAllocator()

std::unique_ptr< ICustomAllocator > GetDefaultAllocator ( ) const

overridevirtual

Returns the default memory allocator for the backend.

Returns

- Returns unique pointer to the Default Allocator of the Backend

Reimplemented from IBackendInternal.

Definition at line 271 of file ClBackend.cpp.

Referenced by ClBackend::GetId().

{
    return std::make_unique<ClBackendDefaultAllocator>();
}

GetHandleFactoryPreferences()

std::vector< ITensorHandleFactory::FactoryId > GetHandleFactoryPreferences ( ) const

overridevirtual

(Optional) Returns a vector of supported TensorHandleFactory ids in preference order.

Reimplemented from IBackendInternal.

Definition at line 170 of file ClBackend.cpp.

References ClTensorHandleFactory::GetIdStatic(), and ClImportTensorHandleFactory::GetIdStatic().

Referenced by ClBackend::GetId().

{
    return std::vector<ITensorHandleFactory::FactoryId> {ClTensorHandleFactory::GetIdStatic(),
                                                         ClImportTensorHandleFactory::GetIdStatic()};
}

GetId()

const BackendId& GetId ( ) const

inlineoverridevirtual

Implements IBackend.

Definition at line 49 of file ClBackend.hpp.

References ClBackend::CreateBackendContext(), ClBackend::CreateBackendProfilingContext(), ClBackend::CreateBackendSpecificModelContext(), ClBackend::CreateMemoryManager(), ClBackend::CreateWorkloadFactory(), ClBackend::GetDefaultAllocator(), ClBackend::GetHandleFactoryPreferences(), ClBackend::GetIdStatic(), ClBackend::GetLayerSupport(), ClBackend::OptimizeSubgraphView(), and ClBackend::RegisterTensorHandleFactories().

{ return GetIdStatic(); }

GetIdStatic()

const BackendId & GetIdStatic ( )

static

Definition at line 45 of file ClBackend.cpp.

References armnn::ClBackendId().

Referenced by ClBackend::ClBackend(), and ClBackend::GetId().

{
    static const BackendId s_Id{ClBackendId()};
    return s_Id;
}

GetLayerSupport() [1/2]

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( ) const

overridevirtual

Implements IBackendInternal.

Definition at line 253 of file ClBackend.cpp.

Referenced by ClBackend::GetId().

{
    static ILayerSupportSharedPtr layerSupport
        {
            new ClLayerSupport(IBackendInternal::IBackendSpecificModelContextPtr{})
        };
    return layerSupport;
}

GetLayerSupport() [2/2]

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( const ModelOptions &  modelOptions ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 262 of file ClBackend.cpp.

References ClBackend::CreateBackendSpecificModelContext().

{
    static ILayerSupportSharedPtr layerSupport
    {
        new ClLayerSupport(CreateBackendSpecificModelContext(modelOptions))
    };
    return layerSupport;
}

GetNumberOfCacheFiles()

virtual unsigned int GetNumberOfCacheFiles ( ) const

inlineoverridevirtual

Returns the number of files cached if backend supports caching.

Returns

- Returns 0 if backend does not support caching otherwise number of files cached

Reimplemented from IBackendInternal.

Definition at line 110 of file ClBackend.hpp.

{ return 1; }

OptimizeSubgraphView()

OptimizationViews OptimizeSubgraphView ( const SubgraphView &  subgraph, const ModelOptions &  modelOptions  ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 276 of file ClBackend.cpp.

References armnn::Activation, armnn::Addition, OptimizationViews::AddUntouchedSubgraph(), armnn::BatchNormalization, SubgraphView::beginIConnectable(), Layer::BeginOutputSlots(), armnn::ClAdditionValidate(), armnn::ClBatchNormalizationValidate(), armnn::ClConvolution2dWorkloadValidate(), armnn::ClDepthwiseConvolutionWorkloadValidate(), armnn::ClDivisionWorkloadValidate(), armnn::ClFullyConnectedWorkloadValidate(), armnn::ClMultiplicationWorkloadValidate(), armnn::ClSubtractionValidate(), armnn::Convolution2d, ClBackend::CreateBackendSpecificModelContext(), armnn::DepthwiseConvolution2d, armnn::Division, SubgraphView::endIConnectable(), Layer::EndOutputSlots(), armnn::FullyConnected, Layer::GetAdditionalInformation(), InputSlot::GetConnectedOutputSlot(), Layer::GetGuid(), Layer::GetInputSlot(), Layer::GetName(), LayerWithParameters< Parameters >::GetParameters(), OptimizationViews::GetSubstitutions(), OutputSlot::GetTensorInfo(), Layer::GetType(), ClBackendModelContext::IsFastMathEnabled(), BatchNormalizationLayer::m_Beta, DepthwiseConvolution2dLayer::m_Bias, Convolution2dLayer::m_Bias, FullyConnectedLayer::m_Bias, FullyConnectedDescriptor::m_BiasEnabled, Convolution2dDescriptor::m_BiasEnabled, DepthwiseConvolution2dDescriptor::m_BiasEnabled, BatchNormalizationLayer::m_Gamma, BatchNormalizationLayer::m_Mean, BatchNormalizationLayer::m_Variance, DepthwiseConvolution2dLayer::m_Weight, Convolution2dLayer::m_Weight, FullyConnectedLayer::m_Weight, armnn::Multiplication, armnn::Reduce, armnn::ReportUntouchedLayers(), and armnn::Subtraction.

Referenced by ClBackend::GetId().

{
    OptimizationViews optimizationViews;

    auto it = subgraph.endIConnectable();
    bool isFastMathEnabled = false;
    std::map<LayerGuid, Layer*> untouched;

    while (it != subgraph.beginIConnectable())
    {
        --it;
        Layer& base = *(PolymorphicDowncast<Layer*>(*it));
        untouched.insert({base.GetGuid(), &base});
    }

    it = subgraph.endIConnectable();
#if defined(ARMCOMPUTECL_ENABLED)
    IBackendInternal::IBackendSpecificModelContextPtr modelContextPtr = CreateBackendSpecificModelContext(modelOptions);

    if (modelContextPtr)
    {
        auto clModelOptions = dynamic_cast<ClBackendModelContext*>(modelContextPtr.get());
        if (clModelOptions)
        {
            isFastMathEnabled = clModelOptions->IsFastMathEnabled();
        }
    }
#endif
    while (it != subgraph.beginIConnectable())
    {
        --it;
        Layer& base = *(PolymorphicDowncast<Layer*>(*it));

        // Fuse activation into previous layer if supported by backend
        if ((base.GetType() == LayerType::DepthwiseConvolution2d || base.GetType() == LayerType::Convolution2d
            || base.GetType() == LayerType::BatchNormalization || base.GetType() == LayerType::FullyConnected
            || base.GetType() == LayerType::Addition || base.GetType() == LayerType::Multiplication
            || base.GetType() == LayerType::Subtraction || base.GetType() == LayerType::Division)
            && (base.GetAdditionalInformation<ActivationDescriptor>() == nullptr))
        {
            for (auto output = base.BeginOutputSlots(); output != base.EndOutputSlots(); ++output)
            {
                if (output->GetNumConnections() == 1)
                {
                    for (auto&& childInput : output->GetConnections())
                    {
                        if ((childInput->GetOwningLayer().GetType() == LayerType::Activation) &&
                            (checkDataTypeInputandOutput(childInput->GetOwningLayer())))
                        {
                            Layer& child = childInput->GetOwningLayer();

                            auto* activationLayer = PolymorphicDowncast<ActivationLayer*>(&child);

                            const std::string name = std::string("fused-") + child.GetName() + std::string("-into-") +
                                                     base.GetName();

                            // Get params from activation layer
                            ActivationDescriptor activationDesc = activationLayer->GetParameters();

                            if (base.GetType() == LayerType::Convolution2d)
                            {
                                Convolution2dLayer* baseLayer = PolymorphicDowncast<Convolution2dLayer*>(&base);

                                Optional<TensorInfo> biases;

                                if (baseLayer->GetParameters().m_BiasEnabled)
                                {
                                    biases = baseLayer->m_Bias->GetTensorInfo();
                                }

                                arm_compute::Status status = ClConvolution2dWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetParameters(),
                                        baseLayer->m_Weight->GetTensorInfo(),
                                        biases,
                                        isFastMathEnabled,
                                        &activationDesc);

                                if (status)
                                {
                                    FuseConvolution2dLayer<Convolution2dLayer>(optimizationViews,
                                                                               baseLayer,
                                                                               activationLayer,
                                                                               activationDesc,
                                                                               name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::DepthwiseConvolution2d)
                            {
                                DepthwiseConvolution2dLayer* baseLayer =
                                        PolymorphicDowncast<DepthwiseConvolution2dLayer*>(&base);

                                Optional<TensorInfo> biases;

                                if (baseLayer->GetParameters().m_BiasEnabled)
                                {
                                    biases = baseLayer->m_Bias->GetTensorInfo();
                                }

                                arm_compute::Status status = ClDepthwiseConvolutionWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetParameters(),
                                        baseLayer->m_Weight->GetTensorInfo(),
                                        biases,
                                        &activationDesc);

                                if (status)
                                {
                                    FuseDepthwiseConvolution2dLayer<DepthwiseConvolution2dLayer>(optimizationViews,
                                                                                                 baseLayer,
                                                                                                 activationLayer,
                                                                                                 activationDesc,
                                                                                                 name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::FullyConnected)
                            {
                                FullyConnectedLayer* baseLayer = PolymorphicDowncast<FullyConnectedLayer*>(&base);

                                Optional<TensorInfo> biases;

                                if (baseLayer->GetParameters().m_BiasEnabled)
                                {
                                    biases = baseLayer->m_Bias->GetTensorInfo();
                                }

                                arm_compute::Status status = ClFullyConnectedWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->m_Weight->GetTensorInfo(),
                                        biases,
                                        baseLayer->GetParameters(),
                                        &activationDesc);

                                if (status)
                                {
                                    FuseFullyConnectedLayer<FullyConnectedLayer>(optimizationViews,
                                                                                 baseLayer,
                                                                                 activationLayer,
                                                                                 activationDesc,
                                                                                 name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::BatchNormalization)
                            {
                                BatchNormalizationLayer* baseLayer =
                                        PolymorphicDowncast<BatchNormalizationLayer*>(&base);

                                arm_compute::Status status = ClBatchNormalizationValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->m_Mean->GetTensorInfo(),
                                        baseLayer->m_Variance->GetTensorInfo(),
                                        baseLayer->m_Beta->GetTensorInfo(),
                                        baseLayer->m_Gamma->GetTensorInfo(),
                                        baseLayer->GetParameters(),
                                        &activationDesc);

                                if (status)
                                {
                                    BatchNormalizationLayer* replacementLayer =
                                        FuseBatchNormalizationLayer<BatchNormalizationLayer>(optimizationViews,
                                                                                             baseLayer,
                                                                                             activationLayer,
                                                                                             activationDesc,
                                                                                             name);

                                    replacementLayer->m_Beta     = std::move(baseLayer->m_Beta);
                                    replacementLayer->m_Gamma    = std::move(baseLayer->m_Gamma);
                                    replacementLayer->m_Mean     = std::move(baseLayer->m_Mean);
                                    replacementLayer->m_Variance = std::move(baseLayer->m_Variance);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Addition)
                            {
                                AdditionLayer* baseLayer = PolymorphicDowncast<AdditionLayer*>(&base);

                                arm_compute::Status status = ClAdditionValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);

                                if (status)
                                {
                                    FuseAdditionLayer<AdditionLayer>(optimizationViews,
                                                                     baseLayer,
                                                                     activationLayer,
                                                                     activationDesc,
                                                                     name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Division)
                            {
                                DivisionLayer* baseLayer = PolymorphicDowncast<DivisionLayer*>(&base);

                                arm_compute::Status status = ClDivisionWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);

                                if (status)
                                {
                                    FuseDivisionLayer<DivisionLayer>(optimizationViews,
                                                                     baseLayer,
                                                                     activationLayer,
                                                                     activationDesc,
                                                                     name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Multiplication)
                            {
                                MultiplicationLayer* baseLayer = PolymorphicDowncast<MultiplicationLayer*>(&base);

                                arm_compute::Status status = ClMultiplicationWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);

                                if (status)
                                {
                                    FuseMultiplicationLayer<MultiplicationLayer>(optimizationViews,
                                                                                 baseLayer,
                                                                                 activationLayer,
                                                                                 activationDesc,
                                                                                 name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Subtraction)
                            {
                                SubtractionLayer* baseLayer = PolymorphicDowncast<SubtractionLayer*>(&base);

                                arm_compute::Status status = ClSubtractionValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);

                                if (status)
                                {
                                    FuseSubtractionLayer<SubtractionLayer>(optimizationViews,
                                                                           baseLayer,
                                                                           activationLayer,
                                                                           activationDesc,
                                                                           name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                        }
                    }
                }
            }
        }

        // Separate reduce layer with multiple axes into multiple reduce layers with 1 axis.
        if (base.GetType() == LayerType::Reduce)
        {
            ReduceLayer* baseLayer            = PolymorphicDowncast<ReduceLayer*>(&base);
            ReduceDescriptor reduceDescriptor = baseLayer->GetParameters();

            if (!reduceDescriptor.m_vAxis.empty() && reduceDescriptor.m_vAxis.size() > 1)
            {
                // Add new layers to the graph and connect them.
                std::vector<IConnectableLayer*> layers = ChainReduceLayers<ReduceLayer>(optimizationViews,
                                                                                        baseLayer,
                                                                                        reduceDescriptor);

                // Replace existing baselayer with new subgraph.
                ReplaceLayers<ReduceLayer>(optimizationViews, baseLayer, layers);
                untouched.erase(baseLayer->GetGuid());
            }
        }
    }

    if (optimizationViews.GetSubstitutions().empty())
    {
        optimizationViews.AddUntouchedSubgraph(SubgraphView(subgraph));
    }
    else
    {
        ReportUntouchedLayers(optimizationViews, untouched);
    }

    return optimizationViews;
}

RegisterTensorHandleFactories() [1/2]

void RegisterTensorHandleFactories ( TensorHandleFactoryRegistry &  )

overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor()/IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 176 of file ClBackend.cpp.

References ClBackend::m_CustomAllocator, ClBackend::m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

Referenced by ClBackend::GetId().

{
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }

    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<ClTensorHandleFactory>(memoryManager);
    std::unique_ptr<ITensorHandleFactory> importFactory = std::make_unique<ClImportTensorHandleFactory>(
        static_cast<MemorySourceFlags>(MemorySource::Malloc), static_cast<MemorySourceFlags>(MemorySource::Malloc));

    registry.RegisterCopyAndImportFactoryPair(factory->GetId(), importFactory->GetId());
    registry.RegisterCopyAndImportFactoryPair(importFactory->GetId(), factory->GetId());

    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
    registry.RegisterFactory(std::move(importFactory));

}

RegisterTensorHandleFactories() [2/2]

void RegisterTensorHandleFactories ( TensorHandleFactoryRegistry &  registry, MemorySourceFlags  inputFlags, MemorySourceFlags  outputFlags  )

overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor()/IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 201 of file ClBackend.cpp.

References ClBackend::m_CustomAllocator, ClBackend::m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterCopyAndImportFactoryPair(), TensorHandleFactoryRegistry::RegisterFactory(), TensorHandleFactoryRegistry::RegisterMemoryManager(), and armnn::Undefined.

{
    // To allow force import if inputFlags/outputFlags are Undefined, set it as Malloc
    if (inputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        inputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
    if (outputFlags == static_cast<MemorySourceFlags>(MemorySource::Undefined))
    {
        outputFlags = static_cast<MemorySourceFlags>(MemorySource::Malloc);
    }
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }

    std::unique_ptr<ITensorHandleFactory> factory = std::make_unique<ClTensorHandleFactory>(memoryManager);
    std::unique_ptr<ITensorHandleFactory> importFactory = std::make_unique<ClImportTensorHandleFactory>(
            inputFlags, outputFlags);

    registry.RegisterCopyAndImportFactoryPair(factory->GetId(), importFactory->GetId());
    registry.RegisterCopyAndImportFactoryPair(importFactory->GetId(), factory->GetId());

    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::move(factory));
    registry.RegisterFactory(std::move(importFactory));
}

UseCustomMemoryAllocator()

virtual bool UseCustomMemoryAllocator ( std::shared_ptr< ICustomAllocator >  allocator, armnn::Optional< std::string &>  errMsg  )

inlineoverridevirtual

Signals the backend to use a custom memory allocator provided by the user.

Parameters

allocator	- a pointer to the provided ICustomAllocator to use with this backend
errMsg	- Optional string variable to return error messages

Returns

- Returns true if switching to custom allocator was successful

Reimplemented from IBackendInternal.

Definition at line 98 of file ClBackend.hpp.

References ARMNN_LOG, armnn::IgnoreUnused(), armnn::info, ClBackend::m_CustomAllocator, and ClBackend::m_UsingCustomAllocator.

Referenced by ClBackend::ClBackend().

    {
        IgnoreUnused(errMsg);
        ARMNN_LOG(info) << "Using Custom Allocator for ClBackend";

        // Set flag to signal the backend to use a custom memory allocator
        m_CustomAllocator = std::make_shared<ClBackendCustomAllocatorWrapper>(std::move(allocator));
        m_UsingCustomAllocator = true;
        return m_UsingCustomAllocator;
    }

Member Data Documentation

m_CustomAllocator

std::shared_ptr<ClBackendCustomAllocatorWrapper> m_CustomAllocator

Definition at line 299 of file ClBackend.hpp.

Referenced by ClBackend::CreateMemoryManager(), ClBackend::CreateWorkloadFactory(), ClBackend::RegisterTensorHandleFactories(), and ClBackend::UseCustomMemoryAllocator().

m_UsingCustomAllocator

bool m_UsingCustomAllocator = false

Definition at line 300 of file ClBackend.hpp.

Referenced by ClBackend::CreateMemoryManager(), ClBackend::CreateWorkloadFactory(), ClBackend::RegisterTensorHandleFactories(), and ClBackend::UseCustomMemoryAllocator().

---

The documentation for this class was generated from the following files:

• src/backends/cl/ClBackend.hpp
• src/backends/cl/ClBackend.cpp