ClBackend Class Reference

#include <ClBackend.hpp>

Inheritance diagram for ClBackend:

Classes
class	ClBackendCustomAllocatorMemoryRegion
class	ClBackendCustomAllocatorWrapper

Public Member Functions
	ClBackend ()
	ClBackend (std::shared_ptr< ICustomAllocator > allocator)
	~ClBackend ()=default
const BackendId &	GetId () const override
IBackendInternal::IMemoryManagerUniquePtr	CreateMemoryManager () const override
IBackendInternal::IWorkloadFactoryPtr	CreateWorkloadFactory (const IBackendInternal::IMemoryManagerSharedPtr &memoryManager=nullptr) const override
IBackendInternal::IWorkloadFactoryPtr	CreateWorkloadFactory (TensorHandleFactoryRegistry &registry) const override
IWorkloadFactoryPtr	CreateWorkloadFactory (const IMemoryManagerSharedPtr &memoryManager, const ModelOptions &modelOptions) const override
IWorkloadFactoryPtr	CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions) const override
IWorkloadFactoryPtr	CreateWorkloadFactory (class TensorHandleFactoryRegistry &tensorHandleFactoryRegistry, const ModelOptions &modelOptions, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) const override
std::vector< ITensorHandleFactory::FactoryId >	GetHandleFactoryPreferences () const override
	(Optional) Returns a vector of supported TensorHandleFactory ids in preference order. More...
void	RegisterTensorHandleFactories (TensorHandleFactoryRegistry &registry) override
	(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor()/IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
void	RegisterTensorHandleFactories (TensorHandleFactoryRegistry &registry, MemorySourceFlags inputFlags, MemorySourceFlags outputFlags) override
	(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor()/IWorkloadFactory::CreateSubtensor() methods must be implemented. More...
IBackendInternal::IBackendContextPtr	CreateBackendContext (const IRuntime::CreationOptions &) const override
	Create the runtime context of the backend. More...
IBackendInternal::IBackendProfilingContextPtr	CreateBackendProfilingContext (const IRuntime::CreationOptions &, IBackendProfilingPtr &backendProfiling) override
	Create context specifically used for profiling interaction from backends. More...
IBackendInternal::Optimizations	GetOptimizations () const override
IBackendInternal::ILayerSupportSharedPtr	GetLayerSupport () const override
IBackendInternal::ILayerSupportSharedPtr	GetLayerSupport (const ModelOptions &modelOptions) const override
OptimizationViews	OptimizeSubgraphView (const SubgraphView &subgraph, const ModelOptions &modelOptions) const override
IBackendInternal::IBackendSpecificModelContextPtr	CreateBackendSpecificModelContext (const ModelOptions &modelOptions) const override
BackendCapabilities	GetCapabilities () const override
	Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted. More...
virtual bool	UseCustomMemoryAllocator (std::shared_ptr< ICustomAllocator > allocator, armnn::Optional< std::string &> errMsg) override
	Signals the backend to use a custom memory allocator provided by the user. More...
Public Member Functions inherited from IBackendInternal
	~IBackendInternal () override=default
	Allow backends created by the factory function to be destroyed through IBackendInternal. More...
virtual ISubGraphConverterPtr	CreateSubGraphConverter (const std::shared_ptr< SubGraph > &subGraph) const
virtual SubGraphUniquePtr	OptimizeSubGraph (const SubGraph &subGraph, bool &optimizationAttempted) const
virtual OptimizationViews	OptimizeSubgraphView (const SubgraphView &subgraph) const
bool	SupportsTensorAllocatorAPI () const
ITensorHandleFactory::FactoryId	GetBackwardCompatibleFavoriteHandleFactory ()
virtual bool	HasCapability (BackendCapability) const
	Returns true if backend support the capability false otherwise. More...

Static Public Member Functions
static const BackendId &	GetIdStatic ()
Static Public Member Functions inherited from IBackendInternal
static constexpr BackendVersion	GetApiVersion ()
	Returns the version of the Backend API. More...

Public Attributes
std::shared_ptr< ClBackendCustomAllocatorWrapper >	m_CustomAllocator
bool	m_UsingCustomAllocator = false

Additional Inherited Members
Public Types inherited from IBackendInternal
using	IWorkloadFactoryPtr = std::unique_ptr< IWorkloadFactory >
using	IBackendContextPtr = std::unique_ptr< IBackendContext >
using	IBackendProfilingContextPtr = std::shared_ptr< armnn::profiling::IBackendProfilingContext >
	This is the bridge between backend and backend profiling we'll keep it in the backend namespace. More...
using	IBackendProfilingPtr = std::unique_ptr< armnn::profiling::IBackendProfiling >
using	OptimizationPtr = std::unique_ptr< Optimization >
using	Optimizations = std::vector< OptimizationPtr >
using	ILayerSupportSharedPtr = std::shared_ptr< ILayerSupport >
using	IBackendSpecificModelContextPtr = std::shared_ptr< IBackendModelContext >
using	IMemoryManagerUniquePtr = std::unique_ptr< IMemoryManager >
using	IMemoryManagerSharedPtr = std::shared_ptr< IMemoryManager >
using	GraphUniquePtr = std::unique_ptr< Graph >
using	SubgraphViewUniquePtr = std::unique_ptr< SubgraphView >
using	supported = std::unique_ptr< ISubGraphConverter >
using	instead = std::unique_ptr< SubGraph >
Protected Member Functions inherited from IBackendInternal
	IBackendInternal ()=default
	Creation must be done through a specific backend interface. More...
Protected Member Functions inherited from IBackend
	IBackend ()
virtual	~IBackend ()

Detailed Description

Definition at line 32 of file ClBackend.hpp.

Constructor & Destructor Documentation

ClBackend() [1/2]

ClBackend ( )

inline

Definition at line 35 of file ClBackend.hpp.

: m_CustomAllocator(nullptr) {};

ClBackend() [2/2]

ClBackend ( std::shared_ptr< ICustomAllocator >  allocator )

inline

Definition at line 36 of file ClBackend.hpp.

References ClBackend::GetIdStatic(), ClBackend::UseCustomMemoryAllocator(), and ClBackend::~ClBackend().

    {
        std::string err;
        UseCustomMemoryAllocator(allocator, err);
    }

~ClBackend()

~ClBackend ( )

default

Referenced by ClBackend::ClBackend().

Member Function Documentation

CreateBackendContext()

IBackendInternal::IBackendContextPtr CreateBackendContext ( const IRuntime::CreationOptions &  ) const

overridevirtual

Create the runtime context of the backend.

Implementations may return a default-constructed IBackendContextPtr if no context is needed at runtime. Implementations must throw BackendUnavailableException if the backend cannot be used (for example, necessary accelerator hardware is not present). The default implementation always returns a default-constructed pointer.

Reimplemented from IBackendInternal.

Definition at line 184 of file ClBackend.cpp.

Referenced by ClBackend::GetId().

{
    return IBackendContextPtr{new ClBackendContext{options}};
}

CreateBackendProfilingContext()

IBackendInternal::IBackendProfilingContextPtr CreateBackendProfilingContext ( const IRuntime::CreationOptions &  creationOptions, IBackendProfilingPtr &  backendProfiling  )

overridevirtual

Create context specifically used for profiling interaction from backends.

Reimplemented from IBackendInternal.

Definition at line 189 of file ClBackend.cpp.

Referenced by ClBackend::GetId().

{
    return IBackendProfilingContextPtr{};
}

CreateBackendSpecificModelContext()

IBackendInternal::IBackendSpecificModelContextPtr CreateBackendSpecificModelContext ( const ModelOptions &  modelOptions ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 200 of file ClBackend.cpp.

Referenced by ClBackend::CreateWorkloadFactory(), ClBackend::GetId(), ClBackend::GetLayerSupport(), and ClBackend::OptimizeSubgraphView().

{
    return IBackendSpecificModelContextPtr{new ClBackendModelContext{modelOptions}};
}

CreateMemoryManager()

IBackendInternal::IMemoryManagerUniquePtr CreateMemoryManager ( ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 50 of file ClBackend.cpp.

References ClBackend::m_CustomAllocator, and ClBackend::m_UsingCustomAllocator.

Referenced by ClBackend::GetId().

{
    if (m_UsingCustomAllocator)
    {
        return std::make_unique<ClMemoryManager>(m_CustomAllocator);
    }
    return std::make_unique<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
}

CreateWorkloadFactory() [1/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IBackendInternal::IMemoryManagerSharedPtr &  memoryManager = nullptr ) const

overridevirtual

Implements IBackendInternal.

Definition at line 59 of file ClBackend.cpp.

Referenced by ClBackend::GetId().

{
    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager));
}

CreateWorkloadFactory() [2/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( TensorHandleFactoryRegistry &  registry ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 73 of file ClBackend.cpp.

References ClBackend::m_CustomAllocator, ClBackend::m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

{
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }

    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::make_unique<ClTensorHandleFactory>(memoryManager));
    registry.RegisterFactory(std::make_unique<ClImportTensorHandleFactory>(
        static_cast<MemorySourceFlags>(MemorySource::Malloc), static_cast<MemorySourceFlags>(MemorySource::Malloc)));

    return std::make_unique<ClWorkloadFactory>(
            PolymorphicPointerDowncast<ClMemoryManager>(memoryManager));
}

CreateWorkloadFactory() [3/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( const IMemoryManagerSharedPtr &  memoryManager, const ModelOptions &  modelOptions  ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 66 of file ClBackend.cpp.

References ClBackend::CreateBackendSpecificModelContext().

{
    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
}

CreateWorkloadFactory() [4/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry &  tensorHandleFactoryRegistry, const ModelOptions &  modelOptions  ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 95 of file ClBackend.cpp.

References ClBackend::CreateBackendSpecificModelContext(), ClBackend::m_CustomAllocator, ClBackend::m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

{
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }

    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::make_unique<ClTensorHandleFactory>(memoryManager));
    registry.RegisterFactory(std::make_unique<ClImportTensorHandleFactory>(
        static_cast<MemorySourceFlags>(MemorySource::Malloc), static_cast<MemorySourceFlags>(MemorySource::Malloc)));

    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
}

CreateWorkloadFactory() [5/5]

IBackendInternal::IWorkloadFactoryPtr CreateWorkloadFactory ( class TensorHandleFactoryRegistry &  tensorHandleFactoryRegistry, const ModelOptions &  modelOptions, MemorySourceFlags  inputFlags, MemorySourceFlags  outputFlags  ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 117 of file ClBackend.cpp.

References ClBackend::CreateBackendSpecificModelContext(), ClBackend::m_CustomAllocator, ClBackend::m_UsingCustomAllocator, TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

{
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }

    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::make_unique<ClTensorHandleFactory>(memoryManager));
    registry.RegisterFactory(std::make_unique<ClImportTensorHandleFactory>(inputFlags, outputFlags));

    return std::make_unique<ClWorkloadFactory>(
        PolymorphicPointerDowncast<ClMemoryManager>(memoryManager), CreateBackendSpecificModelContext(modelOptions));
}

GetCapabilities()

BackendCapabilities GetCapabilities ( ) const

inlineoverridevirtual

Returns a BackendCapability if the backend lists the capability The BackendCapability must then be inspected to check whether or not that BackendCapability is supported Otherwise returns an EmptyOptional if the BackendCapability is unlisted.

Reimplemented from IBackendInternal.

Definition at line 87 of file ClBackend.hpp.

References armnn::gpuAccCapabilities.

    {
        return gpuAccCapabilities;
    };

GetHandleFactoryPreferences()

std::vector< ITensorHandleFactory::FactoryId > GetHandleFactoryPreferences ( ) const

overridevirtual

(Optional) Returns a vector of supported TensorHandleFactory ids in preference order.

Reimplemented from IBackendInternal.

Definition at line 141 of file ClBackend.cpp.

References ClTensorHandleFactory::GetIdStatic(), and ClImportTensorHandleFactory::GetIdStatic().

Referenced by ClBackend::GetId().

{
    return std::vector<ITensorHandleFactory::FactoryId> {ClTensorHandleFactory::GetIdStatic(),
                                                         ClImportTensorHandleFactory::GetIdStatic()};
}

GetId()

const BackendId& GetId ( ) const

inlineoverridevirtual

Implements IBackend.

Definition at line 44 of file ClBackend.hpp.

References ClBackend::CreateBackendContext(), ClBackend::CreateBackendProfilingContext(), ClBackend::CreateBackendSpecificModelContext(), ClBackend::CreateMemoryManager(), ClBackend::CreateWorkloadFactory(), ClBackend::GetHandleFactoryPreferences(), ClBackend::GetIdStatic(), ClBackend::GetLayerSupport(), ClBackend::GetOptimizations(), ClBackend::OptimizeSubgraphView(), and ClBackend::RegisterTensorHandleFactories().

{ return GetIdStatic(); }

GetIdStatic()

const BackendId & GetIdStatic ( )

static

Definition at line 44 of file ClBackend.cpp.

References armnn::ClBackendId().

Referenced by ClBackend::ClBackend(), and ClBackend::GetId().

{
    static const BackendId s_Id{ClBackendId()};
    return s_Id;
}

GetLayerSupport() [1/2]

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( ) const

overridevirtual

Implements IBackendInternal.

Definition at line 206 of file ClBackend.cpp.

Referenced by ClBackend::GetId().

{
    static ILayerSupportSharedPtr layerSupport
        {
            new ClLayerSupport(IBackendInternal::IBackendSpecificModelContextPtr{})
        };
    return layerSupport;
}

GetLayerSupport() [2/2]

IBackendInternal::ILayerSupportSharedPtr GetLayerSupport ( const ModelOptions &  modelOptions ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 215 of file ClBackend.cpp.

References ClBackend::CreateBackendSpecificModelContext().

{
    static ILayerSupportSharedPtr layerSupport
    {
        new ClLayerSupport(CreateBackendSpecificModelContext(modelOptions))
    };
    return layerSupport;
}

GetOptimizations()

IBackendInternal::Optimizations GetOptimizations ( ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 195 of file ClBackend.cpp.

Referenced by ClBackend::GetId().

{
    return Optimizations{};
}

OptimizeSubgraphView()

OptimizationViews OptimizeSubgraphView ( const SubgraphView &  subgraph, const ModelOptions &  modelOptions  ) const

overridevirtual

Reimplemented from IBackendInternal.

Definition at line 224 of file ClBackend.cpp.

References armnn::Activation, armnn::Addition, OptimizationViews::AddUntouchedSubgraph(), armnn::BatchNormalization, SubgraphView::begin(), Layer::BeginOutputSlots(), armnn::ClAdditionValidate(), armnn::ClBatchNormalizationValidate(), armnn::ClConvolution2dWorkloadValidate(), armnn::ClDepthwiseConvolutionWorkloadValidate(), armnn::ClDivisionWorkloadValidate(), armnn::ClFullyConnectedWorkloadValidate(), armnn::ClMultiplicationWorkloadValidate(), armnn::ClSubtractionValidate(), armnn::Convolution2d, ClBackend::CreateBackendSpecificModelContext(), armnn::DepthwiseConvolution2d, armnn::Division, SubgraphView::end(), Layer::EndOutputSlots(), armnn::FullyConnected, Layer::GetAdditionalInformation(), InputSlot::GetConnectedOutputSlot(), Layer::GetGuid(), Layer::GetInputSlot(), Layer::GetName(), LayerWithParameters< Parameters >::GetParameters(), OptimizationViews::GetSubstitutions(), OutputSlot::GetTensorInfo(), Layer::GetType(), ClBackendModelContext::IsFastMathEnabled(), BatchNormalizationLayer::m_Beta, DepthwiseConvolution2dLayer::m_Bias, Convolution2dLayer::m_Bias, FullyConnectedLayer::m_Bias, Convolution2dDescriptor::m_BiasEnabled, DepthwiseConvolution2dDescriptor::m_BiasEnabled, BatchNormalizationLayer::m_Gamma, BatchNormalizationLayer::m_Mean, BatchNormalizationLayer::m_Variance, DepthwiseConvolution2dLayer::m_Weight, FullyConnectedLayer::m_Weight, Convolution2dLayer::m_Weight, armnn::Multiplication, armnn::Reduce, armnn::ReportUntouchedLayers(), and armnn::Subtraction.

Referenced by ClBackend::GetId().

{
    OptimizationViews optimizationViews;

    auto it = subgraph.end();
    bool isFastMathEnabled = false;
    std::map<LayerGuid, Layer*> untouched;

    while (it != subgraph.begin())
    {
        --it;
        Layer& base = **it;
        untouched.insert({base.GetGuid(), &base});
    }

    it = subgraph.end();
#if defined(ARMCOMPUTECL_ENABLED)
    IBackendInternal::IBackendSpecificModelContextPtr modelContextPtr = CreateBackendSpecificModelContext(modelOptions);

    if (modelContextPtr)
    {
        auto clModelOptions = dynamic_cast<ClBackendModelContext*>(modelContextPtr.get());
        if (clModelOptions)
        {
            isFastMathEnabled = clModelOptions->IsFastMathEnabled();
        }
    }
#endif
    while (it != subgraph.begin())
    {
        --it;
        Layer& base = **it;

        // Fuse activation into previous layer if supported by backend
        if ((base.GetType() == LayerType::DepthwiseConvolution2d || base.GetType() == LayerType::Convolution2d
            || base.GetType() == LayerType::BatchNormalization || base.GetType() == LayerType::FullyConnected
            || base.GetType() == LayerType::Addition || base.GetType() == LayerType::Multiplication
            || base.GetType() == LayerType::Subtraction || base.GetType() == LayerType::Division)
            && (base.GetAdditionalInformation<ActivationDescriptor>() == nullptr))
        {
            for (auto output = base.BeginOutputSlots(); output != base.EndOutputSlots(); ++output)
            {
                if (output->GetNumConnections() == 1)
                {
                    for (auto&& childInput : output->GetConnections())
                    {
                        if ((childInput->GetOwningLayer().GetType() == LayerType::Activation) &&
                            (checkDataTypeInputandOutput(childInput->GetOwningLayer())))
                        {
                            Layer& child = childInput->GetOwningLayer();

                            auto* activationLayer = PolymorphicDowncast<ActivationLayer*>(&child);

                            const std::string name = std::string("fused-") + child.GetName() + std::string("-into-") +
                                                     base.GetName();

                            // Get params from activation layer
                            ActivationDescriptor activationDesc = activationLayer->GetParameters();

                            if (base.GetType() == LayerType::Convolution2d)
                            {
                                Convolution2dLayer* baseLayer = PolymorphicDowncast<Convolution2dLayer*>(&base);

                                Optional<TensorInfo> biases;

                                if (baseLayer->GetParameters().m_BiasEnabled)
                                {
                                    biases = baseLayer->m_Bias->GetTensorInfo();
                                }

                                arm_compute::Status status = ClConvolution2dWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetParameters(),
                                        baseLayer->m_Weight->GetTensorInfo(),
                                        biases,
                                        isFastMathEnabled,
                                        &activationDesc);

                                if (status)
                                {
                                    FuseLayerWithWeightsAndBiases<Convolution2dLayer>(optimizationViews,
                                                                                      baseLayer,
                                                                                      activationLayer,
                                                                                      activationDesc,
                                                                                      name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::DepthwiseConvolution2d)
                            {
                                DepthwiseConvolution2dLayer* baseLayer =
                                        PolymorphicDowncast<DepthwiseConvolution2dLayer*>(&base);

                                Optional<TensorInfo> biases;

                                if (baseLayer->GetParameters().m_BiasEnabled)
                                {
                                    biases = baseLayer->m_Bias->GetTensorInfo();
                                }

                                arm_compute::Status status = ClDepthwiseConvolutionWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetParameters(),
                                        baseLayer->m_Weight->GetTensorInfo(),
                                        biases,
                                        &activationDesc);

                                if (status)
                                {
                                    FuseLayerWithWeightsAndBiases<DepthwiseConvolution2dLayer>(optimizationViews,
                                                                                               baseLayer,
                                                                                               activationLayer,
                                                                                               activationDesc,
                                                                                               name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::FullyConnected)
                            {
                                FullyConnectedLayer* baseLayer = PolymorphicDowncast<FullyConnectedLayer*>(&base);

                                arm_compute::Status status = ClFullyConnectedWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->m_Weight->GetTensorInfo(),
                                        baseLayer->m_Bias->GetTensorInfo(),
                                        baseLayer->GetParameters(),
                                        &activationDesc);

                                if (status)
                                {
                                    FuseLayerWithWeightsAndBiases<FullyConnectedLayer>(optimizationViews,
                                                                                       baseLayer,
                                                                                       activationLayer,
                                                                                       activationDesc,
                                                                                       name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::BatchNormalization)
                            {
                                BatchNormalizationLayer* baseLayer =
                                        PolymorphicDowncast<BatchNormalizationLayer*>(&base);

                                arm_compute::Status status = ClBatchNormalizationValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->m_Mean->GetTensorInfo(),
                                        baseLayer->m_Variance->GetTensorInfo(),
                                        baseLayer->m_Beta->GetTensorInfo(),
                                        baseLayer->m_Gamma->GetTensorInfo(),
                                        baseLayer->GetParameters(),
                                        &activationDesc);

                                if (status)
                                {
                                    BatchNormalizationLayer* replacementLayer =
                                            FuseLayerWithParameters<BatchNormalizationLayer>(optimizationViews,
                                                                                             baseLayer,
                                                                                             activationLayer,
                                                                                             activationDesc,
                                                                                             name);

                                    replacementLayer->m_Beta     = std::move(baseLayer->m_Beta);
                                    replacementLayer->m_Gamma    = std::move(baseLayer->m_Gamma);
                                    replacementLayer->m_Mean     = std::move(baseLayer->m_Mean);
                                    replacementLayer->m_Variance = std::move(baseLayer->m_Variance);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Addition)
                            {
                                AdditionLayer* baseLayer = PolymorphicDowncast<AdditionLayer*>(&base);

                                arm_compute::Status status = ClAdditionValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);

                                if (status)
                                {
                                    FuseLayerWithoutParameters<AdditionLayer>(optimizationViews,
                                                                              baseLayer,
                                                                              activationLayer,
                                                                              activationDesc,
                                                                              name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Division)
                            {
                                DivisionLayer* baseLayer = PolymorphicDowncast<DivisionLayer*>(&base);

                                arm_compute::Status status = ClDivisionWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);

                                if (status)
                                {
                                    FuseLayerWithoutParameters<DivisionLayer>(optimizationViews,
                                                                              baseLayer,
                                                                              activationLayer,
                                                                              activationDesc,
                                                                              name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Multiplication)
                            {
                                MultiplicationLayer* baseLayer = PolymorphicDowncast<MultiplicationLayer*>(&base);

                                arm_compute::Status status = ClMultiplicationWorkloadValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);

                                if (status)
                                {
                                    FuseLayerWithoutParameters<MultiplicationLayer>(optimizationViews,
                                                                                    baseLayer,
                                                                                    activationLayer,
                                                                                    activationDesc,
                                                                                    name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                            else if (base.GetType() == LayerType::Subtraction)
                            {
                                SubtractionLayer* baseLayer = PolymorphicDowncast<SubtractionLayer*>(&base);

                                arm_compute::Status status = ClSubtractionValidate(
                                        baseLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        baseLayer->GetInputSlot(1).GetConnectedOutputSlot()->GetTensorInfo(),
                                        activationLayer->GetInputSlot(0).GetConnectedOutputSlot()->GetTensorInfo(),
                                        &activationDesc);

                                if (status)
                                {
                                    FuseLayerWithoutParameters<SubtractionLayer>(optimizationViews,
                                                                                 baseLayer,
                                                                                 activationLayer,
                                                                                 activationDesc,
                                                                                 name);
                                    untouched.erase(baseLayer->GetGuid());
                                    untouched.erase(activationLayer->GetGuid());
                                }
                            }
                        }
                    }
                }
            }
        }

        // Separate reduce layer with multiple axes into multiple reduce layers with 1 axis.
        if (base.GetType() == LayerType::Reduce)
        {
            ReduceLayer* baseLayer            = PolymorphicDowncast<ReduceLayer*>(&base);
            ReduceDescriptor reduceDescriptor = baseLayer->GetParameters();

            if (!reduceDescriptor.m_vAxis.empty() && reduceDescriptor.m_vAxis.size() > 1)
            {
                // Add new layers to the graph and connect them.
                std::vector<Layer*> layers = ChainReduceLayers<ReduceLayer>(optimizationViews,
                                                                            baseLayer,
                                                                            reduceDescriptor);

                // Replace existing baselayer with new subgraph.
                ReplaceLayers<ReduceLayer>(optimizationViews, baseLayer, layers);
                untouched.erase(baseLayer->GetGuid());
            }
        }
    }

    if (optimizationViews.GetSubstitutions().empty())
    {
        optimizationViews.AddUntouchedSubgraph(SubgraphView(subgraph));
    }
    else
    {
        ReportUntouchedLayers(optimizationViews, untouched);
    }

    return optimizationViews;
}

RegisterTensorHandleFactories() [1/2]

void RegisterTensorHandleFactories ( TensorHandleFactoryRegistry &  )

overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor()/IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 147 of file ClBackend.cpp.

References ClBackend::m_CustomAllocator, ClBackend::m_UsingCustomAllocator, armnn::Malloc, TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

Referenced by ClBackend::GetId().

{
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }

    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::make_unique<ClTensorHandleFactory>(memoryManager));
    registry.RegisterFactory(std::make_unique<ClImportTensorHandleFactory>(
        static_cast<MemorySourceFlags>(MemorySource::Malloc), static_cast<MemorySourceFlags>(MemorySource::Malloc)));
}

RegisterTensorHandleFactories() [2/2]

void RegisterTensorHandleFactories ( TensorHandleFactoryRegistry &  registry, MemorySourceFlags  inputFlags, MemorySourceFlags  outputFlags  )

overridevirtual

(Optional) Register TensorHandleFactories Either this method or CreateMemoryManager() and IWorkloadFactory::CreateTensor()/IWorkloadFactory::CreateSubtensor() methods must be implemented.

Reimplemented from IBackendInternal.

Definition at line 165 of file ClBackend.cpp.

References ClBackend::m_CustomAllocator, ClBackend::m_UsingCustomAllocator, TensorHandleFactoryRegistry::RegisterFactory(), and TensorHandleFactoryRegistry::RegisterMemoryManager().

{
    std::shared_ptr<ClMemoryManager> memoryManager;
    if (m_UsingCustomAllocator)
    {
        memoryManager = std::make_shared<ClMemoryManager>(m_CustomAllocator);
    }
    else
    {
        memoryManager = std::make_shared<ClMemoryManager>(std::make_unique<arm_compute::CLBufferAllocator>());
    }

    registry.RegisterMemoryManager(memoryManager);
    registry.RegisterFactory(std::make_unique<ClTensorHandleFactory>(memoryManager));
    registry.RegisterFactory(std::make_unique<ClImportTensorHandleFactory>(inputFlags, outputFlags));
}

UseCustomMemoryAllocator()

virtual bool UseCustomMemoryAllocator ( std::shared_ptr< ICustomAllocator >  allocator, armnn::Optional< std::string &>  errMsg  )

inlineoverridevirtual

Signals the backend to use a custom memory allocator provided by the user.

Parameters

allocator	- a pointer to the provided ICustomAllocator to use with this backend
errMsg	- Optional string variable to return error messages

Returns

- Returns true if switching to custom allocator was successful

Reimplemented from IBackendInternal.

Definition at line 92 of file ClBackend.hpp.

References ARMNN_LOG, armnn::IgnoreUnused(), armnn::info, ClBackend::m_CustomAllocator, and ClBackend::m_UsingCustomAllocator.

Referenced by ClBackend::ClBackend().

    {
        IgnoreUnused(errMsg);
        ARMNN_LOG(info) << "Using Custom Allocator for ClBackend";

        // Set flag to signal the backend to use a custom memory allocator
        m_CustomAllocator = std::make_shared<ClBackendCustomAllocatorWrapper>(std::move(allocator));
        m_UsingCustomAllocator = true;
        return m_UsingCustomAllocator;
    }

Member Data Documentation

m_CustomAllocator

std::shared_ptr<ClBackendCustomAllocatorWrapper> m_CustomAllocator

Definition at line 291 of file ClBackend.hpp.

Referenced by ClBackend::CreateMemoryManager(), ClBackend::CreateWorkloadFactory(), ClBackend::RegisterTensorHandleFactories(), and ClBackend::UseCustomMemoryAllocator().

m_UsingCustomAllocator

bool m_UsingCustomAllocator = false

Definition at line 292 of file ClBackend.hpp.

Referenced by ClBackend::CreateMemoryManager(), ClBackend::CreateWorkloadFactory(), ClBackend::RegisterTensorHandleFactories(), and ClBackend::UseCustomMemoryAllocator().

---

The documentation for this class was generated from the following files:

• src/backends/cl/ClBackend.hpp
• src/backends/cl/ClBackend.cpp