From dd9d8ca997cb6c63677249350247e9f44525104c Mon Sep 17 00:00:00 2001 From: David Beck Date: Wed, 14 Nov 2018 11:33:30 +0000 Subject: IVGCVSW-2064 + IVGCVSW-2066 + IVGCVSW-2125 + IVGCVSW-2128 backend docs update Change-Id: Iccc1272e59bd71c59f810a54d6d3742859789212 --- src/backends/README.md | 118 ++++++++++++++++++++++++++++++++++--------------- 1 file changed, 83 insertions(+), 35 deletions(-) diff --git a/src/backends/README.md b/src/backends/README.md index 80b7096cac..125b41b05f 100644 --- a/src/backends/README.md +++ b/src/backends/README.md @@ -104,51 +104,99 @@ Backends are identified by a string that must be unique across backends. This st wrapped in the [BackendId](../../include/armnn/BackendId.hpp) object for backward compatibility with previous ArmNN versions. -## Registry interfaces +## The IBackendInternal interface + +All backends need to implement the [IBackendInternal](backendsCommon/IBackendInternal.hpp) interface. +The interface functions to be implemented are: + +```c++ + virtual IMemoryManagerUniquePtr CreateMemoryManager() const = 0; + virtual IWorkloadFactoryPtr CreateWorkloadFactory( + const IMemoryManagerSharedPtr& memoryManager = nullptr) const = 0; + virtual IBackendContextPtr CreateBackendContext(const IRuntime::CreationOptions&) const = 0; + virtual Optimizations GetOptimizations() const = 0; + virtual ILayerSupportSharedPtr GetLayerSupport() const = 0; +``` -To integrate a new backend, it needs to be registered through the following singleton classes: +The ArmNN framework then creates instances of the IBackendInternal interface with the help of the +[BackendRegistry](backendsCommon/BackendRegistry.hpp) singleton. -* [BackendRegistry](backendsCommon/BackendRegistry.hpp) -* [LayerSupportRegistry](backendsCommon/LayerSupportRegistry.hpp) +**Important:** the ```IBackendInternal``` object is not guaranteed to have a longer lifetime than +the objects it creates. It is only intended to be a single entry point for the factory functions it has. +The best use of this is to be a lightweight, stateless object and make no assumptions between +its lifetime and the lifetime of the objects it creates. -These Registries can register a factory function together with a [BackendId](../../include/armnn/BackendId.hpp) -key, that allows to create a registered object at a later time when needed. +For each backend one needs to register a factory function that can +be retrieved using a [BackendId](../../include/armnn/BackendId.hpp). +The ArmNN framework creates the backend interfaces dynamically when +it sees fit and it keeps these objects for a short period of time. Examples: -There is support for statically registering the factory functions in -the [RegistryCommon.hpp](backendsCommon/RegistryCommon.hpp) header. +* During optimization ArmNN needs to decide which layers are supported by the backend. + To do this, it creates a backends and calls the ```GetLayerSupport()``` function and creates + an ```ILayerSupport``` object to help deciding this. +* During optimization ArmNN can run backend specific optimizations. It creates backend objects + and calls the ```GetOptimizations()``` function and it runs them on the network. +* When the Runtime is initialized it creates an optional ```IBackendContext``` object and keeps this context alive + for the Runtime's lifetime. It notifies this context object before and after a network is loaded or unloaded. +* When the LoadedNetwork creates the backend specific workloads for the layers, it creates a backend + specific workload factory and calls this to create the workloads. -### The BackendRegistry and the IBackendInternal interface +## The BackendRegistry -The ```BackendRegistry``` registers a function that returns a unique pointer to an object that implements the [IBackendInternal interface](backendsCommon/IBackendInternal.hpp). +As mentioned above, all backends need to be registered through the BackendRegistry so ArmNN knows +about them. Registration requires a unique backend id string and a lambda function that +returns a unique pointer to the [IBackendInternal interface](backendsCommon/IBackendInternal.hpp). -During optimization we assign backends to the layers in the network. -When we pass the resulting optimized network to the ```LoadedNetwork```, -it calls the factory function for all backends that are required by the -given network. The ```LoadedNetwork``` holds a reference to these -backend objects for its lifetime. +For registering a backend only this lambda function needs to exist, not the actual backend. This +allows dynamically creating the backend objects when it is needed. -The ```LoadedNetwork``` also calls the ```CreateWorkloadFactory()``` -function for each created backend once and uses the returned backend -specific workload factory object to create the workloads for the -layers. The ```LoadedNetwork``` holds the workload factory objects for -its lifetime. +The BackendRegistry has a few convenience functions, like we can query the registered backends and + are able to tell if a given backend is registered or not. -Examples for this concept can be found in the [RefBackend header](reference/RefBackend.hpp) and the [RefBackend implementation](reference/RefBackend.cpp). +## The ILayerSupport interface -### The LayerSupportRegistry and the ILayerSupport object +ArmNN uses the [ILayerSupport](../../include/armnn/ILayerSupport.hpp) interface to decide if a layer +with a set of parameters (i.e. input and output tensors, descriptor, weights, filter, kernel if any) are +supported on a given backend. The backends need a way to communicate this information by implementing +the ```GetLayerSupport()``` function on the ```IBackendInternal``` interface. -ArmNN uses the [ILayerSupport](../../include/armnn/ILayerSupport.hpp) -interface to decide if a layer with a set of parameters (ie. input and -output tensors, descriptor, weights, filter, kernel if any) are -supported on a given backend. The backends need a way to communicate -this information. +Examples of this can be found in the [RefLayerSupport header](reference/RefLayerSupport.hpp) +and the [RefLayerSupport implementation](reference/RefLayerSupport.cpp). -In order to achieve this, the backends need to register a factory function -that can create an object that implements the ILayerSupport interface. -When ArmNN needs to decide if a layer is supported on a backend, it -looks up the factory function through the registry. Then it creates an -ILayerSupport object for the given backend and calls the corresponding -API function to check if the layer is supported. +## The IWorkloadFactory interface -Examples of this can be found in the [RefLayerSupport header](reference/RefLayerSupport.hpp) -and the [RefLayerSupport implementation](reference/RefLayerSupport.cpp). \ No newline at end of file +The [IWorkloadFactory interface](backendsCommon/WorkloadFactory.hpp) is used for creating the backend +specific workloads. The factory function that creates the IWorkloadFactory object in the IBackendInterface +takes an IMemoryManager object. + +To create a workload object the ```IWorkloadFactory``` takes a ```WorkloadInfo``` object that holds +the input and output tensor information and a workload specific queue descriptor. + +## The IMemoryManager interface + +Backends may choose to implement custom memory management. ArmNN supports this concept through this mechanism: + +* the ```IBackendInternal``` interface has a ```CreateMemoryManager()``` function which is called before + creating the workload factory +* the memory manager is passed to the ```CreateWorkloadFactory(...)``` function so the workload factory can + use it for creating the backend specific workloads +* the LoadedNetwork calls ```Acquire()``` on the memory manager before it starts executing the network and + it calls ```Release()``` in its destructor + +## The Optimizations + +The backends may choose to implement backend specific optimizations. This is supported through the ```GetOptimizations()``` +method of the IBackendInternal interface. This function may return a vector of optimization objects and the optimizer +runs these after all general optimization is performed on the network. + +## The IBackendContext interface + +Backends may need to be notified when a network is loaded or unloaded. To support that one can implement the optional +[IBackendContext](backendsCommon/IBackendContext.hpp) interface. The framework calls the ```CreateBackendContext(...)``` +method for each backend in the Runtime. If the backend returns a valid unique pointer to a backend context, then the +runtime will hold this for its entire lifetime. It then calls these interface functions for each stored context: + +* ```BeforeLoadNetwork(NetworkId networkId)``` +* ```AfterLoadNetwork(NetworkId networkId)``` +* ```BeforeUnloadNetwork(NetworkId networkId)``` +* ```AfterUnloadNetwork(NetworkId networkId)``` -- cgit v1.2.1