22.05
|
#include <IRuntime.hpp>
Classes | |
struct | CreationOptions |
Public Member Functions | |
Status | LoadNetwork (NetworkId &networkIdOut, IOptimizedNetworkPtr network) |
Loads a complete network into the IRuntime. More... | |
Status | LoadNetwork (NetworkId &networkIdOut, IOptimizedNetworkPtr network, std::string &errorMessage) |
Load a complete network into the IRuntime. More... | |
Status | LoadNetwork (NetworkId &networkIdOut, IOptimizedNetworkPtr network, std::string &errorMessage, const INetworkProperties &networkProperties) |
TensorInfo | GetInputTensorInfo (NetworkId networkId, LayerBindingId layerId) const |
TensorInfo | GetOutputTensorInfo (NetworkId networkId, LayerBindingId layerId) const |
std::vector< ImportedInputId > | ImportInputs (NetworkId networkId, const InputTensors &inputTensors, MemorySource forceImportMemorySource=MemorySource::Undefined) |
ImportInputs separates the importing and mapping of InputTensors from network execution. More... | |
std::vector< ImportedOutputId > | ImportOutputs (NetworkId networkId, const OutputTensors &outputTensors, MemorySource forceImportMemorySource=MemorySource::Undefined) |
ImportOutputs separates the importing and mapping of OutputTensors from network execution. More... | |
void | ClearImportedInputs (NetworkId networkId, const std::vector< ImportedInputId > inputIds) |
Un-import and delete the imported InputTensor/s This function is not thread safe and must not be used while other threads are calling Execute(). More... | |
void | ClearImportedOutputs (NetworkId networkId, const std::vector< ImportedOutputId > outputIds) |
Un-import and delete the imported OutputTensor/s This function is not thread safe and must not be used while other threads are calling Execute(). More... | |
Status | EnqueueWorkload (NetworkId networkId, const InputTensors &inputTensors, const OutputTensors &outputTensors, std::vector< ImportedInputId > preImportedInputIds={}, std::vector< ImportedOutputId > preImportedOutputIds={}) |
Evaluates a network using input in inputTensors and outputs filled into outputTensors. More... | |
Status | Execute (IWorkingMemHandle &workingMemHandle, const InputTensors &inputTensors, const OutputTensors &outputTensors, std::vector< ImportedInputId > preImportedInputs={}, std::vector< ImportedOutputId > preImportedOutputs={}) |
This is an experimental function. More... | |
Status | UnloadNetwork (NetworkId networkId) |
Unloads a network from the IRuntime. More... | |
const IDeviceSpec & | GetDeviceSpec () const |
std::unique_ptr< IWorkingMemHandle > | CreateWorkingMemHandle (NetworkId networkId) |
Create a new unique WorkingMemHandle object. More... | |
const std::shared_ptr< IProfiler > | GetProfiler (NetworkId networkId) const |
Gets the profiler corresponding to the given network id. More... | |
void | RegisterDebugCallback (NetworkId networkId, const DebugCallbackFunction &func) |
Registers a callback function to debug layers performing custom computations on intermediate tensors. More... | |
Static Public Member Functions | |
static IRuntime * | CreateRaw (const CreationOptions &options) |
static IRuntimePtr | Create (const CreationOptions &options) |
static void | Destroy (IRuntime *runtime) |
Protected Member Functions | |
IRuntime () | |
IRuntime (const IRuntime::CreationOptions &options) | |
~IRuntime () | |
Protected Attributes | |
std::unique_ptr< RuntimeImpl > | pRuntimeImpl |
Definition at line 74 of file IRuntime.hpp.
|
protected |
Definition at line 38 of file Runtime.cpp.
Referenced by IRuntime::CreateRaw().
|
protected |
Definition at line 40 of file Runtime.cpp.
References IRuntime::~IRuntime().
|
protecteddefault |
Referenced by IRuntime::IRuntime().
void ClearImportedInputs | ( | NetworkId | networkId, |
const std::vector< ImportedInputId > | inputIds | ||
) |
Un-import and delete the imported InputTensor/s This function is not thread safe and must not be used while other threads are calling Execute().
Only compatible with AsyncEnabled networks
Definition at line 101 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
void ClearImportedOutputs | ( | NetworkId | networkId, |
const std::vector< ImportedOutputId > | outputIds | ||
) |
Un-import and delete the imported OutputTensor/s This function is not thread safe and must not be used while other threads are calling Execute().
Only compatible with AsyncEnabled networks
Definition at line 105 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
|
static |
Definition at line 49 of file Runtime.cpp.
References IRuntime::CreateRaw(), and IRuntime::Destroy().
Referenced by armnn::experimental::AsyncEndToEndTestImpl(), armnn::experimental::AsyncThreadedEndToEndTestImpl(), GetSoftmaxProfilerJson(), InferenceModel< IParser, TDataType >::InferenceModel(), main(), TEST_CASE_FIXTURE(), and TEST_SUITE().
|
static |
Definition at line 44 of file Runtime.cpp.
References IRuntime::IRuntime().
Referenced by IRuntime::Create().
std::unique_ptr< IWorkingMemHandle > CreateWorkingMemHandle | ( | NetworkId | networkId | ) |
Create a new unique WorkingMemHandle object.
Create multiple handles if you wish to have overlapped Execution by calling this function from different threads.
Definition at line 143 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
|
static |
Definition at line 54 of file Runtime.cpp.
Referenced by IRuntime::Create().
Status EnqueueWorkload | ( | NetworkId | networkId, |
const InputTensors & | inputTensors, | ||
const OutputTensors & | outputTensors, | ||
std::vector< ImportedInputId > | preImportedInputIds = {} , |
||
std::vector< ImportedOutputId > | preImportedOutputIds = {} |
||
) |
Evaluates a network using input in inputTensors and outputs filled into outputTensors.
Definition at line 110 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
Status Execute | ( | IWorkingMemHandle & | workingMemHandle, |
const InputTensors & | inputTensors, | ||
const OutputTensors & | outputTensors, | ||
std::vector< ImportedInputId > | preImportedInputs = {} , |
||
std::vector< ImportedOutputId > | preImportedOutputs = {} |
||
) |
This is an experimental function.
Evaluates a network using input in inputTensors and outputs filled into outputTensors. This function performs a thread safe execution of the network. Returns once execution is complete. Will block until this and any other thread using the same workingMem object completes.
Definition at line 120 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
Referenced by Threadpool::TerminateThreadPool().
const IDeviceSpec & GetDeviceSpec | ( | ) | const |
Definition at line 138 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
armnn::TensorInfo GetInputTensorInfo | ( | NetworkId | networkId, |
LayerBindingId | layerId | ||
) | const |
Definition at line 79 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
armnn::TensorInfo GetOutputTensorInfo | ( | NetworkId | networkId, |
LayerBindingId | layerId | ||
) | const |
Definition at line 84 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
Gets the profiler corresponding to the given network id.
networkId | The id of the network for which to get the profile. |
Definition at line 148 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
std::vector< ImportedInputId > ImportInputs | ( | NetworkId | networkId, |
const InputTensors & | inputTensors, | ||
MemorySource | forceImportMemorySource = MemorySource::Undefined |
||
) |
ImportInputs separates the importing and mapping of InputTensors from network execution.
Allowing for a set of InputTensors to be imported and mapped once, but used in execution many times. This function is not thread safe and must not be used while other threads are calling Execute(). Only compatible with AsyncEnabled networks and aligned memory import
Definition at line 89 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
std::vector< ImportedOutputId > ImportOutputs | ( | NetworkId | networkId, |
const OutputTensors & | outputTensors, | ||
MemorySource | forceImportMemorySource = MemorySource::Undefined |
||
) |
ImportOutputs separates the importing and mapping of OutputTensors from network execution.
Allowing for a set of OutputTensors to be imported and mapped once, but used in execution many times. This function is not thread safe and must not be used while other threads are calling Execute(). Only compatible with AsyncEnabled networks and aligned memory import
Definition at line 95 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
Status LoadNetwork | ( | NetworkId & | networkIdOut, |
IOptimizedNetworkPtr | network | ||
) |
Loads a complete network into the IRuntime.
[out] | networkIdOut | - Unique identifier for the network is returned in this reference. |
[in] | network | - Complete network to load into the IRuntime. The runtime takes ownership of the network once passed in. |
Definition at line 59 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
Referenced by RuntimeImpl::LoadNetwork().
Status LoadNetwork | ( | NetworkId & | networkIdOut, |
IOptimizedNetworkPtr | network, | ||
std::string & | errorMessage | ||
) |
Load a complete network into the IRuntime.
[out] | networkIdOut | Unique identifier for the network is returned in this reference. |
[in] | network | Complete network to load into the IRuntime. |
[out] | errorMessage | Error message if there were any errors. The runtime takes ownership of the network once passed in. |
Definition at line 64 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
Status LoadNetwork | ( | NetworkId & | networkIdOut, |
IOptimizedNetworkPtr | network, | ||
std::string & | errorMessage, | ||
const INetworkProperties & | networkProperties | ||
) |
Definition at line 71 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
void RegisterDebugCallback | ( | NetworkId | networkId, |
const DebugCallbackFunction & | func | ||
) |
Registers a callback function to debug layers performing custom computations on intermediate tensors.
networkId | The id of the network to register the callback. |
func | callback function to pass to the debug layer. |
Definition at line 153 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
Unloads a network from the IRuntime.
At the moment this only removes the network from the m_Impl->m_Network. This might need more work in the future to be AndroidNN compliant.
[in] | networkId | - Unique identifier for the network to be unloaded. Generated in LoadNetwork(). |
Definition at line 133 of file Runtime.cpp.
References IRuntime::pRuntimeImpl.
|
protected |
Definition at line 290 of file IRuntime.hpp.
Referenced by IRuntime::ClearImportedInputs(), IRuntime::ClearImportedOutputs(), IRuntime::CreateWorkingMemHandle(), IRuntime::EnqueueWorkload(), IRuntime::Execute(), IRuntime::GetDeviceSpec(), IRuntime::GetInputTensorInfo(), IRuntime::GetOutputTensorInfo(), IRuntime::GetProfiler(), IRuntime::ImportInputs(), IRuntime::ImportOutputs(), IRuntime::LoadNetwork(), IRuntime::RegisterDebugCallback(), and IRuntime::UnloadNetwork().