OptimizerOptions Struct Reference

ArmNN performs an optimization on each model/network before it gets loaded for execution. More...

#include <INetwork.hpp>

Public Member Functions
	OptimizerOptions ()
	OptimizerOptions (bool reduceFp32ToFp16, bool debug, bool reduceFp32ToBf16, bool importEnabled, ModelOptions modelOptions={}, bool exportEnabled=false)
	OptimizerOptions (bool reduceFp32ToFp16, bool debug, bool reduceFp32ToBf16=false, ShapeInferenceMethod shapeInferenceMethod=armnn::ShapeInferenceMethod::ValidateOnly, bool importEnabled=false, ModelOptions modelOptions={}, bool exportEnabled=false)
const std::string	ToString () const

Public Attributes
bool	m_ReduceFp32ToFp16
	Reduces all Fp32 operators in the model to Fp16 for faster processing. More...
bool	m_Debug
bool	m_ReduceFp32ToBf16
	Reduces all Fp32 operators in the model to Bf16 for faster processing. More...
ShapeInferenceMethod	m_shapeInferenceMethod
bool	m_ImportEnabled
ModelOptions	m_ModelOptions
bool	m_ProfilingEnabled
bool	m_ExportEnabled

Detailed Description

ArmNN performs an optimization on each model/network before it gets loaded for execution.

OptimizerOptions provides a set of features that allows the user to customize this optimization on a per model basis.

Examples:

CustomMemoryAllocatorSample.cpp.

Definition at line 127 of file INetwork.hpp.

Constructor & Destructor Documentation

OptimizerOptions() [1/3]

OptimizerOptions ( )

inline

Definition at line 129 of file INetwork.hpp.

        : m_ReduceFp32ToFp16(false)
        , m_Debug(false)
        , m_ReduceFp32ToBf16(false)
        , m_shapeInferenceMethod(armnn::ShapeInferenceMethod::ValidateOnly)
        , m_ImportEnabled(false)
        , m_ModelOptions()
        , m_ProfilingEnabled(false)
        , m_ExportEnabled(false)
    {}

OptimizerOptions() [2/3]

OptimizerOptions ( bool  reduceFp32ToFp16, bool  debug, bool  reduceFp32ToBf16, bool  importEnabled, ModelOptions  modelOptions = {}, bool  exportEnabled = false  )

inline

Definition at line 140 of file INetwork.hpp.

References armnn::ValidateOnly.

                                                 {}, bool exportEnabled = false)
        : m_ReduceFp32ToFp16(reduceFp32ToFp16)
        , m_Debug(debug)
        , m_ReduceFp32ToBf16(reduceFp32ToBf16)
        , m_shapeInferenceMethod(armnn::ShapeInferenceMethod::ValidateOnly)
        , m_ImportEnabled(importEnabled)
        , m_ModelOptions(modelOptions)
        , m_ProfilingEnabled(false)
        , m_ExportEnabled(exportEnabled)
    {
        if (m_ReduceFp32ToFp16 && m_ReduceFp32ToBf16)
        {
            throw InvalidArgumentException("BFloat16 and Float16 optimization cannot be enabled at the same time.");
        }
    }

OptimizerOptions() [3/3]

OptimizerOptions ( bool  reduceFp32ToFp16, bool  debug, bool  reduceFp32ToBf16 = false, ShapeInferenceMethod  shapeInferenceMethod = armnn::ShapeInferenceMethod::ValidateOnly, bool  importEnabled = false, ModelOptions  modelOptions = {}, bool  exportEnabled = false  )

inline

Definition at line 157 of file INetwork.hpp.

                                                                             {}, bool exportEnabled = false)
        : m_ReduceFp32ToFp16(reduceFp32ToFp16)
        , m_Debug(debug)
        , m_ReduceFp32ToBf16(reduceFp32ToBf16)
        , m_shapeInferenceMethod(shapeInferenceMethod)
        , m_ImportEnabled(importEnabled)
        , m_ModelOptions(modelOptions)
        , m_ProfilingEnabled(false)
        , m_ExportEnabled(exportEnabled)
    {
        if (m_ReduceFp32ToFp16 && m_ReduceFp32ToBf16)
        {
            throw InvalidArgumentException("BFloat16 and Float16 optimization cannot be enabled at the same time.");
        }
    }

Member Function Documentation

ToString()

const std::string ToString ( ) const

inline

Definition at line 175 of file INetwork.hpp.

References BackendOptions::BackendOption::GetName(), BackendOptions::BackendOption::GetValue(), BackendOptions::Var::ToString(), and armnn::ValidateOnly.

Referenced by armnn::Optimize().

    {
        std::stringstream stream;
        stream << "OptimizerOptions: \n";
        stream << "\tReduceFp32ToFp16: " << m_ReduceFp32ToFp16 << "\n";
        stream << "\tReduceFp32ToBf16: " << m_ReduceFp32ToBf16 << "\n";
        stream << "\tDebug: " << m_Debug << "\n";
        stream << "\tShapeInferenceMethod: " <<
        (m_shapeInferenceMethod == ShapeInferenceMethod::ValidateOnly ? "ValidateOnly" : "InferAndValidate") << "\n";
        stream << "\tImportEnabled: " << m_ImportEnabled << "\n";
        stream << "\tExportEnabled: " << m_ExportEnabled << "\n";
        stream << "\tProfilingEnabled: " << m_ProfilingEnabled << "\n";

        stream << "\tModelOptions: \n";
        for (auto optionsGroup : m_ModelOptions)
        {
            for (size_t i=0; i < optionsGroup.GetOptionCount(); i++)
            {
                const armnn::BackendOptions::BackendOption option = optionsGroup.GetOption(i);
                stream << "\t\tBackend: "  << optionsGroup.GetBackendId() << "\n"
                       << "\t\t\tOption: " << option.GetName() << "\n"
                       << "\t\t\tValue: "  << std::string(option.GetValue().ToString()) << "\n";
            }
        }

        return stream.str();
    }

Member Data Documentation

m_Debug

bool m_Debug

Definition at line 211 of file INetwork.hpp.

Referenced by InferenceModel< IParser, TDataType >::InferenceModel(), armnn::Optimize(), ArmNNExecutor::PrintNetworkInfo(), TEST_SUITE(), and ExecuteNetworkParams::ValidateParams().

m_ExportEnabled

bool m_ExportEnabled

Definition at line 233 of file INetwork.hpp.

Referenced by armnn::Optimize(), and TEST_SUITE().

m_ImportEnabled

bool m_ImportEnabled

Examples:

CustomMemoryAllocatorSample.cpp.

Definition at line 224 of file INetwork.hpp.

Referenced by armnn::Optimize(), and TEST_SUITE().

m_ModelOptions

ModelOptions m_ModelOptions

Definition at line 227 of file INetwork.hpp.

Referenced by InferenceModel< IParser, TDataType >::InferenceModel(), armnn::Optimize(), ArmNNExecutor::PrintNetworkInfo(), TEST_CASE_FIXTURE(), TEST_SUITE(), and ExecuteNetworkParams::ValidateParams().

m_ProfilingEnabled

bool m_ProfilingEnabled

Definition at line 230 of file INetwork.hpp.

Referenced by GetSoftmaxProfilerJson(), InferenceModel< IParser, TDataType >::InferenceModel(), armnn::Optimize(), ArmNNExecutor::PrintNetworkInfo(), and ExecuteNetworkParams::ValidateParams().

m_ReduceFp32ToBf16

bool m_ReduceFp32ToBf16

Reduces all Fp32 operators in the model to Bf16 for faster processing.

This feature works best if all operators of the model are in Fp32. ArmNN will add conversion layers between layers that weren't in Fp32 in the first place or if the operator is not supported in Bf16. The overhead of these conversions can lead to a slower overall performance if too many conversions are required.

Definition at line 218 of file INetwork.hpp.

Referenced by InferenceModel< IParser, TDataType >::InferenceModel(), armnn::Optimize(), ArmNNExecutor::PrintNetworkInfo(), and ExecuteNetworkParams::ValidateParams().

m_ReduceFp32ToFp16

bool m_ReduceFp32ToFp16

Reduces all Fp32 operators in the model to Fp16 for faster processing.

This feature works best if all operators of the model are in Fp32. ArmNN will add conversion layers between layers that weren't in Fp32 in the first place or if the operator is not supported in Fp16. The overhead of these conversions can lead to a slower overall performance if too many conversions are required.

Definition at line 208 of file INetwork.hpp.

Referenced by InferenceModel< IParser, TDataType >::InferenceModel(), armnn::Optimize(), ArmNNExecutor::PrintNetworkInfo(), TEST_SUITE(), and ExecuteNetworkParams::ValidateParams().

m_shapeInferenceMethod

ShapeInferenceMethod m_shapeInferenceMethod

Definition at line 221 of file INetwork.hpp.

Referenced by InferenceModel< IParser, TDataType >::InferenceModel(), armnn::Optimize(), ArmNNExecutor::PrintNetworkInfo(), and ExecuteNetworkParams::ValidateParams().

---

The documentation for this struct was generated from the following file:

• include/armnn/INetwork.hpp