OptimizerOptions Struct Reference

ArmNN performs an optimization on each model/network before it gets loaded for execution. More...

#include <INetwork.hpp>

Public Member Functions
	OptimizerOptions ()
	OptimizerOptions (bool reduceFp32ToFp16, bool debug, bool reduceFp32ToBf16, bool importEnabled, ModelOptions modelOptions={}, bool exportEnabled=false, bool debugToFile=false)
	OptimizerOptions (bool reduceFp32ToFp16, bool debug, bool reduceFp32ToBf16=false, ShapeInferenceMethod shapeInferenceMethod=armnn::ShapeInferenceMethod::ValidateOnly, bool importEnabled=false, ModelOptions modelOptions={}, bool exportEnabled=false, bool debugToFile=false, bool allowExpandedDims=false)
const std::string	ToString () const

Public Attributes
bool	m_ReduceFp32ToFp16
	Reduces all Fp32 operators in the model to Fp16 for faster processing. More...
bool	m_Debug
bool	m_DebugToFile
bool	m_ReduceFp32ToBf16
	Reduces all Fp32 operators in the model to Bf16 for faster processing. More...
ShapeInferenceMethod	m_shapeInferenceMethod
bool	m_ImportEnabled
ModelOptions	m_ModelOptions
bool	m_ProfilingEnabled
bool	m_ExportEnabled
bool	m_AllowExpandedDims

Detailed Description

ArmNN performs an optimization on each model/network before it gets loaded for execution.

OptimizerOptions provides a set of features that allows the user to customize this optimization on a per model basis.

Examples:

CustomMemoryAllocatorSample.cpp.

Definition at line 127 of file INetwork.hpp.

Constructor & Destructor Documentation

OptimizerOptions() [1/3]

OptimizerOptions ( )

inline

Definition at line 129 of file INetwork.hpp.

        : m_ReduceFp32ToFp16(false)
        , m_Debug(false)
        , m_DebugToFile(false)
        , m_ReduceFp32ToBf16(false)
        , m_shapeInferenceMethod(armnn::ShapeInferenceMethod::ValidateOnly)
        , m_ImportEnabled(false)
        , m_ModelOptions()
        , m_ProfilingEnabled(false)
        , m_ExportEnabled(false)
        , m_AllowExpandedDims(false)
    {}

OptimizerOptions() [2/3]

OptimizerOptions ( bool  reduceFp32ToFp16, bool  debug, bool  reduceFp32ToBf16, bool  importEnabled, ModelOptions  modelOptions = {}, bool  exportEnabled = false, bool  debugToFile = false  )

inline

Definition at line 142 of file INetwork.hpp.

References armnn::ValidateOnly.

                                                 {}, bool exportEnabled = false, bool debugToFile = false)
        : m_ReduceFp32ToFp16(reduceFp32ToFp16)
        , m_Debug(debug)
        , m_DebugToFile(debugToFile)
        , m_ReduceFp32ToBf16(reduceFp32ToBf16)
        , m_shapeInferenceMethod(armnn::ShapeInferenceMethod::ValidateOnly)
        , m_ImportEnabled(importEnabled)
        , m_ModelOptions(modelOptions)
        , m_ProfilingEnabled(false)
        , m_ExportEnabled(exportEnabled)
        , m_AllowExpandedDims(false)
    {
        if (m_ReduceFp32ToFp16 && m_ReduceFp32ToBf16)
        {
            throw InvalidArgumentException("BFloat16 and Float16 optimization cannot be enabled at the same time.");
        }
    }

OptimizerOptions() [3/3]

OptimizerOptions ( bool  reduceFp32ToFp16, bool  debug, bool  reduceFp32ToBf16 = false, ShapeInferenceMethod  shapeInferenceMethod = armnn::ShapeInferenceMethod::ValidateOnly, bool  importEnabled = false, ModelOptions  modelOptions = {}, bool  exportEnabled = false, bool  debugToFile = false, bool  allowExpandedDims = false  )

inline

Definition at line 161 of file INetwork.hpp.

                                                                             {}, bool exportEnabled = false,
                     bool debugToFile = false, bool allowExpandedDims = false)
        : m_ReduceFp32ToFp16(reduceFp32ToFp16)
        , m_Debug(debug)
        , m_DebugToFile(debugToFile)
        , m_ReduceFp32ToBf16(reduceFp32ToBf16)
        , m_shapeInferenceMethod(shapeInferenceMethod)
        , m_ImportEnabled(importEnabled)
        , m_ModelOptions(modelOptions)
        , m_ProfilingEnabled(false)
        , m_ExportEnabled(exportEnabled)
        , m_AllowExpandedDims(allowExpandedDims)
    {
        if (m_ReduceFp32ToFp16 && m_ReduceFp32ToBf16)
        {
            throw InvalidArgumentException("BFloat16 and Float16 optimization cannot be enabled at the same time.");
        }
    }

Member Function Documentation

ToString()

const std::string ToString ( ) const

inline

Definition at line 182 of file INetwork.hpp.

References BackendOptions::BackendOption::GetName(), BackendOptions::BackendOption::GetValue(), BackendOptions::Var::ToString(), and armnn::ValidateOnly.

Referenced by armnn::Optimize().

    {
        std::stringstream stream;
        stream << "OptimizerOptions: \n";
        stream << "\tReduceFp32ToFp16: " << m_ReduceFp32ToFp16 << "\n";
        stream << "\tReduceFp32ToBf16: " << m_ReduceFp32ToBf16 << "\n";
        stream << "\tDebug: " << m_Debug << "\n";
        stream << "\tDebug to file: " << m_DebugToFile << "\n";
        stream << "\tShapeInferenceMethod: " <<
        (m_shapeInferenceMethod == ShapeInferenceMethod::ValidateOnly ? "ValidateOnly" : "InferAndValidate") << "\n";
        stream << "\tImportEnabled: " << m_ImportEnabled << "\n";
        stream << "\tExportEnabled: " << m_ExportEnabled << "\n";
        stream << "\tProfilingEnabled: " << m_ProfilingEnabled << "\n";
        stream << "\tAllowExpandedDims: " << m_AllowExpandedDims << "\n";

        stream << "\tModelOptions: \n";
        for (auto optionsGroup : m_ModelOptions)
        {
            for (size_t i=0; i < optionsGroup.GetOptionCount(); i++)
            {
                const armnn::BackendOptions::BackendOption option = optionsGroup.GetOption(i);
                stream << "\t\tBackend: "  << optionsGroup.GetBackendId() << "\n"
                       << "\t\t\tOption: " << option.GetName() << "\n"
                       << "\t\t\tValue: "  << std::string(option.GetValue().ToString()) << "\n";
            }
        }

        return stream.str();
    }

Member Data Documentation

m_AllowExpandedDims

bool m_AllowExpandedDims

Definition at line 248 of file INetwork.hpp.

m_Debug

bool m_Debug

Definition at line 220 of file INetwork.hpp.

Referenced by armnn::Optimize().

m_DebugToFile

bool m_DebugToFile

Definition at line 223 of file INetwork.hpp.

Referenced by armnn::Optimize().

m_ExportEnabled

bool m_ExportEnabled

Definition at line 245 of file INetwork.hpp.

Referenced by armnn::Optimize().

m_ImportEnabled

bool m_ImportEnabled

Examples:

CustomMemoryAllocatorSample.cpp.

Definition at line 236 of file INetwork.hpp.

Referenced by armnn::Optimize().

m_ModelOptions

ModelOptions m_ModelOptions

Definition at line 239 of file INetwork.hpp.

Referenced by armnn::Optimize(), ArmnnDriverImpl::PrepareArmnnModel(), and ArmnnDriverImpl::PrepareArmnnModelFromCache().

m_ProfilingEnabled

bool m_ProfilingEnabled

Definition at line 242 of file INetwork.hpp.

Referenced by armnn::Optimize(), ArmnnDriverImpl::PrepareArmnnModel(), and ArmnnDriverImpl::PrepareArmnnModelFromCache().

m_ReduceFp32ToBf16

bool m_ReduceFp32ToBf16

Reduces all Fp32 operators in the model to Bf16 for faster processing.

This feature works best if all operators of the model are in Fp32. ArmNN will add conversion layers between layers that weren't in Fp32 in the first place or if the operator is not supported in Bf16. The overhead of these conversions can lead to a slower overall performance if too many conversions are required.

Definition at line 230 of file INetwork.hpp.

Referenced by armnn::Optimize().

m_ReduceFp32ToFp16

bool m_ReduceFp32ToFp16

Reduces all Fp32 operators in the model to Fp16 for faster processing.

This feature works best if all operators of the model are in Fp32. ArmNN will add conversion layers between layers that weren't in Fp32 in the first place or if the operator is not supported in Fp16. The overhead of these conversions can lead to a slower overall performance if too many conversions are required.

Definition at line 217 of file INetwork.hpp.

Referenced by armnn::Optimize(), ArmnnDriverImpl::PrepareArmnnModel(), and ArmnnDriverImpl::PrepareArmnnModelFromCache().

m_shapeInferenceMethod

ShapeInferenceMethod m_shapeInferenceMethod

Definition at line 233 of file INetwork.hpp.

Referenced by armnn::Optimize().

---

The documentation for this struct was generated from the following file:

• include/armnn/INetwork.hpp