ArmNN
 20.08
Parsers

Table of Contents

ArmNN Caffe Parser

armnnCaffeParser is a library for loading neural networks defined in Caffe protobuf files into the Arm NN runtime.

Caffe layers supported by the Arm NN SDK

This reference guide provides a list of Caffe layers the Arm NN SDK currently supports.

Although some other neural networks might work, Arm tests the Arm NN SDK with Caffe implementations of the following neural networks:

  • AlexNet.
  • Cifar10.
  • Inception-BN.
  • Resnet_50, Resnet_101 and Resnet_152.
  • VGG_CNN_S, VGG_16 and VGG_19.
  • Yolov1_tiny.
  • Lenet.
  • MobileNetv1.

The Arm NN SDK supports the following machine learning layers for Caffe networks:

  • BatchNorm, in inference mode.
  • Convolution, excluding the Dilation Size, Weight Filler, Bias Filler, Engine, Force nd_im2col, and Axis parameters. Caffe doesn't support depthwise convolution, the equivalent layer is implemented through the notion of groups. ArmNN supports groups this way:
    • when group=1, it is a normal conv2d
    • when group=#input_channels, we can replace it by a depthwise convolution
    • when group>1 && group<#input_channels, we need to split the input into the given number of groups, apply a separate convolution and then merge the results
  • Concat, along the channel dimension only.
  • Dropout, in inference mode.
  • Element wise, excluding the coefficient parameter.
  • Inner Product, excluding the Weight Filler, Bias Filler, Engine, and Axis parameters.
  • Input.
  • Local Response Normalisation (LRN), excluding the Engine parameter.
  • Pooling, excluding the Stochastic Pooling and Engine parameters.
  • ReLU.
  • Scale.
  • Softmax, excluding the Axis and Engine parameters.
  • Split.

More machine learning layers will be supported in future releases.

Please note that certain deprecated Caffe features are not supported by the armnnCaffeParser. If you think that Arm NN should be able to load your model according to the list of supported layers, but you are getting strange error messages, then try upgrading your model to the latest format using Caffe, either by saving it to a new file or using the upgrade utilities in caffe/tools.



ArmNN Onnx Parser

armnnOnnxParser is a library for loading neural networks defined in ONNX protobuf files into the Arm NN runtime.

ONNX operators that the Arm NN SDK supports

This reference guide provides a list of ONNX operators the Arm NN SDK currently supports.

The Arm NN SDK ONNX parser currently only supports fp32 operators.

Fully supported

Partially supported

  • Conv
    • The parser only supports 2D convolutions with a dilation rate of [1, 1] and group = 1 or group = #Nb_of_channel (depthwise convolution) See the ONNX Conv documentation for more information.
  • BatchNormalization
  • MatMul
    • The parser only supports constant weights in a fully connected layer.

Tested networks

Arm tested these operators with the following ONNX fp32 neural networks:

More machine learning operators will be supported in future releases.



ArmNN Tf Lite Parser

armnnTfLiteParser is a library for loading neural networks defined by TensorFlow Lite FlatBuffers files into the Arm NN runtime.

TensorFlow Lite operators that the Arm NN SDK supports

This reference guide provides a list of TensorFlow Lite operators the Arm NN SDK currently supports.

Fully supported

The Arm NN SDK TensorFlow Lite parser currently supports the following operators:

  • ADD
  • AVERAGE_POOL_2D, Supported Fused Activation: RELU , RELU6 , TANH, NONE
  • BATCH_TO_SPACE
  • CONCATENATION, Supported Fused Activation: RELU , RELU6 , TANH, NONE
  • CONV_2D, Supported Fused Activation: RELU , RELU6 , TANH, NONE
  • DEPTHWISE_CONV_2D, Supported Fused Activation: RELU , RELU6 , TANH, NONE
  • DIV
  • EXP
  • FULLY_CONNECTED, Supported Fused Activation: RELU , RELU6 , TANH, NONE
  • LOGISTIC
  • L2_NORMALIZATION
  • LEAKY_RELU
  • MAX_POOL_2D, Supported Fused Activation: RELU , RELU6 , TANH, NONE
  • MAXIMUM
  • MEAN
  • MINIMUM
  • MUL
  • NEG
  • PACK
  • PAD
  • RELU
  • RELU6
  • RESHAPE
  • RESIZE_BILINEAR
  • SLICE
  • SOFTMAX
  • SPACE_TO_BATCH
  • SPLIT
  • SPLIT_V
  • SQUEEZE
  • STRIDED_SLICE
  • SUB
  • TANH
  • TRANSPOSE
  • TRANSPOSE_CONV
  • UNPACK

Custom Operator

  • TFLite_Detection_PostProcess

Tested networks

Arm tested these operators with the following TensorFlow Lite neural network:

More machine learning operators will be supported in future releases.



ArmNN Tensorflow Parser

armnnTfParser is a library for loading neural networks defined by TensorFlow protobuf files into the Arm NN runtime.

TensorFlow operators that the Arm NN SDK supports

This reference guide provides a list of TensorFlow operators the Arm NN SDK currently supports.

The Arm NN SDK TensorFlow parser currently only supports fp32 operators.

Fully supported

Partially supported

  • add
  • add_n
  • concat
    • Arm NN supports concatenation along the channel dimension for data formats NHWC and NCHW.
  • constant
    • The parser does not support the optional shape argument. It always infers the shape of the output tensor from value. See the TensorFlow constant documentation for further information.
  • depthwise_conv2d_native
  • equal
  • fused_batch_norm
  • greater
  • matmul
    • The parser only supports constant weights in a fully connected layer. See the TensorFlow matmul documentation for more information.
  • maximum where maximum is used in one of the following ways
    • max(mul(a, x), x)
    • max(mul(x, a), x)
    • max(x, mul(a, x))
    • max(x, mul(x, a) This is interpreted as a ActivationLayer with a LeakyRelu activation function. Any other usage of max will result in the insertion of a simple maximum layer. The parser does not support all forms of broadcast composition. See the TensorFlow maximum documentation for more information.
  • minimum
  • multiply
  • pad
    • Only supports tf.pad function with mode = 'CONSTANT' and constant_values = 0. See the TensorFlow pad documentation for more information.
  • realdiv
  • reshape
    • The parser does not support reshaping to or from 4D. See the TensorFlow reshape documentation for more information.
  • resize_images
    • The parser only supports ResizeMethod.BILINEAR with align_corners=False. See the TensorFlow resize_images documentation for more information.
  • softmax
    • The parser only supports 2D inputs and does not support selecting the softmax dimension. See the TensorFlow softmax documentation for more information.
  • split
    • Arm NN supports split along the channel dimension for data formats NHWC and NCHW.
  • subtract

Tested networks

Arm tests these operators with the following TensorFlow fp32 neural networks:

  • Lenet
  • mobilenet_v1_1.0_224. The Arm NN SDK only supports the non-quantized version of the network. See the https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet_v1.md "MobileNet_v1 documentation" for more information on quantized networks.
  • inception_v3. The Arm NN SDK only supports the official inception_v3 transformed model. See the TensorFlow documentation on preparing models for mobile deployment for more information on how to transform the inception_v3 network.

Using these datasets:

  • Cifar10
  • Simple MNIST. For more information check out the tutorial on the Arm Developer portal.

More machine learning operators will be supported in future releases.