ArmNN test example apply in SP7021

Owned by akira.yu
Oct 12, 2020
10 min read

ArmNN ENV variable set:

export BASEDIR=~/armnn-pi

export PATH=$BASEDIR/boost.build/bin:$PATH

export PATH=$BASEDIR/protobuf-host/bin:$PATH

export LD_LIBRARY_PATH=$BASEDIR/protobuf-host/lib:$LD_LIBRARY_PATH

export LD_LIBRARY_PATH=$BASEDIR/armnn/build:$LD_LIBRARY_PATH

export ARMNN_INCLUDE=$BASEDIR/armnn/include

export ARMNN_LIB=$BASEDIR/armnn/build


  1. Arm Compute Library

It is in $BASEDIR/ComputeLibrary
It's library built is in $BASEDIR/ComputeLibrary/build

    1. Running a DNN with random weights and inputs

Arm compute library comes with examples for most common DNN architectures like: AlexNet, MobileNet, ResNet, Inception v3, Inception v4, Squeezenet, etc.
All available examples source code can be found in this example location: $BASEDIR/ComputeLibrary/example .
All available examples can be found in this example build location:
$BASEDIR/ComputeLibrary/build/example
Each model architecture can be tested with graph_[dnn_model] application. For example, to run the MobileNet v2 DNN model with random weights, run the example application without any argument:

Export LD_LIBRARY_PATH=$BASEDIR/ComputeLibrary/build:$LD_LIBRARY_PATH

./graph_mobilenet_v2

The application will create the DNN with random weights and run it with random inputs. If inference finishes, the "Test passed" message should be printed.

1.2 Running AlexNet using graph API
Download the archive file to the example location folder from this link:
cd $BASEDIR/ComputeLibrary/build/examples
_curl -L -o computer_library_alexnet.zip

https://developer.arm.com/-/media/Arm%20Developer%20Community/Images/Tutorial%20Guide%20Diagrams%20and%20Screenshots/Machine%20Learning/Running%20AlexNet%20on%20Pi%20with%20Compute%20Library/compute_library_alexnet.zip?revision=c1a232fa-f328-451f-9bd6-250b83511e01_


Create a new sub-folder and unzip the file.
mkdir assets_alexnet
unzip compute_library_alexnet.zip -d assets_alexnet
Set environment variables for execution:
export LD_LIBRARY_PATH=$BASEDIR/ComputeLibrary/build/examples:$LD_LIBRARY_PATH
export PATH_ASSETS=$BASEDIR/ComputeLibrary/build/examples/assets_alexnet
Run the program.
./graph_alexnet --data=$PATH_ASSETS --image=$PATH_ASSETS/go_kart.ppm --labels=$PATH_ASSETS/labels.txt

  1. Running Arm NN tests

    Note that input data, model configurations, and model weights are not distributed with Arm NN. The user must download them separately and make sure they are available on the device before running the tests. However, Arm NN tests don't come with an documentation. Input file names are hardcoded, so investigate the code to find out what input file names are expected.
    Following sections provide details about how to prepare the input data and how to run Arm NN tests. All of them use well known neural network models, therefore, with only few exceptions, such pre-trained networks are available freely on the Internet. Input images, models, formats and their content was deduced using code analysis.
    Arm NN test source code is in $BASEDIR/armnn/tests
    Arm NN test binary code is in $BASEDIR/armnn/build/tests
    General work-flow is first to prepare data on a host machine and then to deploy it on the board, where the actual Arm NN tests will be run. The following sections assume that neural network model files are stored in a folder called models and input image files are stored in a folder called data. Both of them are created inside a folder called ArmnnTests. Create this folder structure on the larger partition using the following commands:
    mkdir ArmnnTests
    cd ArmnnTests
    mkdir data
    mkdir models
    export PATH=$BASEDIR/armnn/build/tests:$PATH
    2.1 Caffe tests
    Arm NN SDK provides the following set of tests for Caffe models:
    pi@raspberrypi:~/armnn-pi/armnn/build/tests$ ls -l

grep "Caff"
-rwxr-xr-x 1 pi pi 915248 Sep 2 05:51 CaffeAlexNet-Armnn
-rwxr-xr-x 1 pi pi 725688 Sep 2 05:23 CaffeCifar10AcrossChannels-Armnn
-rwxr-xr-x 1 pi pi 915252 Sep 2 05:41 CaffeInception_BN-Armnn
-rwxr-xr-x 1 pi pi 729920 Sep 2 05:49 CaffeMnist-Armnn
-rwxr-xr-x 1 pi pi 915236 Sep 2 05:15 CaffeResNet-Armnn
-rwxr-xr-x 1 pi pi 915248 Sep 2 05:33 CaffeVGG-Armnn
-rwxr-xr-x 1 pi pi 920148 Sep 2 05:43 CaffeYolo-Armnn
Two important limitations might require preprocessing of the Caffe model file prior to running an Arm NN Caffe test.
First, Arm NN tests require batch size to be set to 1.
Second, Arm NN does not support all Caffe syntaxes, therefore some older neural network model files will require updates to the latest Caffe syntax.
For example, if a Caffe model has a batch size different from one or uses an older Caffe version defined by files model_name.prototxt and model_name.caffemodel, create a copy of the .prototxt file (new_model_name.prototxt), modify this file to use the new Caffe syntax and change the batch size to 1 and finally run the following python script:
import caffe
net = caffe.Net('model_name.prototxt', 'model_name.caffemodel', caffe.TEST)
new_net = caffe.Net('new_model_name.prototxt', 'model_name.caffemodel', caffe.TEST)
new_net.save('new_model_name.caffemodel')

      1. CaffeAlexNet-Armnn
  2. Use A linux host with py-caffe installed

    Download the model files:
    cd ~/ArmnnTests
    curl -L -o deploy.prototxt https://raw.githubusercontent.com/BVLC/caffe/master/models/bvlc_alexnet/deploy.prototxt
    curl -L -o bvlc_alexnet.caffemodel http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel
    cp deploy.prototxt bvlc_alexnet_1.prototxt
    nano bvlc_alexnet_1.prototxt

name: "AlexNet"

 

 

layer {

 

 

name: "data"

 

 

type: "Input"

 

 

top: "data"

 

 

input_param { shape: { dim: 1 dim: 3 dim: 227 dim: 227 } }

 

 

 

 

 

 

 

 

name: "AlexNet"

 

 

layer {

 

 

name: "data"

 

 

type: "Input"

 

 

top: "data"

 

 

input_param { shape: { dim: 10 dim: 3 dim: 227 dim: 227 } }

 

 

 

 

 

 

 

 

change the batch size to 1







Run the following python script to transform the network
python3
import caffe
net = caffe.Net('deploy.prototxt', 'bvlc_alexnet.caffemodel', caffe.TEST)
new_net = caffe.Net('bvlc_alexnet_1.prototxt', 'bvlc_alexnet.caffemodel', caffe.TEST)
new_net.save('bvlc_alexnet_1.caffemodel')




Copy bvlc_alexnet_1.caffemodel from linux host to ~/ArmnnTests/models in SP7021

  1. Find a .jpg file containing a shark (great white shark). Rename it to shark.jpg and copy it to the data folder on the device.
  2. Run the test

    CaffeAlexNet-Armnn --data-dir=data --model-dir=models

      1. CaffeInception_BN-Armnn
  3. Use A linux host with py-caffe installed

    Download the model files:
    cd ~/ArmnnTests
    curl -L -o deploy.prototxt https://raw.githubusercontent.com/pertusa/InceptionBN-21K-for-Caffe/master/deploy.prototxt
    curl -L -o Inception21k.caffemodel http://www.dlsi.ua.es/~pertusa/deep/Inception21k.caffemodel
    cp deploy.prototxt Inception-BN-batchsize1.prototxt
    nano Inception-BN-batchsize1.prototxt

    name: "Inception21k"

     

     

    layer {

     

     

    name: "data"

     

     

    type: "Input"

     

     

    top: "data"

     

     

    input_param { shape: { dim: 1 dim: 3 dim: 224 dim: 224 } }

     

     

     

     

     

     

     

     
    change the batch size to 1

    name: "Inception21k"

     

     

    layer {

     

     

    name: "data"

     

     

    type: "Input"

     

     

    top: "data"

     

     

    input_param { shape: { dim: 10 dim: 3 dim: 224 dim: 224 } }

     

     

     

     

     

     

     

     







    Run the following python script to transform the network
    import caffe
    net = caffe.Net('deploy.prototxt', 'Inception21k.caffemodel', caffe.TEST)
    new_net = caffe.Net('Inception-BN-batchsize1.prototxt', 'Inception21k.caffemodel', caffe.TEST)
    new_net.save(' Inception-BN-batchsize1.caffemodel')python3





    Copy Inception-BN-batchsize1.caffemodel to ~/ArmnnTests/models in SP7021
  4. Find a .jpg file containing a shark (great white shark). Rename it to shark.jpg and copy it to the data folder on the SP7021.
  5. Run the test

    CaffeInception_BN-Armnn --data-dir=data --model-dir=models





    2.1.3 CaffeMnist-Armnn
  6. Use A linux host with py-caffe installed

    Download the model files:
    cd ~/ArmnnTests
    curl -L -o lenet.prototxt https://raw.githubusercontent.com/BVLC/caffe/master/examples/mnist/lenet.prototxt
    curl -L -o lenet_iter_9000_ori.caffemodel https://github.com/ARM-software/ML-examples/blob/master/armnn-mnist/model/lenet_iter_9000.caffemodel
    cp lenet.prototxt lenet_iter_9000.prototxt
    nano lenet_iter_9000.prototxt
    change the batch size to 1

    name: "LeNet"

     

     

    layer {

     

     

    name: "data"

     

     

    type: "Input"

     

     

    top: "data"

     

     

    input_param { shape: { dim: 1 dim: 1 dim: 28 dim: 28 } }

     

     

     

     

     

     

     

     

    name: "LeNet"

     

     

    layer {

     

     

    name: "data"

     

     

    type: "Input"

     

     

    top: "data"

     

     

    input_param { shape: { dim: 64 dim: 1 dim: 28 dim: 28 } }

     

     

     

     

     

     

     

     







    Run the following python script to transform the network
    python3
    import caffe
    net = caffe.Net(lenet.prototxt', lenet_iter_9000_ori.caffemodel', caffe.TEST)
    new_net = caffe.Net(' lenet_iter_9000.prototxt', lenet_iter_9000_ori.caffemodel', caffe.TEST)
    new_net.save(' lenet_iter_9000.caffemodel')



    |
    Copy lenet_iter_9000.caffemodel to ~/ArmnnTests/models in SP7021
  7. Find a .jpg file containing a shark (great white shark). Rename it to shark.jpg and copy it to the data folder on SP7021.
  8. Download the two archives below and unpack them:

curl -L -o t10k-images-idx3-ubyte.gz http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
curl -L -o t10k-labels-idx1-ubyte.gz http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz
gzip -d t10k-images-idx3-ubyte.gz
gzip -d t10k-labels-idx1-ubyte.gz

  1. Rename two files to be t10k-images.idx3-ubyte and t10k-labels.idx1-ubyte and copy files to the data folder on the device.

mv t10k-images-idx3-ubyte t10k-images.idx3-ubyte
mv t10k-labels-idx1-ubyte t10k-labels.idx1-ubyte
cp t10k-images-idx3-ubyte ./data/
cp t10k-labels-idx1-ubyte ./data/

  1. Run the test

CaffeMnist-Armnn --data-dir=data --model-dir=models

2.2 TensorFlow tests
Arm NN SDK provides the following set of tests for TensorFlow models:
pi@raspberrypi:~/armnn-pi/armnn/build/tests$ ls -l | grep "Tf"
-rwxr-xr-x 1 pi pi 725444 Sep 2 05:51 TfCifar10-Armnn
-rwxr-xr-x 1 pi pi 920116 Sep 2 05:58 TfInceptionV3-Armnn
-rwxr-xr-x 1 pi pi 729696 Sep 2 05:49 TfMnist-Armnn
-rwxr-xr-x 1 pi pi 920116 Sep 2 05:56 TfMobileNet-Armnn
-rwxr-xr-x 1 pi pi 914928 Sep 2 05:58 TfResNext-Armnn
2.2.1 TfInceptionV3-Armnn
1. Download the model files. Unzip and move file to action folder :
cd ~/ArmnnTests
curl -L -o inception_v3_2016_08_28_frozen.pb.tar.gz https://storage.googleapis.com/download.tensorflow.org/models/inception_v3_2016_08_28_frozen.pb.tar.gz
tar zxvf inception_v3_2016_08_28_frozen.pb.tar.gz
mv inception_v3_2016_08_28_frozen.pb ./models/

  1. Find a .jpg file containing a shark (great white shark). Rename it to shark.jpg and copy it to the data folder on the SP7021.
  2. Find a .jpg file containing a dog (labrador retriever). Rename it to Dog.jpg and copy it to the data folder on the SP7021.
  3. Find a .jpg file containing a cat (tiger cat). Rename it to Cat.jpg and copy it to the data folder on the SP7021.
  4. Run the test:

TfInceptionV3-Armnn --data-dir=data --model-dir=models

This is not an execution error. This occurs because the TfInceptionV3-Armnn test expects a specific type of dog, cat and shark to be found so if a different type/breed of these animals is passed to the test, it returns a case failed.
The expected inputs for this test are:

ID

Label

File name

208

Golden Retriever

Dog.jpg

283

Tiger Cat

Cat.jpg

3

White Shark

shark.jpg

The complete list of supported objects can be found in https://github.com/ARM-software/armnn/blob/branches/armnn_18_11/tests/TfLiteMobilenetQuantized-Armnn/labels.txt

      1. TfMnist-Armnn
  2. Download the model files and copy file to action folder :

cd ~/ArmnnTests
curl -L -o simple_mnist_tf.prototxt https://raw.githubusercontent.com/ARM-software/ML-examples/master/armnn-mnist/model/simple_mnist_tf.prototxt
cp simple_mnist_tf.prototxt ./models/

  1. Download the two archives below and unpack them:

curl -L -o t10k-images-idx3-ubyte.gz http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
curl -L -o t10k-labels-idx1-ubyte.gz http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz
gzip -d t10k-images-idx3-ubyte.gz
gzip -d t10k-labels-idx1-ubyte.gz

  1. Rename two files to be t10k-images.idx3-ubyte and t10k-labels.idx1-ubyte and copy files to the data folder on the SP7021.

mv t10k-images-idx3-ubyte t10k-images.idx3-ubyte
mv t10k-labels-idx1-ubyte t10k-labels.idx1-ubyte
cp t10k-images-idx3-ubyte ./data/
cp t10k-labels-idx1-ubyte ./data/

  1. Run the test:


      1. TfMobileNet-Armnn
  2. Download and unpack the model file:

cd ~/ArmnnTests
curl -L -o mobilenet_v1_1.0_224.tgz http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz
tar zxvf mobilenet_v1_1.0_224.tgz

  1. Copy mobilenet_v1_1.0_224_frozen.pb to the models folder on the SP7021.

cp mobilenet_v1_1.0_224_frozen.pb ./models/

  1. Find a .jpg file containing a shark (great white shark). Rename it to shark.jpg and copy it to the data folder on the SP7021.
  2. Find a .jpg file containing a dog (labrador retriever). Rename it to Dog.jpg and copy it to the data folder on the SP7021.
  3. Find a .jpg file containing a cat (tiger cat). Rename it to Cat.jpg and copy it to the data folder on the SP7021.
  4. Run the test:

TfMobileNet-Armnn --data-dir=data --model-dir=models

    1. TensorFlow Lite tests

Arm NN SDK provides the following test for TensorFlow Lite models:
pi@raspberrypi:~/ArmnnTests$ ls -l ~/armnn-pi/armnn/build/tests/ | grep "TfL"
-rwxr-xr-x 1 pi pi 919808 Sep 2 05:08 TfLiteInceptionV3Quantized-Armnn
-rwxr-xr-x 1 pi pi 919808 Sep 2 05:28 TfLiteInceptionV4Quantized-Armnn
-rwxr-xr-x 1 pi pi 919656 Sep 2 05:42 TfLiteMnasNet-Armnn
-rwxr-xr-x 1 pi pi 921120 Sep 2 05:29 TfLiteMobilenetQuantized-Armnn
-rwxr-xr-x 1 pi pi 919812 Sep 2 05:14 TfLiteMobileNetQuantizedSoftmax-Armnn
-rwxr-xr-x 1 pi pi 915588 Sep 2 05:20 TfLiteMobileNetSsd-Armnn
-rwxr-xr-x 1 pi pi 919808 Sep 2 05:15 TfLiteMobilenetV2Quantized-Armnn
-rwxr-xr-x 1 pi pi 919808 Sep 2 05:02 TfLiteResNetV2-50-Quantized-Armnn
-rwxr-xr-x 1 pi pi 919656 Sep 2 05:06 TfLiteResNetV2-Armnn
-rwxr-xr-x 1 pi pi 919800 Sep 2 05:42 TfLiteVGG16Quantized-Armnn
-rwxr-xr-x 1 pi pi 666068 Sep 2 05:23 TfLiteYoloV3Big-Armnn
2.3.1 TfLiteInceptionV3Quantized-Armn

  1. Download and unpack the model file:

cd ~/ArmnnTests
curl -L -o inception_v3_quant.tgz http://download.tensorflow.org/models/tflite_11_05_08/inception_v3_quant.tgz
tar zxvf inception_v3_quant.tgz

  1. Copy inception_v3_quant.tflite to the models folder on the SP7021

cp inception_v3_quant.tflite ./models/

  1. Find a .jpg file containing a shark (great white shark). Rename it to shark.jpg and copy it to the data folder on the SP7021.
  2. Find a .jpg file containing a dog (labrador retriever). Rename it to Dog.jpg and copy it to the data folder on the SP7021.
  3. Find a .jpg file containing a cat (tiger cat). Rename it to Cat.jpg and copy it to the data folder on the SP7021.
  4. Run the test:

TfLiteInceptionV3Quantized-Armnn --data-dir=data --model-dir=models

      1. TfLiteMnasNet-Armnn
  2. Download and unpack the model file:

cd ~/ArmnnTests
curl -L -o mnasnet_1.3_224_09_07_2018.tgz http://download.tensorflow.org/models/tflite/mnasnet_1.3_224_09_07_2018.tgz
tar zxvf mnasnet_1.3_224_09_07_2018.tgz

  1. Copy mnasnet_1.3_224/mnasnet_1.3_224.tflite to the models folder on the SP7021.

cp ./mnasnet_1.3_224/mnasnet_1.3_224.tflite ./models/

  1. Find a .jpg file containing a shark (great white shark). Rename it to shark.jpg and copy it to the data folder on the SP7021.
  2. Find a .jpg file containing a dog (labrador retriever). Rename it to Dog.jpg and copy it to the data folder on the SP7021.
  3. Find a .jpg file containing a cat (tiger cat). Rename it to Cat.jpg and copy it to the data folder on the SP7021.
  4. Run the test:

TfLiteMnasNet-Armnn --data-dir=data --model-dir=models

      1. TfLiteMobilenetQuantized-Armnn
  2. Download the model file:

cd ~/ArmnnTests
curl -L -o mobilenet_v1_1.0_224_quant.tgz http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz
tar zxvf mobilenet_v1_1.0_224_quant.tgz

  1. Copy mobilenet_v1_1.0_224_quant.tflite to the models folder on the SP7021.

cp mobilenet_v1_1.0_224_quant.tflite ./models/

  1. Find a .jpg file containing a shark (great white shark). Rename it to shark.jpg and copy it to the data folder on the SP7021.
  2. Find a .jpg file containing a dog (labrador retriever). Rename it to Dog.jpg and copy it to the data folder on the SP7021.
  3. Find a .jpg file containing a cat (tiger cat). Rename it to Cat.jpg and copy it to the data folder on the SP7021.
  4. Run the test:

TfLiteMobilenetQuantized-Armnn --data-dir=data --model-dir=models

      1. TfLiteMobilenetV2Quantized-Armnn
  2. Download the model file:

cd ~/ArmnnTests
curl -L -o mobilenet_v2_1.0_224_quant.tgz http://download.tensorflow.org/models/tflite_11_05_08/mobilenet_v2_1.0_224_quant.tgz
tar zxvf mobilenet_v2_1.0_224_quant.tgz

  1. Copy mobilenet_v1_1.0_224_quant.tflite to the models folder on the SP7021.

cp mobilenet_v2_1.0_224_quant.tflite ./models/

  1. Find a .jpg file containing a shark (great white shark). Rename it to shark.jpg and copy it to the data folder on the SP7021.
  2. Find a .jpg file containing a dog (labrador retriever). Rename it to Dog.jpg and copy it to the data folder on the SP7021.
  3. Find a .jpg file containing a cat (tiger cat). Rename it to Cat.jpg and copy it to the data folder on the SP7021.
  4. Run the test:

TfLiteMobilenetV2Quantized-Armnn --data-dir=data --model-dir=models

    1. ONNX tests

The Arm NN provides the following set of tests for ONNX models:
pi@raspberrypi:~/ArmnnTests$ ls -l ~/armnn-pi/armnn/build/tests/ | grep "Onn"
-rwxr-xr-x 1 pi pi 729136 Sep 2 05:08 OnnxMnist-Armnn
-rwxr-xr-x 1 pi pi 915132 Sep 2 05:01 OnnxMobileNet-Armnn
2.4.1 OnnxMnist-Armnn
1. Download and unpack the model file:
cd ~/ArmnnTests
curl -L -o mnist.tar.gz https://onnxzoo.blob.core.windows.net/models/opset_8/mnist/mnist.tar.gz
tar zxvf mnist.tar.gz

  1. Rename the model.onnx file to mnist_onnx.onnx and copy it to the models folder on the SP7021

mv ./mnist/model.onnx ./mnist/mnist_onnx.onnx
cp ./mnist/mnist_onnx.onnx ./models/

  1. Download the two archives below and unpack them:

curl -L -o t10k-images-idx3-ubyte.gz http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
curl -L -o t10k-labels-idx1-ubyte.gz http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz
gzip -d t10k-images-idx3-ubyte.gz
gzip -d t10k-labels-idx1-ubyte.gz

  1. Rename two files to be t10k-images.idx3-ubyte and t10k-labels.idx1-ubyte and copy files to the data folder on the SP7021.

mv t10k-images-idx3-ubyte t10k-images.idx3-ubyte
mv t10k-labels-idx1-ubyte t10k-labels.idx1-ubyte
cp t10k-images-idx3-ubyte ./data/
cp t10k-labels-idx1-ubyte ./data/

  1. Run the test:

OnnxMnist-Armnn --data-dir=data --model-dir=models

      1. OnnxMobileNet-Armnn
  2. Download and unpack the model file:

cd ~/ArmnnTests
curl -L -o mobilenetv2-1.0.tar.gz https://s3.amazonaws.com/onnx-model-zoo/mobilenet/mobilenetv2-1.0/mobilenetv2-1.0.tar.gz
tar zxvf mobilenetv2-1.0.tar.gz

  1. Copy the unpacked mobilenetv2-1.0.onnx file to the models folder on the SP7021

cp ./mobilenetv2-1.0/mobilenetv2-1.0.onnx ./models/

  1. Find a .jpg file containing a shark (great white shark). Rename it to shark.jpg and copy it to the data folder on the SP7021.
  2. Find a .jpg file containing a dog (labrador retriever). Rename it to Dog.jpg and copy it to the data folder on the SP7021.
  3. Find a .jpg file containing a cat (tiger cat). Rename it to Cat.jpg and copy it to the data folder on the SP7021.
  4. Run the test:

OnnxMobileNet-Armnn --data-dir=data --model-dir=models

3. Python interface to Arm NN (PyArmNN)
cd ~/armnn-pi/armnn/python/pyarmnn/examples/
python3 tflite_mobilenetv1_quantized.py

python3 onnx_mobilenetv2.py