From 6b857b8a245ceeedd70bc768784cf617db132d0d Mon Sep 17 00:00:00 2001 From: Kshitij Sisodia Date: Tue, 24 May 2022 17:46:55 +0100 Subject: MLECO-3185: Documentation updates Documentation updated to reflect refactoring changes in the last few weeks. Change-Id: Ic7abf5cc3af9898123049e890c189ed74e505309 --- Readme.md | 37 ++++- docs/documentation.md | 90 +++++++---- docs/media/APIs_description.png | Bin 10277 -> 99711 bytes docs/media/build_graph.png | Bin 93276 -> 98332 bytes docs/sections/building.md | 2 +- docs/sections/coding_guidelines.md | 2 +- docs/sections/customizing.md | 300 ++++++++++++++++++------------------- download_dependencies.py | 8 +- scripts/mps3/sse-310/images.txt | 27 ++-- 9 files changed, 251 insertions(+), 215 deletions(-) diff --git a/Readme.md b/Readme.md index 38942d0..4d6e03c 100644 --- a/Readme.md +++ b/Readme.md @@ -52,30 +52,53 @@ by displaying different performance metrics such as inference cycle count estima ## Software and hardware overview -The evaluation kit is based on the [Arm® Corstone™-300 reference package](https://developer.arm.com/ip-products/subsystem/corstone/corstone-300). -Arm® Corstone™-300 helps you build SoCs quickly on the Arm® Cortex™-M55 and Arm® Ethos™-U55 designs. Arm® Corstone™-300 design -implementation is publicly available on an [Arm MPS3 FPGA board](https://developer.arm.com/tools-and-software/development-boards/fpga-prototyping-boards/download-fpga-images), +The evaluation kit primarily supports [Arm® Corstone™-300](https://developer.arm.com/ip-products/subsystem/corstone/corstone-300) +and [Arm® Corstone™-310](https://developer.arm.com/ip-products/subsystem/corstone/corstone-310) reference packages as its +primary targets. Arm® Corstone™-300 design implementation is publicly available on an [Arm MPS3 FPGA board](https://developer.arm.com/tools-and-software/development-boards/fpga-prototyping-boards/download-fpga-images), or as a [Fixed Virtual Platform of the MPS3 development board](https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps). The Ethos-U NPU software stack is described [here](https://developer.arm.com/documentation/101888/0500/NPU-software-overview/NPU-software-components?lang=en). All ML use cases, albeit illustrating a different application, have common code such as initializing the Hardware -Abstraction Layer (HAL). The application common code can be run on x86 or Arm Cortex-M architecture thanks to the HAL. +Abstraction Layer (HAL). The application common code can be run on native host machine (x86_64 or aarch64) or Arm +Cortex-M architecture thanks to the HAL. For the ML application-specific part, Google® TensorFlow™ Lite for Microcontrollers inference engine is used to schedule the neural networks models executions. TensorFlow Lite for Microcontrollers is integrated with the [Ethos-U NPU driver](https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ethos-u-core-driver) and delegates execution of certain operators to the NPU or, if the neural network model operators are not supported on -NPU, to the CPU. [CMSIS-NN](https://github.com/ARM-software/CMSIS_5) is used to optimise CPU workload execution -with int8 data type. +NPU, to the CPU. If the operator is supported, [CMSIS-NN](https://github.com/ARM-software/CMSIS_5) is used to optimise +CPU workload execution with int8 data type. Else, TensorFlow™ Lite for Microcontrollers' reference kernels are used as +a final fall-back. Common ML application functions will help you to focus on implementing logic of your custom ML use case: you can modify only the use case code and leave all other components unchanged. Supplied build system will discover new ML application code and automatically include it into compilation flow. +A high level overview of the different components in the software, and the platforms supported out-of-the-box, is shown +in the diagram below. + ![APIs](docs/media/APIs_description.png) +For a more detailed description of the build graph with all major components, see [Building](./docs/documentation.md#building). + +### Reusable software + +There are source files in the repository that form the core of the Machine Leaning flow for all the use cases. These +are exposed as APIs that the examples can use and even be combined to form chained use cases. The API sources are +designed to be portable across platforms and provide functionality for preprocessing of data, running an inference, and +postprocessing of results. These allow a common flow for all use cases with minor differences in how each of these +blocks are instantiated. + +As an independent CMake project, these APIs can be used by or integrated into other projects easily. We also produce +[CMSIS Packs](https://developer.arm.com/tools-and-software/embedded/cmsis/cmsis-packs) with these sources, so they +could be used in all tools/IDEs (for example, +[Arm® Development Studio](https://developer.arm.com/Tools%20and%20Software/Arm%20Development%20Studio) and +[Keil® µVision®](https://www2.keil.com/mdk5/uvision/)) that support the use of CMSIS Packs. + +### Getting started + To run an ML application on the Cortex-M and Ethos-U NPU, please, follow these steps: -1. Setup your environment by installing [the required prerequisites](./docs/sections/building.md#Build-prerequisites). +1. Set up your environment by installing [the required prerequisites](./docs/sections/building.md#Build-prerequisites). 2. Generate an optimized neural network model for Ethos-U with a Vela compiler by following instructions [here](./docs/sections/building.md#Add-custom-model). 3. [Configure the build system](./docs/sections/building.md#Build-process). 4. [Compile the project](./docs/sections/building.md#Building-the-configured-project) with a `make` command. diff --git a/docs/documentation.md b/docs/documentation.md index b391092..8998adb 100644 --- a/docs/documentation.md +++ b/docs/documentation.md @@ -22,14 +22,15 @@ elsewhere. - Arm® and Corstone™ are registered trademarks or trademarks of Arm® Limited (or its subsidiaries) in the US and/or elsewhere. +- Arm®, Keil® and µVision® are registered trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere. - TensorFlow™, the TensorFlow logo, and any related marks are trademarks of Google Inc. ## Prerequisites Before starting the setup process, please make sure that you have: -- A Linux x86_64 based machine, the Windows Subsystem for Linux is preferable. - > **Note:** Currently, Windows is not supported as a build environment. +- A Linux x86_64 based machine. If using Microsoft® Windows®, Windows Subsystem for Linux (WSL) is preferable. + > **Note:** Currently, Microsoft® Windows® is not supported as a build environment. - At least one of the following toolchains: - GNU Arm Embedded toolchain (version 10.2.1 or above) - @@ -38,12 +39,13 @@ Before starting the setup process, please make sure that you have: [Arm Compiler download Page](https://developer.arm.com/tools-and-software/embedded/arm-compiler/downloads) - An Arm® MPS3 FPGA prototyping board and components for FPGA evaluation or a `Fixed Virtual Platform` binary: - - An MPS3 board loaded with Arm® Corstone™-300 reference package (`AN552`) from: - . You + - An MPS3 board loaded with Arm® Corstone™-300 (`AN552`) or Corstone™-310 reference package. See + . You must have a USB connection between your machine and the MPS3 board - for UART menu and for deploying the - application. - - `Arm Corstone-300` based FVP for MPS3 is available from: + application. + - `Arm® Corstone™-300` based FVP for MPS3 is available from: . + - `Arm® Corstone™-310` based FVP is available under Arm® Virtual Hardware: > **Note:**: There are two Arm® Corstone™-300 implementations available for the MPS3 FPGA board - application > notes `AN547` and `AN552`. We are aligned with the latest application note `AN552`. However, the application built @@ -60,6 +62,8 @@ for additional information: - Arm® `Cortex-M55`® processor: +- Arm® `Cortex-M85`® processor: + - ML processor, also referred to as a Neural Processing Unit (NPU) - Arm® `Ethos™-U55`: @@ -69,9 +73,13 @@ for additional information: - Arm® MPS3 FPGA Prototyping Board: +- Arm® Fixed Virtual Platforms: + +- Arm® Virtual Hardware: + - Arm® ML-Zoo: -- Arm® Ethos-U software: +- Arm® Ethos-U NPU™ software: To access Arm documentation online, please visit: @@ -80,40 +88,49 @@ To access Arm documentation online, please visit: The repository has the following structure: ```tree -. +├── CMakeLists.txt ├── dependencies ├── docs ├── model_conditioning_examples ├── resources ├── /resources_downloaded/ ├── scripts -│ ├── platforms -│ │ ├── mps3 -│ │ ├── native -│ │ └── simple_platform -│ └── ... +│ ├── cmake +│ │ ├── platforms +│ │ │ ├── mps3 +│ │ │ ├── native +│ │ │ └── simple_platform +│ │ └── ... +│ └── ... ├── source -│ ├── application -│ │ ├── main -│ │ └── tensorflow-lite-micro -│ ├── hal -│ ├── log -│ ├── math -│ ├── profiler -│ └── use_case +│ ├── application +│ │ ├── api +│ │ │ ├── common +│ │ │ └── use_case +│ │ └── main +│ ├── hal +│ │ ├── include +│ │ └── source +│ ├── log +│ │ └── include +│ ├── math +│ │ └── include +│ ├── profiler +│ │ └── include +│ use_case │ └── │ ├── include │ ├── src │ └── usecase.cmake -├── tests -└── CMakeLists.txt +└── tests ``` What these folders contain: -- `dependencies`: All the third-party dependencies for this project. +- `dependencies`: All the third-party dependencies for this project. These are either populated by `git submodule` or by + downloading packages in the required hierarchy. See `download_dependencies.py`. -- `docs`: The documentation for this ML application. +- `docs`: Detailed documentation for this repository. - `model_conditioning_examples`: short example scripts that demonstrate some methods available in TensorFlow to condition your model in preparation for deployment on Arm Ethos NPU. @@ -121,7 +138,8 @@ What these folders contain: - `resources`: contains ML use-cases applications resources such as input data, label files, etc. - `resources_downloaded`: created by `set_up_default_resources.py`, contains downloaded resources for ML use-cases - applications such as models, test data, etc. + applications such as models, test data, etc. It also contains a Python virtual environment with all the required + packages installed. - `scripts`: Build and source generation scripts. @@ -133,21 +151,33 @@ What these folders contain: Native profile related script compiles unit-tests. - `source`: C/C++ sources for the platform and ML applications. - > **Note:** Common code related to the `Ethos-U` NPU software framework resides in *application* subfolder. - The contents of the *application* subfolder is as follows: + The contents of the *application* sub-folder is as follows: - `application`: All sources that form the *core* of the application. The `use-case` part of the sources depend on the sources themselves, such as: - `main`: Contains the main function and calls to platform initialization logic to set up things before launching the main loop. Also contains sources common to all use-case implementations. - - `tensorflow-lite-micro`: Contains abstraction around TensorFlow Lite Micro API. This abstraction implements common - functions to initialize a neural network model, run an inference, and access inference results. + - `api`: Contains **platform-agnostic** API that all the use case examples can use. It depends only on TensorFlow + Lite Micro and math functionality exposed by `math` module. It is further subdivided into: + + - `common`: Common part of the API. This consists of the generic code like neural network model initialisation, + running an inference, and some common logic used for image and audio use cases. + + - `use_case`: This contains "model" and "processing" APIs for each individual use case. For example, KWS use case + contains a class for a generic KWS neural network model and the "processing" API give user an easier way to drive + the MFCC calculations. + +> **NOTE:** The API here is also used to export a CMSIS-pack from this repository and therefore, it is imperative to +> that the sources here do not depend on any HAL component or drive any platform dependent logic. If you are looking to +> reuse components from this repository for your application level logic, this directory should be the prime candidate. - `hal`: Contains Hardware Abstraction Layer (HAL) sources, providing a platform-agnostic API to access hardware platform-specific functions. +> **Note:** Common code related to the `Arm Ethos-U NPU` software framework resides in *hal/components* sub-folder. + - `log`: Common to all code logging macros managing log levels. - `math`: Math functions to be used in ML pipelines. Some of them use CMSIS DSP for optimized execution on Arm CPUs. diff --git a/docs/media/APIs_description.png b/docs/media/APIs_description.png index ce9f035..db86de0 100644 Binary files a/docs/media/APIs_description.png and b/docs/media/APIs_description.png differ diff --git a/docs/media/build_graph.png b/docs/media/build_graph.png index 76f4a40..ce80569 100644 Binary files a/docs/media/build_graph.png and b/docs/media/build_graph.png differ diff --git a/docs/sections/building.md b/docs/sections/building.md index c135afd..eba90b5 100644 --- a/docs/sections/building.md +++ b/docs/sections/building.md @@ -26,7 +26,7 @@ - [Building for different Ethos-U NPU variants](./building.md#building-for-different-ethos_u-npu-variants) - [Automatic file generation](./building.md#automatic-file-generation) -This section assumes that you are using an **x86 Linux** build machine. +This section assumes that you are using an **x86_64 Linux** build machine. ## Build prerequisites diff --git a/docs/sections/coding_guidelines.md b/docs/sections/coding_guidelines.md index 57c45e0..0306430 100644 --- a/docs/sections/coding_guidelines.md +++ b/docs/sections/coding_guidelines.md @@ -210,7 +210,7 @@ Software components written in C/C++ may use the language features allowed and i namespace nspace { void FunctionInNamespace(); - }; + } ``` - Source code must use Hungarian notation to annotate the name of a variable with information about its meaning. diff --git a/docs/sections/customizing.md b/docs/sections/customizing.md index d97aa9e..42be12a 100644 --- a/docs/sections/customizing.md +++ b/docs/sections/customizing.md @@ -10,7 +10,7 @@ - [Adding custom ML use-case](./customizing.md#adding-custom-ml-use_case) - [Implementing main loop](./customizing.md#implementing-main-loop) - [Implementing custom NN model](./customizing.md#implementing-custom-nn-model) - - [Define ModelPointer and ModelSize methods](./customizing.md#define-modelpointer-and-modelsize-methods) + - [Using GetModelPointer and GetModelLen methods](./customizing.md#using-getmodelpointer-and-getmodellen-methods) - [Executing inference](./customizing.md#executing-inference) - [Printing to console](./customizing.md#printing-to-console) - [Reading user input from console](./customizing.md#reading-user-input-from-console) @@ -32,49 +32,10 @@ The following sign indicates the important conventions to apply: ## Software project description -As mentioned in the [Repository structure](../documentation.md#repository-structure) section, project sources are: +See [Repository structure](../documentation.md#repository-structure) section for the outline of the repo. -```tree -├── dependencies -├── docs -│ ├── ... -│ └── Documentation.md -├── model_conditioning_examples -├── resources -│ └── img_class -│ └── ... -├── /resources_downloaded/ -│ └── img_class -│ └── ... -├── scripts -│ ├── platforms -│ │ ├── mps3 -│ │ ├── native -│ │ └── simple_platform -│ └── ... -├── source -│ ├── application -│ │ ├── main -│ │ └── tensorflow-lite-micro -│ ├── hal -│ ├── log -│ ├── math -│ ├── profiler -│ └── use_case -│ └── -│ ├── include -│ ├── src -│ └── usecase.cmake -├── tests -└── CMakeLists.txt -``` - -Where the `source` folder contains C/C++ sources for the platform and ML applications. Common code related to the -*Ethos-U* code samples software framework resides in the `application` sub-folder and ML application-specific logic, -use-cases, sources are in the `use-case` subfolder. - -> **Convention**: Separate use-cases must be organized in sub-folders under the use-case folder. The name of the -> directory is used as a name for this use-case and can be provided as a `USE_CASE_BUILD` parameter value. The build +> **Convention**: Separate use-cases must be organized in sub-folders under the `source/use-case` folder. The name of +> the directory is used as a name for this use-case and can be provided as a `USE_CASE_BUILD` parameter value. The build > system expects that sources for the use-case are structured as follows: Headers in an `include` directory and C/C++ > sources in a `src` directory. For example: > @@ -86,6 +47,11 @@ use-cases, sources are in the `use-case` subfolder. > └── src > └── *.cc > ``` +> +> It is important to note that each use case example has at least one associated API that it uses from +> `source/application/api/use_case`. The API sources are **platform-agnostic** by design so the use cases example +> implementations can re-use one or more of these components, and they can be used on any target. However, it +> is not mandatory to use an API, or to implement one if you are adding a use-case. ## Hardware Abstraction Layer API @@ -94,9 +60,9 @@ The HAL is represented by the following interfaces. To access them, include the - `hal_platform_init` function: Initializes the HAL platform and every module on the platform that the application requires to run. - | Parameter name | Description | - |--------------------------------------| ------------------------------------------------------------------- | - | `return` | true if successful, false otherwise. | + | Parameter name | Description | + |-----------------|-----------------------------------------| + | `return` | true if successful, false otherwise. | - `hal_platform_release` function Releases the HAL platform and any acquired resources. @@ -140,11 +106,11 @@ void main_loop() Application context can be used as a holder for a state between main loop iterations. Include `AppContext.hpp` to use `ApplicationContext` class. -| Method name | Description | -|--------------|------------------------------------------------------------------| -| `Set` | Saves given value as a named attribute in the context. | -| `Get` | Gets the saved attribute from the context by the given name. | -| `Has` | Checks if an attribute with a given name exists in the context. | +| Method name | Description | +|-------------|-----------------------------------------------------------------| +| `Set` | Saves given value as a named attribute in the context. | +| `Get` | Gets the saved attribute from the context by the given name. | +| `Has` | Checks if an attribute with a given name exists in the context. | For example: @@ -172,13 +138,13 @@ It uses platform timer to get system timing information. | Method name | Description | |-------------------------|----------------------------------------------------------------| -| `StartProfiling` | Starts profiling and records the starting timing data. | -| `StopProfiling` | Stops profiling and records the ending timing data. | -| `StopProfilingAndReset` | Stops the profiling and internally resets the platform timers. | -| `Reset` | Resets the profiler and clears all collected data. | -| `GetAllResultsAndReset` | Gets all the results as string and resets the profiler. | -| `PrintProfilingResult` | Prints collected profiling results and resets the profiler. | -| `SetName` | Set the profiler name. | +| `StartProfiling` | Starts profiling and records the starting timing data. | +| `StopProfiling` | Stops profiling and records the ending timing data. | +| `StopProfilingAndReset` | Stops the profiling and internally resets the platform timers. | +| `Reset` | Resets the profiler and clears all collected data. | +| `GetAllResultsAndReset` | Gets all the results as string and resets the profiler. | +| `PrintProfilingResult` | Prints collected profiling results and resets the profiler. | +| `SetName` | Set the profiler name. | An example of it in use: @@ -199,38 +165,36 @@ The Model, which refers to neural network model, is an abstract class wrapping t It provides methods to perform common operations such as TensorFlow Lite Micro framework initialization, inference execution, accessing input, and output tensor objects. -To use this abstraction, import the `TensorFlowLiteMicro.hpp` header. - -| Method name | Description | -|--------------------------|------------------------------------------------------------------------------| -| `GetInputTensor` | Returns the pointer to the model's input tensor. | -| `GetOutputTensor` | Returns the pointer to the model's output tensor | -| `GetType` | Returns the model's data type | -| `GetInputShape` | Return the pointer to the model's input shape | -| `GetOutputShape` | Return the pointer to the model's output shape. | -| `GetNumInputs` | Return the number of input tensors the model has. | -| `GetNumOutputs` | Return the number of output tensors the model has. | -| `LogTensorInfo` | Logs the tensor information to `stdout` for the given tensor pointer. Includes: Tensor name, tensor address, tensor type, tensor memory size, and quantization params. | -| `LogInterpreterInfo` | Logs the interpreter information to stdout. | -| `Init` | Initializes the TensorFlow Lite Micro framework, allocates require memory for the model. | -| `GetAllocator` | Gets the allocator pointer for the instance. | -| `IsInited` | Checks if this model object has been initialized. | -| `IsDataSigned` | Checks if the model uses signed data type. | -| `RunInference` | Runs the inference, so invokes the interpreter. | -| `ShowModelInfoHandler` | Model information handler common to all models. | -| `GetTensorArena` | Returns pointer to memory region to be used for tensors allocations. | -| `ModelPointer` | Returns the pointer to the NN model data array. | -| `ModelSize` | Returns the model size. | -| `GetOpResolver` | Returns the reference to the TensorFlow Lite Micro operator resolver. | -| `EnlistOperations` | Registers required operators with TensorFlow Lite Micro operator resolver. | -| `GetActivationBufferSize` | Returns the size of the tensor arena memory region. | +To use this abstraction, import the `Model.hpp` header. + +| Method name | Description | +|---------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `GetInputTensor` | Returns the pointer to the model's input tensor. | +| `GetOutputTensor` | Returns the pointer to the model's output tensor | +| `GetType` | Returns the model's data type | +| `GetInputShape` | Return the pointer to the model's input shape | +| `GetOutputShape` | Return the pointer to the model's output shape. | +| `GetNumInputs` | Return the number of input tensors the model has. | +| `GetNumOutputs` | Return the number of output tensors the model has. | +| `LogTensorInfo` | Logs the tensor information to `stdout` for the given tensor pointer. Includes: Tensor name, tensor address, tensor type, tensor memory size, and quantization params. | +| `LogInterpreterInfo` | Logs the interpreter information to stdout. | +| `Init` | Initializes the TensorFlow Lite Micro framework, allocates require memory for the model. | +| `GetAllocator` | Gets the allocator pointer for the instance. | +| `IsInited` | Checks if this model object has been initialized. | +| `IsDataSigned` | Checks if the model uses signed data type. | +| `RunInference` | Runs the inference, so invokes the interpreter. | +| `ShowModelInfoHandler` | Model information handler common to all models. | +| `GetTensorArena` | Returns pointer to memory region to be used for tensors allocations. | +| `ModelPointer` | Returns the pointer to the NN model data array. | +| `ModelSize` | Returns the model size. | +| `GetOpResolver` | Returns the reference to the TensorFlow Lite Micro operator resolver. | +| `EnlistOperations` | Registers required operators with TensorFlow Lite Micro operator resolver. | +| `GetActivationBufferSize` | Returns the size of the tensor arena memory region. | > **Convention:** Each ML use-case must have an extension of this class and an implementation of the protected virtual > methods: > > ```C++ -> virtual const uint8_t* ModelPointer() = 0; -> virtual size_t ModelSize() = 0; > virtual const tflite::MicroOpResolver& GetOpResolver() = 0; > virtual bool EnlistOperations() = 0; > virtual size_t GetActivationBufferSize() = 0; @@ -241,7 +205,7 @@ To use this abstraction, import the `TensorFlowLiteMicro.hpp` header. > tensor arena memory for TensorFlow Lite Micro framework by the `GetTensorArena` and `GetActivationBufferSize` methods. > > **Note:** Please see `MobileNetModel.hpp` and `MobileNetModel.cc` files from the image classification ML application -> use-case as an example of the model base class extension. +> API as an example of the model base class extension. ## Adding custom ML use-case @@ -263,8 +227,8 @@ use_case └── src ``` -Start with creation of a sub-directory under the `use_case` directory and two additional directories `src` and `include` -as described in the [Software project description](./customizing.md#software-project-description) section. +Start with creation of a subdirectory under the `source/use_case` directory and two additional directories `src` and +`include` as described in the [Software project description](./customizing.md#software-project-description) section. ## Implementing main loop @@ -277,7 +241,7 @@ Main loop has knowledge about the platform and has access to the platform compon Layer (HAL). Start by creating a `MainLoop.cc` file in the `src` directory (the one created under -[Adding custom ML use case](./customizing.md#adding-custom-ml-use-case)). The name used is not important. +[Adding custom ML use-case](./customizing.md#adding-custom-ml-use_case)). The name used is not important. Now define the `main_loop` function with the signature described in [Main loop function](./customizing.md#main-loop-function): @@ -285,12 +249,13 @@ Now define the `main_loop` function with the signature described in [Main loop f #include "hal.h" #include "log_macros.h" -void main_loop() { - printf("Hello world!"); +void main_loop() +{ + printf("Hello world!"); } ``` -The preceding code is already a working use-case. If you compile and run it (see [Building custom usecase](./customizing.md#building-custom-use-case)), +The preceding code is already a working use-case. If you compile and run it (see [Building custom use-case](./customizing.md#building-custom-use_case)), then the application starts and prints a message to console and exits straight away. You can now start filling this function with logic. @@ -301,7 +266,7 @@ Before inference could be run with a custom NN model, TensorFlow Lite Micro fram layers, included in the model. You must register operators using the `MicroMutableOpResolver` API. The *Ethos-U* code samples project has an abstraction around TensorFlow Lite Micro API (see [NN model API](./customizing.md#nn-model-api)). -Create `HelloWorldModel.hpp` in the use-case include sub-directory, extend Model abstract class, +Create `HelloWorldModel.hpp` in the use-case include subdirectory, extend Model abstract class, and then declare the required methods. For example: @@ -336,7 +301,7 @@ class HelloWorldModel: public Model { #endif /* HELLOWORLDMODEL_HPP */ ``` -Create the `HelloWorldModel.cc` file in the `src` sub-directory and define the methods there. Include +Create the `HelloWorldModel.cc` file in the `src` subdirectory and define the methods there. Include `HelloWorldModel.hpp` created earlier. > **Note:** The `Model.hpp` included in the header provides access to TensorFlow Lite Micro's operation resolver API. @@ -374,25 +339,58 @@ bool arm::app::HelloWorldModel::EnlistOperations() { To minimize the memory footprint of the application, we advise you to only register operators that are used by the NN model. -### Define ModelPointer and ModelSize methods +### Using GetModelPointer and GetModelLen methods + +These functions generated in the C++ file containing the neural network model as an array. This logic for generation of +the C++ array from the `.tflite` file needs to be defined in the `usecase.cmake` file for this `HelloWorld` example. +In the root of the `source/use_case/hello_world`, create a file called `usecase.cmake` and add the following lines to +it: + +```cmake +# Generate model file +USER_OPTION(${${use_case}_MODEL_TFLITE_PATH} + "NN model tflite path" + "Path-to-your-model.tflite" + FILEPATH) + +generate_tflite_code( + MODEL_PATH ${${use_case}_MODEL_TFLITE_PATH} + DESTINATION ${SRC_GEN_DIR} + EXPRESSIONS ${EXTRA_MODEL_CODE} + NAMESPACE "arm" "app" "hello_world") +``` -These functions are wrappers around the functions generated in the C++ file containing the neural network model as an -array. This logic for generation of the C++ array from the `.tflite` file needs to be defined in the `usecase.cmake` file for -this `HelloWorld` example. +Use the `${use-case}_MODEL_TFLITE_PATH` CMake configuration parameter to include custom model in the generation or +compilation process. Please refer to: [Build options](./building.md#build-options) for further information. For more details on `usecase.cmake`, refer to: [Building options](./building.md#build-options). For details on code generation flow in general, refer to: [Automatic file generation](./building.md#automatic-file-generation). -The TensorFlow Lite model data is read during the `Model::Init()` method execution. Please refer to -`application/tensorflow-lite-micro/Model.cc` for more details. +The TensorFlow Lite model data is read during the `Model::Init` method execution. Please refer to +`source/application/api/common/source/Model.cc` for more details. + +`Model::Init` will need the pointer to the model. The `arm::app::hello_world::GetModelPointer()` function is generated +during the build and can be found in the file `/generated/hello_world/src/.cc`. The file +generated is automatically added to the compilation. + +At the top of `MainLoop.cc`, add: + +```c++ +namespace arm { +namespace app { + namespace hello_world { + + extern uint8_t* GetModelPointer(); + extern size_t GetModelLen(); + } /* namespace hello_world */ -Model invokes the `ModelPointer()` function which calls the `GetModelPointer()` function to get the neural network model -data memory address. The `GetModelPointer()` function is generated during the build and can be found in the file -`build/generated/hello_world/src/.cc`. The file generated is automatically added to the compilation. + static uint8_t tensorArena[ACTIVATION_BUF_SZ] ACTIVATION_BUF_ATTRIBUTE; +} /* namespace app */ +} /* namespace arm */ +``` -Use the `${use-case}_MODEL_TFLITE_PATH` build parameter to include custom model in the generation or compilation -process. Please refer to: [Build options](./building.md#build-options) for further information. +These functions can now be used in the `Model.Init` call. ## Executing inference @@ -404,8 +402,9 @@ To run an inference successfully, you must use: - A main loop function, - And some input data. -For the `hello_world` example below the input array is not populated. However, for real-world deployment this data must either be read from an on-board device or be prepared in -the form of C++ sources and baked into the application before compilation. +For the `hello_world` example below the input array is not populated. However, for real-world deployment this data must +either be read from an on-board device or be prepared in the form of C++ sources and baked into the application before +compilation. For example, the image classification application requires extra build steps to generate C++ sources from the provided images with `generate_images_code` CMake function. @@ -418,47 +417,48 @@ images with `generate_images_code` CMake function. The following code adds inference invocation to the main loop function: -```C++ +```c++ #include "hal.h" -#include "HelloWorldModel.hpp" #include "log_macros.h" +#include "HelloWorldModel.hpp" - namespace arm { - namespace app { - static uint8_t tensorArena[ACTIVATION_BUF_SZ] ACTIVATION_BUF_ATTRIBUTE; - } /* namespace app */ - } /* namespace arm */ - - extern uint8_t* GetModelPointer(); - extern size_t GetModelLen(); +namespace arm { +namespace app { + namespace hello_world { - void main_loop() { + extern uint8_t* GetModelPointer(); + extern size_t GetModelLen(); + } /* namespace hello_world */ - /* model wrapper object */ - arm::app::HelloWorldModel model; + static uint8_t tensorArena[ACTIVATION_BUF_SZ] ACTIVATION_BUF_ATTRIBUTE; +} /* namespace app */ +} /* namespace arm */ - /* Load the model */ - if (!model.Init(arm::app::tensor_arena, - sizeof(arm::app::tensor_arena), - GetModelPointer(), - GetModelLen())) { - printf_err("failed to initialise model\n"); - return; - } +void main_loop() +{ + printf("Hello world!"); - TfLiteTensor *outputTensor = model.GetOutputTensor(); - TfLiteTensor *inputTensor = model.GetInputTensor(); + arm::app::HelloWorldModel model; - /* dummy input data*/ - uint8_t inputData[1000]; + /* Load the model. */ + if (!model.Init(arm::app::tensorArena, + sizeof(arm::app::tensorArena), + arm::app::hello_world::GetModelPointer(), + arm::app::hello_world::GetModelLen())) { + printf_err("failed to initialise model\n"); + return; + } - memcpy(inputTensor->data.data, inputData, 1000); + /* Populate input tensors here */ + // Your-custom-code; - /* run inference */ - model.RunInference(); + /* Run inference */ + model.RunInference(); - const uint32_t tensorSz = outputTensor->bytes; - const uint8_t * outputData = tflite::GetTensorData(outputTensor); + /* Read or post-process output here */ + const uint32_t tensorSz = outputTensor->bytes; + const uint8_t * outputData = tflite::GetTensorData(outputTensor); + // Your-custom-code; } ``` @@ -466,18 +466,18 @@ The code snippet has several important blocks: - Creating HelloWorldModel object and initializing it. - ```C++ +```C++ arm::app::HelloWorldModel model; /* Load the model */ - if (!model.Init(arm::app::tensor_arena, - sizeof(arm::app::tensor_arena), - GetModelPointer(), - GetModelLen())) { - printf_err(\"failed to initialise model\\n\"); - return; + if (!model.Init(arm::app::tensorArena, + sizeof(arm::app::tensorArena), + arm::app::hello_world::GetModelPointer(), + arm::app::hello_world::GetModelLen())) { + printf_err("failed to initialise model\n"); + return; } - ``` +``` - Getting pointers to allocated input and output tensors. @@ -485,13 +485,6 @@ The code snippet has several important blocks: TfLiteTensor *outputTensor = model.GetOutputTensor(); TfLiteTensor *inputTensor = model.GetInputTensor(); ``` - -- Copying input data to the input tensor. We assume input tensor size to be 1000 `uint8` elements. - - ```C++ - memcpy(inputTensor->data.data, inputData, 1000); - ``` - - Running inference ```C++ @@ -623,8 +616,7 @@ in the root of your use-case. However, the name of the file is not important. > - `CMAKE_CXX_FLAGS` and `CMAKE_C_FLAGS` – The compilation flags. > - `CMAKE_EXE_LINKER_FLAGS` – The linker flags. -For the hello world use-case, it is enough to create a `helloworld.cmake` file and set the `DEFAULT_MODEL_PATH`, like -so: +For the hello world use-case, it is enough to create a `helloworld.cmake` file and set the `DEFAULT_MODEL_PATH`, like: ```cmake if (ETHOS_U_NPU_ENABLED) diff --git a/download_dependencies.py b/download_dependencies.py index b62c9b1..483fe84 100755 --- a/download_dependencies.py +++ b/download_dependencies.py @@ -24,10 +24,10 @@ from urllib.request import urlopen from zipfile import ZipFile from pathlib import Path -TF = "https://github.com/tensorflow/tflite-micro/archive/1a0287fc5fa81fa6aa1dcfb0c5b2e01f74164393.zip" -CMSIS = "https://github.com/ARM-software/CMSIS_5/archive/9b5df640c777563919affb4e9201c96c657adbb2.zip" -ETHOS_U_CORE_DRIVER = "https://git.mlplatform.org/ml/ethos-u/ethos-u-core-driver.git/snapshot/ethos-u-core-driver-22.02.tar.gz" -ETHOS_U_CORE_PLATFORM = "https://git.mlplatform.org/ml/ethos-u/ethos-u-core-platform.git/snapshot/ethos-u-core-platform-22.02.tar.gz" +TF = "https://github.com/tensorflow/tflite-micro/archive/07821fd35e8aaca66855fa07b402325be3b70398.zip" +CMSIS = "https://github.com/ARM-software/CMSIS_5/archive/6a18a74b46ac1501a7a750dd83b8bfb06fb24504.zip" +ETHOS_U_CORE_DRIVER = "https://git.mlplatform.org/ml/ethos-u/ethos-u-core-driver.git/snapshot/ethos-u-core-driver-22.05-rc2.tar.gz" +ETHOS_U_CORE_PLATFORM = "https://git.mlplatform.org/ml/ethos-u/ethos-u-core-platform.git/snapshot/ethos-u-core-platform-22.05-rc2.tar.gz" def download(url_file: str, post_process=None): diff --git a/scripts/mps3/sse-310/images.txt b/scripts/mps3/sse-310/images.txt index 8a920cb..ef9ae8e 100644 --- a/scripts/mps3/sse-310/images.txt +++ b/scripts/mps3/sse-310/images.txt @@ -1,23 +1,14 @@ TITLE: Arm MPS3 FPGA prototyping board Images Configuration File -; MCC mapping for Corstone-310 MPS3 bitfile package AN555 -; +-------------+---------------+-------------------------------+ -; | FPGA addr | MCC addr | Region | -; +-------------+---------------+-------------------------------+ -; | 0x00000000 | 0x00000000 | ITCM (NS) | -; | 0x01000000 | 0x02000000 | BRAM or FPGA's data SRAM (NS) | -; | 0x60000000 | 0x08000000 | DDR (NS) | -; | 0x70000000 | 0x0c000000 | DDR (S) | -; +-------------+---------------+-------------------------------+ - [IMAGES] +TOTALIMAGES: 2 ;Number of Images (Max: 32) -TOTALIMAGES: 2 ;Number of Images (Max: 32) - -IMAGE0ADDRESS: 0x00000000 ; MCC@0x00000000 <=> FPGA@0x00000000 -IMAGE0UPDATE: AUTO -IMAGE0FILE: \SOFTWARE\itcm.bin +IMAGE0PORT: 1 +IMAGE0ADDRESS: 0x00_1100_0000 ; Address to load into +IMAGE0UPDATE: RAM ; Image Update:NONE/AUTO/FORCE +IMAGE0FILE: \SOFTWARE\bram.bin ; Image/data to be loaded -IMAGE1ADDRESS: 0x0c000000 ; MCC@0x0c000000 <=> FPGA@0x70000000 -IMAGE1UPDATE: AUTO -IMAGE1FILE: \SOFTWARE\ddr.bin +IMAGE1PORT: 1 +IMAGE1ADDRESS: 0x00_7000_0000 ; Address to load into +IMAGE1UPDATE: RAM ; Image Update:NONE/AUTO/FORCE +IMAGE1FILE: \SOFTWARE\ddr.bin ; Image/data to be loaded -- cgit v1.2.1