From b9e9c899dcbbe35cac72bceb117ae4ec56494d81 Mon Sep 17 00:00:00 2001 From: Kshitij Sisodia Date: Thu, 27 May 2021 13:57:35 +0100 Subject: MLECO-1943: Documentation review Major update for the documentation. Also, a minor logging change in helper scripts. Change-Id: Ia79f78a45c9fa2d139418fbc0ca9e52245704ba3 --- docs/documentation.md | 203 ++++++++++++++++++++++++++------------------------ 1 file changed, 105 insertions(+), 98 deletions(-) (limited to 'docs/documentation.md') diff --git a/docs/documentation.md b/docs/documentation.md index d08e313..a55e577 100644 --- a/docs/documentation.md +++ b/docs/documentation.md @@ -17,40 +17,45 @@ ## Trademarks - Arm® and Cortex® are registered trademarks of Arm® Limited (or its subsidiaries) in the US and/or elsewhere. -- Arm® and Ethos™ are registered trademarks or trademarks of Arm® Limited (or its subsidiaries) in the US and/or elsewhere. -- Arm® and Corstone™ are registered trademarks or trademarks of Arm® Limited (or its subsidiaries) in the US and/or elsewhere. -- TensorFlow™, the TensorFlow logo and any related marks are trademarks of Google Inc. +- Arm® and Ethos™ are registered trademarks or trademarks of Arm® Limited (or its subsidiaries) in the US and/or + elsewhere. +- Arm® and Corstone™ are registered trademarks or trademarks of Arm® Limited (or its subsidiaries) in the US and/or + elsewhere. +- TensorFlow™, the TensorFlow logo, and any related marks are trademarks of Google Inc. ## Prerequisites Before starting the setup process, please make sure that you have: -- Linux x86_64 based machine or Windows Subsystem for Linux is preferable. - Unfortunately, Windows is not supported as a build environment yet. +- A Linux x86_64 based machine, the Windows Subsystem for Linux is preferable. + > **Note:** Currently, Windows is not supported as a build environment. - At least one of the following toolchains: - - GNU Arm Embedded Toolchain (version 10.2.1 or above) - [GNU Arm Embedded Toolchain Downloads](https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads) - - Arm Compiler (version 6.14 or above) with a valid license - [Arm Compiler Download Page](https://developer.arm.com/tools-and-software/embedded/arm-compiler/downloads/) + - GNU Arm Embedded toolchain (version 10.2.1 or above) - + [GNU Arm Embedded toolchain downloads](https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads) + - Arm Compiler (version 6.14 or above) with a valid license - + [Arm Compiler download Page](https://developer.arm.com/tools-and-software/embedded/arm-compiler/downloads/) - An Arm® MPS3 FPGA prototyping board and components for FPGA evaluation or a `Fixed Virtual Platform` binary: - - An MPS3 board loaded with Arm® Corstone™-300 reference package (`AN547`) from: - . - You would also need to have a USB connection between your machine and the MPS3 board - for UART menu and for - deploying the application. - - `Arm Corstone-300` based FVP for MPS3 is available from: . + - An MPS3 board loaded with Arm® Corstone™-300 reference package (`AN547`) from: + . You + must have a USB connection between your machine and the MPS3 board - for UART menu and for deploying the + application. + - `Arm Corstone-300` based FVP for MPS3 is available from: + . ### Additional reading -This document contains information that is specific to Arm® Ethos™-U55 products. -See the following documents for other relevant information: +This document contains information that is specific to Arm® Ethos™-U55 products. Please refer to the following documents +for additional information: - ML platform overview: - Arm® ML processors technical overview: -- Arm® Cortex-M55® processor: +- Arm® `Cortex-M55`® processor: -- ML processor, also referred to as a Neural Processing Unit (NPU) - Arm® Ethos™-U55: +- ML processor, also referred to as a Neural Processing Unit (NPU) - Arm® `Ethos™-U55`: - Arm® MPS3 FPGA Prototyping Board: @@ -58,7 +63,7 @@ See the following documents for other relevant information: - Arm® ML-Zoo: -See for access to Arm documentation. +To access Arm documentation online, please visit: ## Repository structure @@ -80,56 +85,54 @@ The repository has the following structure: │ │ └── tensorflow-lite-micro │ └── use_case │ └── -│ ├── include -│ ├── src -│ └── usecase.cmake +│   ├── include +│   ├── src +│   └── usecase.cmake ├── tests │ └── ... └── CMakeLists.txt ``` -Where: +What these folders contain: -- `dependencies`: contains all the third party dependencies for this project. +- `dependencies`: All the third-party dependencies for this project. -- `docs`: contains the documentation for this ML applications. +- `docs`: The documentation for this ML application. -- `model_conditioning_examples`: contains short example scripts that demonstrate some methods available in TensorFlow +- `model_conditioning_examples`: short example scripts that demonstrate some methods available in TensorFlow to condition your model in preparation for deployment on Arm Ethos NPU. -- `resources`: contains ML use cases applications resources such as input data, label files, etc. +- `resources`: contains ML use-cases applications resources such as input data, label files, etc. -- `resources_downloaded`: created by `set_up_default_resources.py`, contains downloaded resources for ML use cases +- `resources_downloaded`: created by `set_up_default_resources.py`, contains downloaded resources for ML use-cases applications such as models, test data, etc. -- `scripts`: contains build related and source generation scripts. +- `scripts`: Build and source generation scripts. -- `source`: contains C/C++ sources for the platform and ML applications. - Common code related to the Ethos-U55 NPU software - framework resides in *application* sub-folder with the following - structure: +- `source`: C/C++ sources for the platform and ML applications. + > **Note:** Common code related to the `Ethos-U55` NPU software framework resides in *application* subfolder. - - `application`: contains all the sources that form the *core* of the application. - The `use case` part of the sources depend on sources here. + The contents of the *application* subfolder is as follows: - - `hal`: contains hardware abstraction layer sources providing a - platform agnostic API to access hardware platform specific functions. + - `application`: All sources that form the *core* of the application. The `use-case` part of the sources depend on the + sources themsleves, such as: - - `main`: contains the main function and calls to platform initialization - logic to set things up before launching the main loop. - It also contains sources common to all use case implementations. + - `hal`: Contains Hardware Abstraction Layer (HAL) sources, providing a platform agnostic API to access hardware + platform-specific functions. - - `tensorflow-lite-micro`: contains abstraction around TensorFlow Lite Micro API - implementing common functions to initialize a neural network model, run an inference, and - access inference results. + - `main`: Contains the main function and calls to platform initialization logic to set up things before launching + the main loop. Also contains sources common to all use-case implementations. - - `use_case`: contains the ML use-case specific logic. Having this as a separate sub-folder isolates ML specific - application logic with the assumption that the `application` will do all the required set up for logic here to run. - It also makes it easier to add a new use case block. + - `tensorflow-lite-micro`: Contains abstraction around TensorFlow Lite Micro API. This abstraction implements common + functions to initialize a neural network model, run an inference, and access inference results. - - `tests`: contains the x86 tests for the use case applications. + - `use_case`: Contains the ML use-case specific logic. Stored as a separate subfolder, it helps isolate the + ML-specific application logic. With the assumption that the `application` performs the required setup for logic to + run. It also makes it easier to add a new use-case block. -Hardware abstraction layer has the following structure: + - `tests`: Contains the x86 tests for the use-case applications. + +The HAL has the following structure: ```tree hal @@ -157,71 +160,75 @@ hal └── native ``` -- `include` and `hal.c`: contains the hardware abstraction layer (HAL) top level platform API and data acquisition, data -presentation and timer interfaces. - > Note: the files here and lower in the hierarchy have been written in - C and this layer is a clean C/C++ boundary in the sources. +What these folders contain: + +- The folders `include` and `hal.c` contain the HAL top-level platform API and data acquisition, data presentation, and + timer interfaces. + > **Note:** the files here and lower in the hierarchy have been written in C and this layer is a clean C/ + boundary + > in the sources. - `platforms/bare-metal/data_acquisition`\ -`platforms/bare-metal/data_presentation`\ -`platforms/bare-metal/timer`\ -`platforms/bare-metal/utils`: contains bare metal HAL support layer and platform initialisation helpers. Function calls - are routed to platform specific logic at this level. For example, for data presentation, an `lcd` module has been used. - This wraps the LCD driver calls for the actual hardware (for example MPS3). + `platforms/bare-metal/data_presentation`\ + `platforms/bare-metal/timer`\ + `platforms/bare-metal/utils`: + + These folders contain the bare-metal HAL support layer and platform initialization helpers. Function calls are routed + to platform-specific logic at this level. For example, for data presentation, an `lcd` module has been used. This + `lcd` module wraps the LCD driver calls for the actual hardware (for example, MPS3). -- `platforms/bare-metal/bsp/bsp-packs`: contains the core low-level drivers (written in C) for the platform reside. - For supplied examples this happens to be an MPS3 board, but support could be added here for other platforms too. - The functions defined in this space are wired to the higher level functions under HAL (as those in `platforms/bare-metal/` level). +- `platforms/bare-metal/bsp/bsp-packs`: The core low-level drivers (written in C) for the platform reside. For supplied + examples, this happens to be an MPS3 board. However, support can be added here for other platforms. The functions + defined in this space are wired to the higher-level functions under HAL and is like those in the + `platforms/bare-metal/` level). - `platforms/bare-metal/bsp/bsp-packs/mps3/include`\ -`platforms/bare-metal/bsp/bsp-packs/mps3`: contains the peripheral (LCD, UART and timer) drivers specific to MPS3 board. + `platforms/bare-metal/bsp/bsp-packs/mps3`: Contains the peripheral (LCD, UART, and timer) drivers specific to MPS3 + board. - `platforms/bare-metal/bsp/bsp-core`\ -`platforms/bare-metal/bsp/include`: contains the BSP core sources common to all BSPs. These include a UART header - (only the implementation of this is platform specific, but the API is common) and "re-targeting" of the standard output - and error streams to the UART block. + `platforms/bare-metal/bsp/include`: Contains the BSP core sources common to all BSPs and includes a UART header. + However, the implementation of this is platform-specific, while the API is common. Also "re-targets" the standard + output and error streams to the UART block. -- `platforms/bare-metal/bsp/cmsis-device`: contains the CMSIS template implementation for the CPU and also device - initialisation routines. It is also where the system interrupts are set up and handlers are overridden. - The main entry point of a bare metal application will most likely reside in this space. This entry point is - responsible for setting up before calling the user defined "main" function in the higher level `application` logic. +- `platforms/bare-metal/bsp/cmsis-device`: Contains the CMSIS template implementation for the CPU and also device + initialization routines. It is also where the system interrupts are set up and the handlers are overridden. The main + entry point of a bare-metal application most likely resides in this space. This entry point is responsible for the + set-up before calling the user defined "main" function in the higher-level `application` logic. -- `platforms/bare-metal/bsp/mem_layout`: contains the platform specific linker scripts. +- `platforms/bare-metal/bsp/mem_layout`: Contains the platform-specific linker scripts. ## Models and resources -The models used in the use cases implemented in this project can be downloaded -from [Arm ML-Zoo](https://github.com/ARM-software/ML-zoo/). +The models used in the use-cases implemented in this project can be downloaded from: [Arm ML-Zoo](https://github.com/ARM-software/ML-zoo/). - [Mobilenet V2](https://github.com/ARM-software/ML-zoo/blob/master/models/image_classification/mobilenet_v2_1.0_224/tflite_uint8). - [DS-CNN](https://github.com/ARM-software/ML-zoo/blob/master/models/keyword_spotting/ds_cnn_large/tflite_clustered_int8). - [Wav2Letter](https://github.com/ARM-software/ML-zoo/tree/1a92aa08c0de49a7304e0a7f3f59df6f4fd33ac8/models/speech_recognition/wav2letter/tflite_pruned_int8). - [Anomaly Detection](https://github.com/ARM-software/ML-zoo/raw/7c32b097f7d94aae2cd0b98a8ed5a3ba81e66b18/models/anomaly_detection/micronet_medium/tflite_int8/ad_medium_int8.tflite). -When using Ethos-U55 NPU backend, the NN model is assumed to be optimized by Vela compiler. -However, even if not, it will fall back on the CPU and execute, if supported by TensorFlow Lite Micro. +When using *Ethos-U55* NPU backend, Vela compiler optimizes the the NN model. However, if not and it is supported by +TensorFlow Lite Micro, then it falls back on the CPU and execute. ![Vela compiler](./media/vela_flow.jpg) -The Vela compiler is a tool that can optimize a neural network model -into a version that can run on an embedded system containing Ethos-U55 NPU. +The Vela compiler is a tool that can optimize a neural network model into a version that run on an embedded system +containing the *Ethos-U55* NPU. -The optimized model will contain custom operators for sub-graphs of the -model that can be accelerated by Ethos-U55 NPU, the remaining layers that -cannot be accelerated are left unchanged and will run on the CPU using -optimized (CMSIS-NN) or reference kernels provided by the inference -engine. +The optimized model contains custom operators for sub-graphs of the model that the *Ethos-U55* NPU can accelerate. The +remaining layers that cannot be accelerated, are left unchanged, and are run on the CPU using optimized, `CMSIS-NN`, or +reference kernels provided by the inference engine. -For detailed information see [Optimize model with Vela compiler](./sections/building.md#Optimize-custom-model-with-Vela-compiler). +For detailed information, see: [Optimize model with Vela compiler](./sections/building.md#Optimize-custom-model-with-Vela-compiler). ## Building -This section describes how to build the code sample applications from sources - illustrating the build +This section describes how to build the code sample applications from sources and includes illustrating the build options and the process. -The project can be built for MPS3 FPGA and FVP emulating MPS3. Default values for configuration parameters -will build executable models with Ethos-U55 NPU support. -See: +The project can be built for MPS3 FPGA and FVP emulating MPS3. Using default values for configuration parameters builds +executable models that support the *Ethos-U55* NPU. + +For further information, please see: - [Building the ML embedded code sample applications from sources](./sections/building.md#building-the-ml-embedded-code-sample-applications-from-sources) - [Build prerequisites](./sections/building.md#build-prerequisites) @@ -232,10 +239,10 @@ See: - [Fetching resource files](./sections/building.md#fetching-resource-files) - [Create a build directory](./sections/building.md#create-a-build-directory) - [Configuring the build for MPS3 SSE-300](./sections/building.md#configuring-the-build-for-mps3-sse-300) - - [Using GNU Arm Embedded Toolchain](./sections/building.md#using-gnu-arm-embedded-toolchain) + - [Using GNU Arm embedded toolchain](./sections/building.md#using-gnu-arm-embedded-toolchain) - [Using Arm Compiler](./sections/building.md#using-arm-compiler) - [Generating project for Arm Development Studio](./sections/building.md#generating-project-for-arm-development-studio) - - [Working with model debugger from Arm FastModel Tools](./sections/building.md#working-with-model-debugger-from-arm-fastmodel-tools) + - [Working with model debugger from Arm Fast Model Tools](./sections/building.md#working-with-model-debugger-from-arm-fastmodel-tools) - [Configuring with custom TPIP dependencies](./sections/building.md#configuring-with-custom-tpip-dependencies) - [Configuring native unit-test build](./sections/building.md#configuring-native-unit-test-build) - [Configuring the build for simple_platform](./sections/building.md#configuring-the-build-for-simple_platform) @@ -248,8 +255,9 @@ See: ## Deployment -This section describes how to deploy the code sample applications on the Fixed Virtual Platform or the MPS3 board. -See: +This section describes how to deploy the code sample applications on the Fixed Virtual Platform (FVP) or the MPS3 board. + +For further information, please see: - [Deployment](./sections/deployment.md) - [Fixed Virtual Platform](./sections/deployment.md#fixed-virtual-platform) @@ -260,15 +268,14 @@ See: ## Implementing custom ML application -This section describes how to implement a custom Machine Learning application running -on a platform supported by the repository (Fixed Virtual Platform or an MPS3 board). +This section describes how to implement a custom Machine Learning application running on a platform supported by the +repository, either an FVP or an MPS3 board. -Cortex-M55 CPU and Ethos-U55 NPU Code Samples software project offers a simple way to incorporate additional -use-case code into the existing infrastructure and provides a build -system that automatically picks up added functionality and produces -corresponding executable for each use-case. +Both the *Cortex-M55* CPU and *Ethos-U55* NPU Code Samples software project offers a way to incorporate extra use-case +code into the existing infrastructure. It also provides a build system that automatically picks up added functionality +and produces corresponding executable for each use-case. -See: +For further information, please see: - [Implementing custom ML application](./sections/customizing.md) - [Software project description](./sections/customizing.md#software-project-description) @@ -288,15 +295,15 @@ See: ## Testing and benchmarking -See [Testing and benchmarking](./sections/testing_benchmarking.md). +Please refer to: [Testing and benchmarking](./sections/testing_benchmarking.md). ## Memory Considerations -See [Memory considerations](./sections/memory_considerations.md) +Please refer to: [Memory considerations](./sections/memory_considerations.md) ## Troubleshooting -See: +For further information, please see: - [Troubleshooting](./sections/troubleshooting.md) - [Inference results are incorrect for my custom files](./sections/troubleshooting.md#inference-results-are-incorrect-for-my-custom-files) @@ -304,7 +311,7 @@ See: ## Appendix -See: +Please refer to: - [Appendix](./sections/appendix.md) - [Cortex-M55 Memory map overview](./sections/appendix.md#arm_cortex_m55-memory-map-overview-for-corstone_300-reference-design) -- cgit v1.2.1