MLECO-2925: updated documentation with new repo structure and general guidance to add custom platform.

Signed-off-by: alexander <alexander.efremov@arm.com>
Change-Id: Ib2eb2b7460c0ee8161403e5b135cd8b5cd854334

3 files changed, 183 insertions, 72 deletions

diff --git a/docs/documentation.md b/docs/documentation.md
index 48db0a0..3555095 100644
--- a/docs/documentation.md
+++ b/docs/documentation.md
@@ -86,19 +86,25 @@ The repository has the following structure:
 ├── resources
 ├── /resources_downloaded/
 ├── scripts
+│   ├── platforms
+│   │   ├── mps3
+│   │   ├── native
+│   │   └── simple_platform
 │   └── ...
 ├── source
 │   ├── application
-│   │ ├── hal
-│   │ ├── main
-│   │ └── tensorflow-lite-micro
+│   │   ├── main
+│   │   └── tensorflow-lite-micro
+│   ├── hal
+│   ├── log
+│   ├── math
+│   ├── profiler
 │   └── use_case
-│     └── <usecase_name>
-│          ├── include
-│          ├── src
-│          └── usecase.cmake
+│       └── <usecase_name>
+│           ├── include
+│           ├── src
+│           └── usecase.cmake
 ├── tests
-│   └── ...
 └── CMakeLists.txt
 ```
 
@@ -118,6 +124,13 @@ What these folders contain:
 
 - `scripts`: Build and source generation scripts.
 
+- `scripts/cmake/platforms`: Platform build configuration scripts `build_configuration.cmake` are located here. 
+   These scripts are adding platform sources into the application build stream. The script has 2 functions:
+   * `set_platform_global_defaults` - to set platform source locations and other build options.
+   * `platform_custom_post_build` - to execute specific post build steps. For example, MPS3 board related script adds
+                                    board specific `images.txt` file creation and calls bin generation command. 
+                                    Native profile related script compiles unit-tests.
+
 - `source`: C/C++ sources for the platform and ML applications.
   > **Note:** Common code related to the `Ethos-U` NPU software framework resides in *application* subfolder.
 
@@ -125,16 +138,23 @@ What these folders contain:
 
   - `application`: All sources that form the *core* of the application. The `use-case` part of the sources depend on the
     sources themselves, such as:
-
-    - `hal`: Contains Hardware Abstraction Layer (HAL) sources, providing a platform agnostic API to access hardware
-      platform-specific functions.
-
     - `main`: Contains the main function and calls to platform initialization logic to set up things before launching
       the main loop. Also contains sources common to all use-case implementations.
 
     - `tensorflow-lite-micro`: Contains abstraction around TensorFlow Lite Micro API. This abstraction implements common
       functions to initialize a neural network model, run an inference, and access inference results.
-
+  
+  - `hal`: Contains Hardware Abstraction Layer (HAL) sources, providing a platform agnostic API to access hardware
+    platform-specific functions.
+
+  - `log`: Common to all code logging macros managing log levels.
+  
+  - `math`: Math functions to be used in ML pipelines. Some of them use CMSIS DSP for optimized execution on Arm CPUs.
+     It is a separate CMake project that is built into a static library `libarm_math.a`.         
+  
+  - `profiler`: profiling utilities code to collect and output cycle counts and PMU information.
+    It is a separate CMake project that is built into a static library `libprofiler.a`.
+        
   - `use_case`: Contains the ML use-case specific logic. Stored as a separate subfolder, it helps isolate the
     ML-specific application logic. With the assumption that the `application` performs the required setup for logic to
     run. It also makes it easier to add a new use-case block.
@@ -145,29 +165,27 @@ The HAL has the following structure:
 
 ```tree
 hal
-├── hal.c
+├── cmsis_device
+│   └── ...
+├── components
+│   └── ...
 ├── include
-│   └── ...
-└── platforms
-    ├── bare-metal
-    │   ├── bsp
-    │   │   ├── bsp-core
-    │   │   │   └── include
-    │   │   ├── bsp-packs
-    │   │   │   └── mps3
-    │   │   ├── cmsis-device
-    │   │   ├── include
-    │   │   └── mem_layout
-    │   ├── data_acquisition
-    │   ├── data_presentation
-    │   │   ├── data_psn.c
-    │   │   └── lcd
-    │   │       └── include
-    │   ├── images
-    │   ├── timer
-    │   └── utils
-    └── native
+│   └── ...
+├── platform
+│   ├── mps3
+│   └── simple
+├── profiles
+│   ├── bare-metal
+│   │   ├── bsp
+│   │   ├── data_acquisition
+│   │   ├── data_presentation
+│   │   ├── timer
+│   │   └── utils
+│   └── native
+├── CMakeLists.txt
+└── hal.c
 ```
+HAL is built as a separate project into a static library `libhal.a`. It is linked with use-case executable.
 
 What these folders contain:
 
@@ -176,33 +194,37 @@ What these folders contain:
     > **Note:** the files here and lower in the hierarchy have been written in C and this layer is a clean C/ + boundary
     > in the sources.
 
-- `platforms/bare-metal/data_acquisition`\
-  `platforms/bare-metal/data_presentation`\
-  `platforms/bare-metal/timer`\
-  `platforms/bare-metal/utils`:
-
-  These folders contain the bare-metal HAL support layer and platform initialization helpers. Function calls are routed
+- `cmsis_device` has a common startup code for Cortex-M based systems. The package defines interrupt vector table and
+  handlers. Reset handler - starting point of our application - is also defined here. This entry point is responsible
+  for the set-up before calling the user defined "main" function in the higher-level `application` logic.
+  It is a separate CMake project that is built into a static library `libcmsis_device.a`. It depends on a CMSIS repo
+  through `CMSIS_SRC_PATH` variable.
+  The static library is used by platform code.
+   
+- `components` directory contains drivers code for different devices used in platforms. Such as UART, LCD and others.
+  A platform can include those as sources in a build to enable usage of corresponding HW devices. Most of the use-cases
+  use UART and LCD, thus if you want to run default ML use-cases on a custom platform, you will have to add 
+  implementation for your devices here (or re-use existing code if it is compatible with your platform).
+
+- `platform/mps3`\
+  `platform/simple`:
+  These folders contain platform specific declaration and defines, such as, platform initialisation code, peripheral 
+  memory map, system registers, system specific timer implementation and other. 
+  Platform is built from selected components and configured cmsis device. The platform could be used with different
+  profiles. Profile is included into the platform build based on `PLATFORM_PROFILE` build parameter.
+  Platform code is a separate CMake project and it is built into a static library `libplatform-drivers.a`. It is linked 
+  into HAL library.
+
+- `profiles/bare-metal`\
+  `profiles/native`:
+  As mentioned before, profiles are added into platform build. Currently we support bare-metal and native profiles.    
+  bare-metal contains the HAL support layer and platform initialization helpers. Function calls are routed
   to platform-specific logic at this level. For example, for data presentation, an `lcd` module has been used. This
-  `lcd` module wraps the LCD driver calls for the actual hardware (for example, MPS3).
-
-- `platforms/bare-metal/bsp/bsp-packs`: The core low-level drivers (written in C) for the platform reside. For supplied
-  examples, this happens to be an MPS3 board. However, support can be added here for other platforms. The functions
-  defined in this space are wired to the higher-level functions under HAL and is like those in the
-  `platforms/bare-metal/` level).
-
-- `platforms/bare-metal/bsp/bsp-packs/mps3/include`\
-  `platforms/bare-metal/bsp/bsp-packs/mps3`: Contains the peripheral (LCD, UART, and timer) drivers specific to MPS3
-  board.
-
-- `platforms/bare-metal/bsp/bsp-core`\
-  `platforms/bare-metal/bsp/include`: Contains the BSP core sources common to all BSPs and includes a UART header.
-  However, the implementation of this is platform-specific, while the API is common. Also "re-targets" the standard
+  `lcd` module wraps the LCD driver calls for the actual hardware (for example, MPS3). Also "re-targets" the standard
   output and error streams to the UART block.
 
-- `platforms/bare-metal/bsp/cmsis-device`: Contains the CMSIS template implementation for the CPU and also device
-  initialization routines. It is also where the system interrupts are set up and the handlers are overridden. The main
-  entry point of a bare-metal application most likely resides in this space. This entry point is responsible for the
-  set-up before calling the user defined "main" function in the higher-level `application` logic.
+  Native profile allows to build application to be executed on a build machine, i.e. x86. It bypasses and stubs platform
+  devices replacing them with standard C or C++ library calls.
 
 - `platforms/bare-metal/bsp/mem_layout`: Contains the platform-specific linker scripts.
 
@@ -233,8 +255,11 @@ For detailed information, see: [Optimize model with Vela compiler](./sections/bu
 
 ## Building
 
-This section describes how to build the code sample applications from sources and includes illustrating the build
-options and the process.
+This section explains the build process and intra-project dependencies, describes how to build the code sample 
+applications from sources and includes illustrating the build options and the process.
+
+The following graph of source modules aims to explain better intra-project code and build execution dependencies.
+![intra-project dependencies](./media/build_graph.png)
 
 The project can be built for MPS3 FPGA and FVP emulating MPS3. Using default values for configuration parameters builds
 executable models that support the *Ethos-U* NPU.
@@ -304,6 +329,7 @@ For further information, please see:
   - [Reading user input from console](./sections/customizing.md#reading-user-input-from-console)
   - [Output to MPS3 LCD](./sections/customizing.md#output-to-mps3-lcd)
   - [Building custom use-case](./sections/customizing.md#building-custom-use_case)
+  - [Adding custom platform support](./sections/customizing.md#adding-custom-platform-support)
 
 ## Testing and benchmarking
 
diff --git a/docs/media/build_graph.png b/docs/media/build_graph.png
new file mode 100644
index 0000000..b4ce9fc
--- /dev/null
+++ b/docs/media/build_graph.png
diff --git a/docs/sections/customizing.md b/docs/sections/customizing.md
index 3bf9b26..17b8040 100644
--- a/docs/sections/customizing.md
+++ b/docs/sections/customizing.md
@@ -16,6 +16,7 @@
   - [Reading user input from console](./customizing.md#reading-user-input-from-console)
   - [Output to MPS3 LCD](./customizing.md#output-to-mps3-lcd)
   - [Building custom use-case](./customizing.md#building-custom-use_case)
+  - [Adding custom platform support](./customizing.md#adding-custom-platform-support)
 
 This section describes how to implement a custom Machine Learning application running on Arm® *Corstone™-300* based FVP
 or on the Arm® MPS3 FPGA prototyping board.
@@ -35,23 +36,38 @@ The following sign indicates the important conventions to apply:
 As mentioned in the [Repository structure](../documentation.md#repository-structure) section, project sources are:
 
 ```tree
+├── dependencies
 ├── docs
 │ ├── ...
 │ └── Documentation.md
+├── model_conditioning_examples
 ├── resources
 │ └── img_class
 │      └── ...
+├── /resources_downloaded/
+│ └── img_class
+│      └── ...
 ├── scripts
-│ └── ...
+│   ├── platforms
+│   │   ├── mps3
+│   │   ├── native
+│   │   └── simple_platform
+│   └── ...
 ├── source
-│ ├── application
-│ │ ├── hal
-│ │ ├── main
-│ │ └── tensorflow-lite-micro
-│ └── use_case
-│     └──img_class
-├── CMakeLists.txt
-└── Readme.md
+│   ├── application
+│   │   ├── main
+│   │   └── tensorflow-lite-micro
+│   ├── hal
+│   ├── log
+│   ├── math
+│   ├── profiler
+│   └── use_case
+│       └── <usecase_name>
+│           ├── include
+│           ├── src
+│           └── usecase.cmake
+├── tests
+└── CMakeLists.txt
 ```
 
 Where the `source` folder contains C/C++ sources for the platform and ML applications. Common code related to the
@@ -342,13 +358,14 @@ Now define the `main_loop` function with the signature described in [Main loop f
 
 ```C++
 #include "hal.h"
+#include "log_macros.h"
 
 void main_loop(hal_platform& platform) {
   printf("Hello world!");
 }
 ```
 
-The preceeding code is already a working use-case. If you compile and run it (see [Building custom usecase](./customizing.md#building-custom-use-case)),
+The preceding code is already a working use-case. If you compile and run it (see [Building custom usecase](./customizing.md#building-custom-use-case)),
 then the application starts and prints a message to console and exits straight away.
 
 You can now start filling this function with logic.
@@ -415,6 +432,7 @@ For example:
 
 ```C++
 #include "HelloWorldModel.hpp"
+#include "log_macros.h"
 
 bool arm::app::HelloWorldModel::EnlistOperations() {
 
@@ -483,6 +501,7 @@ The following code adds inference invocation to the main loop function:
 ```C++
 #include "hal.h"
 #include "HelloWorldModel.hpp"
+#include "log_macros.h"
 
   void main_loop(hal_platform& platform) {
 
@@ -548,7 +567,7 @@ The code snippet has several important blocks:
   data type.
 
   ```C++
-  Const uint32_t tensorSz = outputTensor->bytes ;
+  Const uint32_t tensorSz = outputTensor->bytes;
 
   const uint8_t *outputData = tflite::GetTensorData<uint8>(outputTensor);
   ```
@@ -570,7 +589,7 @@ profiler.PrintProfilingResult();
 
 ## Printing to console
 
-The preceding examples used some function to print messages to the console.
+The preceding examples used some function to print messages to the console. To use them, include `log_macros.h` header.
 
 However, for clarity, here is the full list of available functions:
 
@@ -711,3 +730,69 @@ As a result, the file `ethos-u-hello_world.axf` is created. The MPS3 build also
 directory with binaries and the file `sectors/images.txt` to be copied to the MicroSD card on the board.
 
 The next section of the documentation covers: [Testing and benchmarking](testing_benchmarking.md).
+
+## Adding custom platform support
+
+Platform build configuration script `build_configuration.cmake` is the main build entry point for platform sources. 
+It is used by top level CMakeLists.txt script to add a platform into the public build stream.
+Platform build configuration script must have 2 functions:
+ * `set_platform_global_defaults` - to set platform source locations and other build options.
+ * `platform_custom_post_build` - to execute specific post build steps.
+ 
+The function `set_platform_global_defaults` must set `PLATFORM_DRIVERS_DIR` variable
+```
+    set(PLATFORM_DRIVERS_DIR "${HAL_PLATFORM_DIR}/mps3" PARENT_SCOPE)
+```
+location of the platform library sources.
+
+> **Convention:**  The default search path for platform build configuration scripts is in `scripts/cmake/platforms`.
+> The directory name for a platform configuration script is also used as a value for `TARGET_PLATFORM`
+> build option. For example:
+> `scripts/cmake/platforms/my_platform` results in having `my_platform` as a `TARGET_PLATFORM` option for the build.
+
+The function `platform_custom_post_build` could be used to add platform specific post use-case application build steps. 
+
+Repository's root level CMakeLists.txt calls common utility function `add_platform_build_configuration(TARGET_PLATFORM ${TARGET_PLATFORM})` 
+to add given target platform to the build stream. The function finds the script and includes 
+`build_configuration.cmake` file. After that public build can invoke
+* `set_platform_global_defaults`
+* `platform_custom_post_build`
+for a specified platform.
+
+New platform sources, that are pointed to by `PLATFORM_DRIVERS_DIR` variable, could be placed anywhere, conventional location 
+is `source/hal/platform`. Platform must be a separate CMake project with CMakeLists.txt script and build into a static
+library `libplatform-drivers.a`.
+HAL expects platform to have `platfrom_drivers.h`  header file with required interfaces for included peripherals.
+
+If the new platform uses existing cmsis device project then it should be linked with it like this:
+```
+    target_link_libraries(${PLATFORM_DRIVERS_TARGET} PUBLIC cmsis_device)
+```
+Cmsis device exposes an entry point `--entry Reset_Handler` as a link interface.
+
+If the new platform defines custom cmsis device and has custom application entry point,
+it must tell linker about it like this:
+```
+    target_link_options(${PLATFORM_DRIVERS_TARGET} INTERFACE --entry <custom handler name>)
+```
+
+Most of the ML use-case applications use UART and LCD, thus it is a hard requirement to implement at least stubs for
+those. UART driver must implement functions from `uart_stdout.h` header. LCD driver must provide implementation for 
+functions declared in `glcd_mps3.h` header. For stubs examples, please, see simple platform sources.
+
+If the new platform does not use UART, it is possible to run application with semi-hosting enabled - printf 
+statements will be shown in the host machine console. Please, comment out all content of the 
+`source/hal/profiles/bare-metal/bsp/retarget.c` file in this case.
+
+Examples of the UART and LCD drivers implementation could be found here: `source/hal/components`.
+
+Linker scripts for armclang and GCC should be added. The location of the files is on your discretion. The new 
+platform build configuration script must add it in the `platform_custom_post_build` function like this:
+```
+    add_linker_script(
+            ${PARSED_TARGET_NAME}                  # Target
+            ${CMAKE_SCRIPTS_DIR}/platforms/mps3    # linker scripts directory path
+            ${LINKER_SCRIPT_NAME})                 # Name of the file without suffix
+```
+
+Please see existing platforms sources and build scripts for more details. 
\ No newline at end of file