1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
|
# Arm(R) Ethos(TM)-U core platform
Arm(R) Ethos(TM)-U core platform is provided as an example of how to produce a
firmware binary for a given target platform. This software is primarily intended
for guidance, to demonstrate how to boot up a firmware binary and how to run an
inference on an Arm Ethos-U compatible platform.
This repository contains target specific files, like linker scripts. Target
agnostic software components are provided in the
[core_software](https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ethos-u-core-software)
repository.
# Targets
## Arm(R) Corstone(TM)-300
The
[Arm(R) Corstone(TM)-300](https://developer.arm.com/ip-products/subsystem/corstone/corstone-300)
is a reference design of how to to build a secure System on Chip (SoC). A fixed
virtual platform (FVP) of the Arm Corstone-300 including the Arm Ethos-U can be
downloaded from the Ecosystem page at
[developer.arm.com](https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps).
### Building
Building with default settings requires CMake for the configuration and make for
building. This will produce an elf file which can be run on the FVP.
```
$ cmake -B build/corstone-300 targets/corstone-300
$ cd build/corstone-300
$ make
```
It is also possible to build with a different toolchain.
```
$ cmake -B build/corstone-300 targets/corstone-300 -DCMAKE_TOOLCHAIN_FILE=$PWD/cmake/toolchain/arm-none-eabi-gcc.cmake
$ cd build/corstone-300
$ make
```
Please see [README_WINDOWS.md](README_WINDOWS.md) for additional information
regarding building on a Windows system.
### Testing
Assuming that the Corstone-300 FVP has been downloaded, installed and placed in
the PATH variable. Then the software binaries can be tested like this.
```
$ ctest
```
Individual applications can also be run directly with the FVP, for example like
this.
```
$ FVP_Corstone_SSE-300_Ethos-U55 applications/freertos/freertos.elf
```
# Multi NPU
The Tensorflow Lite for Microcontrollers (TFLu) framework supports running
multiple parallel inferences. Each parallel inference requires a TFLu arena
(costs memory) and a stack (requires an RTOS). The examples provided in this
repo are implemented in the application layer, which means that any RTOS could
be used.
The Ethos-U NPU driver is implemented in plain C. To enable thread safety in a
multi-threading environment the driver defines a set of weak functions that the
application is expected to override, providing implementations for mutex and
semaphore primitives.
The weak function can be found in
[ethosu_driver.c](https://git.mlplatform.org/ml/ethos-u/ethos-u-core-driver.git/tree/src/ethosu_driver.c?id=35b5d0eebf9709a3439d362a0b53d6270cbc4a94#n173).
An example based on FreeRTOS how to override and implement these functions can
be found in
[applications/freertos/main.cpp](https://git.mlplatform.org/ml/ethos-u/ethos-u-core-platform.git/tree/applications/freertos/main.cpp?id=991af2bd8fb6c79dfb317837353857f34a727b17#n108).
The sequence diagram below illustrates the call stack for a multi NPU system.
Please note how the `ethosu_mutex_*` and `ethosu_semaphore_*` functions are
implemented in the application layer. Mutexes are used for thread safety and
semaphores for sleeping.
## Tradeoffs
A single Cortex-M is capable of driving multiple Ethos-U. What the optimal
number of Ethos-U is, that is impossible to tell without knowing which network
to run or without detailed knowledge about the limitations of the embedded
system.
Each parallel inference requires an arena. The arena should for optimal
performance be placed in a high bandwidth low latency memory like SRAM, which is
a cost that has to be considered. The size of the arena varies greatly depending
on the network.
For networks that map fully to Ethos-U, the memory bandwidth might become a
limiting factor. For networks that run partly in software, the Cortex-M might
become the limiting factor. The placement of the TFLu model and arena (flash,
DRAM, SRAM, etc) will also have a big impact on the performance.
# Startup
The applications in this repo use
[CMSIS Device](https://github.com/ARM-software/CMSIS_5/tree/develop/Device/) to
startup the Cortex-M. The standard procedure is to copy and modify the CMSIS
templates, but in this repo we have chosen to include the unmodified templates
directly from CMSIS.
The sequence diagram below describes what happens after the Cortex-M reset is
lifted, up until the execution enters the application `main()`.
## CMSIS Device
First thing that happens is that the CPU loads index 0 from the interrupt vector
into the SP register and index 1 into the PC register, and then starts executing
from the PC location.
Index 1 in the VTOR is referred to as the reset handler and is resposible for
initializing the CPU. If the CPU for example has a FPU or MVE extension, then
these are enabled.
## Compiler runtime
The entry function for the compiler runtime setup varies depending on which
compiler that is used. For Arm Clang this function is called `__main()`, not to
be confused with the application `main()`!
The runtime is responsible for initializing the memory segments and setting up
the runtime environment. Please refer to the compiler documentation for detailed
information about the runtime setup.
## Target
The [`init()`](targets/common/src/init.cpp) is defined as a constructor, which
will be called before the application `main()`. We use this constructor to run
`targetSetup()` to initialize the platform.
For each target there is a `targets/<target>` directory, which contains linker
scripts and code needed to setup the target. `targetSetup()` is implemented in
this folder and is responsible for initializing drivers, configuring the MPU,
enabling caches etc.
Adding a new target would involve creating a new `targets/<target>` directory,
providing linker scripts and implementing `targetSetup()`.
## Application
Finally the runtime calls application `main()`. Ideally the application code
should be generic and have no knowledge about which target it is executing on.
# License
The Arm Ethos-U core platform is provided under an Apache-2.0 license. Please
see [LICENSE.txt](LICENSE.txt) for more information.
# Contributions
The Arm Ethos-U project welcomes contributions under the Apache-2.0 license.
Before we can accept your contribution, you need to certify its origin and give
us your permission. For this process we use the Developer Certificate of Origin
(DCO) V1.1 (https://developercertificate.org).
To indicate that you agree to the terms of the DCO, you "sign off" your
contribution by adding a line with your name and e-mail address to every git
commit message. You must use your real name, no pseudonyms or anonymous
contributions are accepted. If there are more than one contributor, everyone
adds their name and e-mail to the commit message.
```
Author: John Doe \<john.doe@example.org\>
Date: Mon Feb 29 12:12:12 2016 +0000
Title of the commit
Short description of the change.
Signed-off-by: John Doe john.doe@example.org
Signed-off-by: Foo Bar foo.bar@example.org
```
The contributions will be code reviewed by Arm before they can be accepted into
the repository.
# Security
Please see [Security](SECURITY.md).
# Trademark notice
Arm, Cortex, Corstone and Ethos are registered trademarks of Arm Limited (or its
subsidiaries) in the US and/or elsewhere.
|
