1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
|
# Testing and benchmarking
- [Testing and benchmarking](./testing_benchmarking.md#testing-and-benchmarking)
- [Testing](./testing_benchmarking.md#testing)
- [Benchmarking](./testing_benchmarking.md#benchmarking)
## Testing
The `tests` folder uses the following structure:
```tree
.
├── common
│ └── ...
├── use_case
│ ├── <usecase1>
│ │ └── ...
│ ├── <usecase2>
│ │ └── ...
└── utils
└── ...
```
The folders contain the following information:
- `common`: The tests for generic and common application functions.
- `use_case`: Every use-case specific test in their respective folders.
- `utils`: Utility sources that are only used within the tests.
When [configuring](./building.md#configuring-native-unit_test-build) and
[building](./building.md#building-the-configured-project) for your `native` target platform, the results of the build is
placed under `<build folder>/bin/` folder. For example:
```tree
.
├── arm_ml_embedded_evaluation_kit-<usecase1>-tests
├── arm_ml_embedded_evaluation_kit-<usecase2>-tests
├── ethos-u-<usecase1>
└── ethos-u-<usecase1>
```
To view all the available tests to run, use the following command in the `<build folder>`:
```commandline
ctest -N
```
Sample output:
```commandline
Test #1: ad-tests
Test #2: asr-tests
Test #3: img_class-tests
Test #4: kws-tests
Test #5: kws_asr-tests
Test #6: noise_reduction-tests
Test #7: object_detection-tests
Test #8: vww-tests
Total Tests: 8
```
To execute a specific unit-test from the above list, in addition to the common tests, run the following command in the `<build folder>`:
```commandline
ctest -R <test_name>
```
To run every test that has been built, run the following command in the `<build folder>`:
```commandline
ctest
```
Sample output:
```log
Start 1: ad-tests
1/8 Test #1: ad-tests ......................... Passed 0.17 sec
Start 2: asr-tests
2/8 Test #2: asr-tests ........................ Passed 3.04 sec
Start 3: img_class-tests
3/8 Test #3: img_class-tests .................. Passed 0.49 sec
Start 4: kws-tests
4/8 Test #4: kws-tests ........................ Passed 7.52 sec
Start 5: kws_asr-tests
5/8 Test #5: kws_asr-tests .................... Passed 2.85 sec
Start 6: noise_reduction-tests
6/8 Test #6: noise_reduction-tests ............ Passed 16.41 sec
Start 7: object_detection-tests
7/8 Test #7: object_detection-tests ........... Passed 0.58 sec
Start 8: vww-tests
8/8 Test #8: vww-tests ........................ Passed 0.07 sec
Total Test time (real) = 34.71 sec
```
> **Note:** Test outputs could contain `[ERROR]` messages. This is OK as they are coming from negative scenarios tests.
## Benchmarking
Profiling is enabled by default when configuring the project. Profiling enables you to display:
- NPU event counters when the Arm® *Ethos™-U* NPU is enabled (see `ETHOS_U_NPU_ENABLED` in [Build options](./building.md#build-options) )
- Total cycle: The number of NPU cycles
- Active cycles: number of NPU cycles that were used for computation
- Idle cycles: number of cycles for which the NPU was idle
- AXI0 read beats: The number of AXI beats with read transactions from AXI0 bus. AXI0 is the bus where
*Ethos-U* NPU reads and writes to the computation buffers, activation buf, or tensor arenas.
- AXI0 write beats: The number of AXI beats with write transactions to AXI0 bus.
- AXI1 read beats: The number of AXI beats with read transactions from AXI1 bus. AXI1 is the bus where
*Ethos-U* NPU reads the model. So, read-only.
- If CPU profiling is enabled, the CPU cycle counts and the time elapsed, in milliseconds, for inferences performed. For
further information, please refer to the `CPU_PROFILE_ENABLED` in [Build options](./building.md#build-options).
> **Note:** Only do this when running on a physical FPGA board as the FVP does not contain a cycle-approximate or
> cycle-accurate *Cortex-M* model.
For example:
- On the FVP:
```log
INFO - Profile for Inference:
INFO - NPU AXI0_RD_DATA_BEAT_RECEIVED beats: 628122
INFO - NPU AXI0_WR_DATA_BEAT_WRITTEN beats: 135087
INFO - NPU AXI1_RD_DATA_BEAT_RECEIVED beats: 62870
INFO - NPU ACTIVE cycles: 1081007
INFO - NPU IDLE cycles: 626
INFO - NPU TOTAL cycles: 1081634
```
- For the MPS3 platform, the time duration in milliseconds is also reported when `-DCPU_PROFILE_ENABLED=1` is added to
CMake configuration command, like so:
```log
INFO - Profile for Inference:
INFO - NPU AXI0_RD_DATA_BEAT_RECEIVED beats: 628122
INFO - NPU AXI0_WR_DATA_BEAT_WRITTEN beats: 135087
...
INFO - Time ms: 210
```
The next section of the documentation refers to: [Memory Considerations](memory_considerations.md).
|
