From c081e5954cd92165b139488e76bdfef1402acee6 Mon Sep 17 00:00:00 2001 From: Mikael Olsson Date: Mon, 30 Oct 2023 11:10:56 +0100 Subject: Change create network UAPI to take a user buffer To not allow the buffer for a network instance to be changed after creation, the create network UAPI will now take the network model data as a user buffer. The content of the user buffer is copied into an internally allocated DMA buffer that cannot be accessed by the user. This breaks the current API so the Linux kernel NPU driver version and the driver library version have been given major version bumps. All the tests, documentation and other applications affected by the changes have been updated accordingly. Change-Id: I25c785d75a24794c3db632e4abe5cfbb1c7ac190 Signed-off-by: Mikael Olsson --- README.md | 16 ++-- docs/driver_library_component.puml | 3 +- docs/driver_library_component.svg | 16 ++-- docs/driver_library_sequence.puml | 11 +-- docs/driver_library_sequence.svg | 18 +---- docs/kernel_network.puml | 8 +- docs/kernel_network.svg | 15 ++-- driver_library/include/ethosu.hpp | 10 +-- driver_library/python/README.md | 14 ++-- .../ethosu_driver/_utilities/driver_utilities.py | 3 +- .../python/src/ethosu_driver/swig/driver.i | 51 ++++++++----- .../src/ethosu_driver/swig/typemaps/buffer.i | 23 +++--- driver_library/python/test/test_driver.py | 86 +++++++++++++--------- driver_library/src/ethosu.cpp | 15 ++-- kernel/ethosu_device.c | 3 +- kernel/ethosu_inference.c | 4 +- kernel/ethosu_mailbox.c | 14 ++-- kernel/ethosu_mailbox.h | 14 ++-- kernel/ethosu_network.c | 46 ++++++++---- kernel/ethosu_network.h | 7 +- kernel/ethosu_network_info.c | 4 +- kernel/uapi/ethosu.h | 18 +++-- tests/cancel_inference_test.cpp | 8 +- tests/run_inference_test.cpp | 48 +++++++++--- utils/inference_runner/inference_runner.cpp | 13 ++-- 25 files changed, 259 insertions(+), 209 deletions(-) diff --git a/README.md b/README.md index 1b9a692..cdab1d4 100644 --- a/README.md +++ b/README.md @@ -219,7 +219,7 @@ dispatching inferences to the Ethos-U kernel driver. As the component diagram below illustrates the network is separated from the inference, allowing multiple inferences to share the same network. The buffer -class is used to store any data. +class is used to store IFM and OFM data. ![Driver library](docs/driver_library_component.svg "Driver library component diagram") @@ -232,9 +232,9 @@ The `Device` class opens a file descriptor to the device node `/dev/ethosu`. This file descriptor is used to issue IOCTL request to kernel space to create buffers and networks. -The `Network` class uses the `Device` object to create a new network object. The -network model is stored in a `Buffer` that the network parses to discover the -dimensions of the network model. +The `Network` class uses the `Device` object to create a new network object. +The network model is provided in a user buffer that is copied into an internal +buffer and parsed to discover the dimensions of the network model. The `Inference` class uses the `Network` object to create an inference. The array of IFM `Buffers` need to be populated with data before the inference @@ -259,13 +259,11 @@ for reading and/or writing. ## Network Creating a network assumes that the device node has already been opened, and -that a buffer has been allocated and populated with the network model. +that a user buffer populated with the network model is available. A new network is created by issuing an IOCTL command on the device node file -descriptor. A file descriptor to a buffer - containing the network model - is -passed in the IOCTL data. The network class increases the reference count on the -buffer, preventing the buffer from being freed before the network object has -been destructed. +descriptor. A pointer to the user buffer - containing the network model - and +its size is passed in the IOCTL data. ![Create network](docs/kernel_network.svg "Create network") diff --git a/docs/driver_library_component.puml b/docs/driver_library_component.puml index 1b640e2..6e39456 100644 --- a/docs/driver_library_component.puml +++ b/docs/driver_library_component.puml @@ -11,8 +11,7 @@ inf -> net inf -> buf: IFM and OFM net --> dev -net --> buf: Network model buf -> dev -@enduml \ No newline at end of file +@enduml diff --git a/docs/driver_library_component.svg b/docs/driver_library_component.svg index bcb43f9..715da50 100644 --- a/docs/driver_library_component.svg +++ b/docs/driver_library_component.svg @@ -1,4 +1,12 @@ -InferenceNetworkBufferDeviceIFM and OFMNetwork modelInferenceNetworkBufferDevice