aboutsummaryrefslogtreecommitdiff
path: root/utils/inference_runner/inference_runner.cpp
AgeCommit message (Collapse)Author
2024-02-19Fix unnecessary string copying in inference runnerMikael Olsson
To avoid copying strings unnecessarily in the inference runner the functions have been changed to take the strings as const reference rather than copying them. Change-Id: I173e37c7f7b08ada5d421bb9008a4bf36284f6c5 Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2024-02-19Add NPU device type to capabilitiesMikael Olsson
In preparation to support the Linux kernel driver directly managing the NPU without a subsystem. A NPU device type has been added to the capabilities to be able to specify what kind of NPU is used. Change-Id: I45345b7d75d234f821d70b858453d3af2a99194e Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-12-20Change PMU event counter values to use 64-bitMikael Olsson
The PMU event counter value is an accumulation of 32-bit values during the inference and to ensure the total value fits in the rpmsg message and UAPI, the variable holding the value has been changed to 64-bit. The driver library, Python wrapper and inference runner have been changed accordingly to support the 64-bit values. Change-Id: I09a8e45eb75800c8a787f83abff5a3693148cc15 Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-11-06Change create network UAPI to take a user bufferMikael Olsson
To not allow the buffer for a network instance to be changed after creation, the create network UAPI will now take the network model data as a user buffer. The content of the user buffer is copied into an internally allocated DMA buffer that cannot be accessed by the user. This breaks the current API so the Linux kernel NPU driver version and the driver library version have been given major version bumps. All the tests, documentation and other applications affected by the changes have been updated accordingly. Change-Id: I25c785d75a24794c3db632e4abe5cfbb1c7ac190 Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-11-06Remove buffer capacity, offset and resize in UAPIMikael Olsson
The UAPI no longer supports the buffer capacity, offset and resize functionality. Instead, the UAPI now only accepts a fixed size given at the creation of the buffer. This change was made because the features were not used and made the buffer handling more complicated. The user knows how big buffers they need for their networks so they don't need resize support or partial buffer usage support by having separate size and capacity with an offset. Without these features, the buffer instance no longer needs any IOCTL call support so it has been removed. However, to still be able to check the size of a buffer from its file descriptor, seek support has been implemented so lseek and similar functions can be used to get the size. The driver library's clear function that previously only reset the size and offset values of the buffer will now clear the buffer content instead. These are breaking changes so the Linux kernel NPU driver version and the driver library version have been given major version bumps. All the tests and other applications affected by these changes have been updated accordingly. Change-Id: Ifc34cf04724a95853ad23fd7398dd286f73bcdab Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-08-09Add version to driver library23.08-rc1Mikael Olsson
A version has been added to the driver library so users can check if they are compatible with the driver library in use. The Python wrapper has been updated accordingly to make the version information available. Change-Id: I8affbf7068c057f7103adf14c9e4a331d547fbcc Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-08-09Add kernel driver version check in driver libraryMikael Olsson
The driver library will now check that it's compatible with the kernel driver in use by checking the kernel driver's version. The kernel driver version has also been made available to the library users and the Python wrapper has been updated accordingly. Change-Id: Ieae8c0bfc323f945038e7264eceeab90c833f76d Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-08-09Remove protocol version request IOCTL from UAPIMikael Olsson
The current implementation of the protocol version request IOCTL call doesn't return any version information and the protocol version is not needed by any userspace program as it's for the communication between the Linux kernel NPU driver and firmware. The IOCTL call has therefore been removed from the UAPI and the inference runner has been updated accordingly. Change-Id: I9ed09e55d9522996a182f237c8a6db2f45bc40a9 Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2022-05-05Add support for inference cancellationDavide Grohmann
Send cancel inference messages to the ethosu subsystem to abort inference execution there. Also mark inference as aborted in the linux driver stack itself, so pending inference messages are not resent when resetting the firmware. Change-Id: I244c2b119fd7995d14e3859815abf2a00c7f0583
2022-05-02Add support for rejected inference responsesDavide Grohmann
Change-Id: I62e13df69c712ed4cf70766429b4cb13fc9c9dcb
2022-03-14Network infoKristofer Jonsson
Add UAPI and core message that allows user space space to fetch information about network models built into the firmware. Change-Id: Ic92529bce3edd0a5499e691a566bd065da2a72ad
2022-03-10Firmware resident modelKristofer Jonsson
Support referencing a network model by index that has been built into the firmware binary. Change-Id: Idd5294376ea82503dfeafe1203dcc0694d296dfe
2021-10-27Improve granularity of wait call in Inference classDavide Grohmann
It now accepts a timeout expressed in nanoseconds. Change-Id: I77fb89c33dc117f846b86494883548ef3241f0ab
2021-08-13Do not expose device specific hw config in capabilities21.08-rc1Davide Grohmann
Change-Id: Ia90e2d74c3e071c6e3de381cfada9a5a155d7529
2021-08-12Fix: do not attempt to read data from mailbox when length is 0Davide Grohmann
Also add a call to send a ping message in the inference_runner example app. Change-Id: Ia2b4abf5fec8285c320ec0f28df167ff2ea13d82
2021-06-10Add support for handling capabilities requestsDavide Grohmann
Change-Id: Id5aa197312c88b0c448dc085d8477ed67da24724
2021-03-31Update license headersAnton Moberg
Update all license headers to the correct years. Change-Id: Ie1aee82af91c744a3cdb949044334ade0d834c16
2021-01-26Improve mailbox message handlingJonny Svärd
Introduce a 32b magic for each message. Verify the magic for all incoming messages. Add reset function - in case of protocol error, effectively reset/empty the incoming queue. Add an error message type and message Add version request/response Verify payload length of responses (when applicable) Change-Id: I8aadd4012024492533d52e2cdb38630fce5c36e2
2020-11-16Configure PMUs from inference_runner20.11-rc2Per Åstrand
Change-Id: I3291de22a5461e285b8750094aa1c65716374bd7
2020-10-16Clang formatKristofer Jonsson
Change-Id: I2ebac7df9c4716be7fee10ce4cba350e40860ca2
2020-09-17Support inferences with multiple inputs and outputsKristofer Jonsson
Build flatbuffers library. Update network class to extract IFM and OFM dimensions from the tflite file. Update the uapi and core apis to support up to 16 IFM and OFM buffers per inference. Change-Id: I2f2f177aa4c2d5f9f50f23eb33c44e01ec2cbe09
2020-08-27Initial commit20.08-rc220.08Kristofer Jonsson
Change-Id: I14b6becc908a0ac215769c32ee9c43db192ae6c8