aboutsummaryrefslogtreecommitdiff
path: root/kernel/ethosu_inference.c
AgeCommit message (Collapse)Author
2024-02-12Restructure kernel driver source treeMikael Olsson
As a first step to have a clearer separation of the different parts of the kernel driver, the source files have been placed into separate directories according to their purpose and the different parts are only allowed to use headers from another part in the include folder. Files have been renamed accordingly to namespace them by their purpose. Change-Id: I75e09ebf0002c99a22b6d4b09d34504d186c32b3 Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2024-01-16Fix inference cancellation behaviorMikael Olsson
There is currently a possible race when canceling an inference, where the inference may be the one currently running and a response will be sent later or the inference response is already in the mailbox queue. To handle these cases, the kernel will no longer process the inference response for an inference that has been marked as aborted or is in the process of being aborted. Change-Id: Ifcd86591b09075f994ed2e903cb11ba7c0ee0418 Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-12-20Change PMU event counter values to use 64-bitMikael Olsson
The PMU event counter value is an accumulation of 32-bit values during the inference and to ensure the total value fits in the rpmsg message and UAPI, the variable holding the value has been changed to 64-bit. The driver library, Python wrapper and inference runner have been changed accordingly to support the 64-bit values. Change-Id: I09a8e45eb75800c8a787f83abff5a3693148cc15 Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-11-14Fix NPU driver inference poll callback return typeMikael Olsson
The poll file operation function pointer is expected to return the __poll_t type but the currently implemented poll inference function returns an unsigned int. This breaks the type safety and causes Sparse to generate a warning. To resolve this, the return type has been changed to __poll_t and the function now uses with EPOLL constants that share the same type. Change-Id: I41dadb758c7c4b42b431d96a94c6b2cc9f960013 Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-11-06Change create network UAPI to take a user bufferMikael Olsson
To not allow the buffer for a network instance to be changed after creation, the create network UAPI will now take the network model data as a user buffer. The content of the user buffer is copied into an internally allocated DMA buffer that cannot be accessed by the user. This breaks the current API so the Linux kernel NPU driver version and the driver library version have been given major version bumps. All the tests, documentation and other applications affected by the changes have been updated accordingly. Change-Id: I25c785d75a24794c3db632e4abe5cfbb1c7ac190 Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-11-06Remove buffer capacity, offset and resize in UAPIMikael Olsson
The UAPI no longer supports the buffer capacity, offset and resize functionality. Instead, the UAPI now only accepts a fixed size given at the creation of the buffer. This change was made because the features were not used and made the buffer handling more complicated. The user knows how big buffers they need for their networks so they don't need resize support or partial buffer usage support by having separate size and capacity with an offset. Without these features, the buffer instance no longer needs any IOCTL call support so it has been removed. However, to still be able to check the size of a buffer from its file descriptor, seek support has been implemented so lseek and similar functions can be used to get the size. The driver library's clear function that previously only reset the size and offset values of the buffer will now clear the buffer content instead. These are breaking changes so the Linux kernel NPU driver version and the driver library version have been given major version bumps. All the tests and other applications affected by these changes have been updated accordingly. Change-Id: Ifc34cf04724a95853ad23fd7398dd286f73bcdab Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-11-01Decrease log verbosity in the kernel modulesLedion Daja
Changed several logs level from info to dbg and removed redundant or uninformative logging in order to reduce verbosity of the kernel modules. Change-Id: Ie9ff7f3ae6478007ea58547380b3ddfef5d280b4 Signed-off-by: Ledion Daja <ledion.daja@arm.com>
2023-09-07Fix mailbox clean up on inference request failureMikael Olsson
When an inference request fails, the mailbox message registered for the request is not unregistered, causing the mailbox's IDR to be left with an entry for the message with a dangling pointer to the freed inference. The dangling pointer later causes the mailbox clean up to crash when it attempts to call the fail callback for the IDR entry. The clean up for a failed inference request has now been fixed so it unregisters the message from the mailbox. Change-Id: Ida8015c88afee9cad074289be4d65947854c9e6b Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-09-07Initialize UAPI structs copied to user spaceMikael Olsson
To not leak any information from kernel space when IOCTL calls copy data from kernel space to user space, the source struct in the IOCTL calls is now initialized. Change-Id: I0825e82ccdb51ced747e160dd7385fa1ed227eaf Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-09-07Fix IOCTL calls returning the wrong valueMikael Olsson
In some cases, the IOCTL functions didn't update the return value variable so the wrong value was returned from the IOCTL calls. Change-Id: I56529f8fb03cec52dbc555d5ed3d4e68e4bfab70 Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-08-11Clear inference related data before freeing itMikael Olsson
To not leave behind any data in memory from buffers, inferences and networks, the memory allocated for them is now cleared before it is freed. Change-Id: I68c221b190e78f9b4c5022016c38f4ced5e78166 Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-08-09Fix concurrent access to IDR in kernel driverMikael Olsson
The IDR instance must be protected against concurrent access and the device mutex in the kernel driver should be used to protect it. However, the device mutex is not locked when an inference is released or when the mailbox is cleared which means the IDR is not protected in these instances. To resolve this, the missing mutex locks have been added and the functions using the IDR instance have been updated to make it clearer that the device mutex is expected to be locked when called. Change-Id: Id0b314db556836c36663d6481806b7c113e55e5f Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
2023-06-16Add type check when getting message by IDMikael Olsson
When the kernel driver handles incoming rpmsg messages, it uses the ID in the message header to find the corresponding mailbox message. The mailbox messages are stored as a generic message struct that is later cast to the specific message type. There is currently no type information stored in the generic message struct so only the ID is used to distinguish between the mailbox messages. This means if an incorrect ID is received that matches a mailbox message with a different type, the kernel driver will cast the message struct to the wrong type. Type information has now been added to the generic message struct and will be checked when trying to find the corresponding mailbox message so both the ID and type must be correct to find a matching message. Change-Id: Ifdbceea6ec4ae7078f424a32ce1ff5474bd22fef
2023-02-02Break circulare dependency on struct ethosu_deviceKristofer Jonsson
The 'struct ethosu_device' has been passed as argument to classes. This creates a circular dependency dependency and it gives all classes full visibility to all resources in the device struct. This patch removes the circular dependency. Using device_lock() and device_unlock() to for synchronization. Change-Id: I8322e6530c72d7bd67f48f411b4f14b612be2706
2023-02-02Converting Ethos-U driver to rpmsgKristofer Jonsson
The Ethos-U kernel driver has been converted from a platform driver with a custom firmware interface into a rpmsg driver. Change-Id: I9ae449f5e79eb02924e6630611d0893e5fec86be
2023-02-02Removing watchdog and resetKristofer Jonsson
Removing watchdog and firmware reset as a preparations for the migrations to rpmsg. Change-Id: Ic1053e3f4301ecadbde8c59dbaed437625a0a5ea
2022-06-30Improved logging22.08-rc222.08-rc122.08Kristofer Jonsson
Add logging class for driver library. Log severity can be configured using an environment variable. Add information to prints in kernel space. Change-Id: I19a1078869733746726515a6cafb79110314066d
2022-06-30Fix bug in inference releaseKristofer Jonsson
Once the inference file descriptor has been created the inference release must be handled by the file descriptor release callback. Change-Id: Ic514983e85821ea3893e11c5c682610f67d1ae31
2022-05-23Use ids for identifying messages sent to EthosU firmwareDavide Grohmann
Avoid to use pointers to kernel memory as identify messages, prior this change, if memory is reused that can lead to unexpected conflicts. Remove ref counting from capabilities and network_info since memory is freed in only one place. Finally, extract the capabilities code in its own files. Change-Id: I9ab7257f3ab85fa3347be65b3739e0daf9b5a441
2022-05-05Add support for inference cancellationDavide Grohmann
Send cancel inference messages to the ethosu subsystem to abort inference execution there. Also mark inference as aborted in the linux driver stack itself, so pending inference messages are not resent when resetting the firmware. Change-Id: I244c2b119fd7995d14e3859815abf2a00c7f0583
2022-05-02Add support for rejected inference responsesDavide Grohmann
Change-Id: I62e13df69c712ed4cf70766429b4cb13fc9c9dcb
2022-04-04Reset firmwareKristofer Jonsson
Reset the firmware if it becomes unresponsive. Use ping to send keep alive requests. Only monitor ping and inference request messages. The other messages pass no resources to the firmware and can be cancelled without resetting the firmware. Change-Id: Ifbcc370f02d79a64f25598f11376a1dc84a7a066
2022-03-14Network infoKristofer Jonsson
Add UAPI and core message that allows user space space to fetch information about network models built into the firmware. Change-Id: Ic92529bce3edd0a5499e691a566bd065da2a72ad
2022-03-10Firmware resident modelKristofer Jonsson
Support referencing a network model by index that has been built into the firmware binary. Change-Id: Idd5294376ea82503dfeafe1203dcc0694d296dfe
2020-12-10Fix args order to PMU log messageJonny Svärd
Change-Id: Iaf6f26e12fdd4a6d8e975a304a7836b26222f270
2020-11-16Add PMU config to inference creationPer Åstrand
Add PMU configuration information to inference creation. Change-Id: Id8a69517a5d3e4822dbaf46f76cadb2700b3d981
2020-09-17Support inferences with multiple inputs and outputsKristofer Jonsson
Build flatbuffers library. Update network class to extract IFM and OFM dimensions from the tflite file. Update the uapi and core apis to support up to 16 IFM and OFM buffers per inference. Change-Id: I2f2f177aa4c2d5f9f50f23eb33c44e01ec2cbe09
2020-08-27Initial commit20.08-rc220.08Kristofer Jonsson
Change-Id: I14b6becc908a0ac215769c32ee9c43db192ae6c8