| Age | Commit message (Collapse) | Author |
|---|
|
To make it easier to distinguish where an error occurred in the NPU
driver, additional logging has been added to the error cases.
Change-Id: I9d744e7adf3ba3098d456d1f3c2859aa0ce66de4
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
The device types in the UAPI have been renamed to follow the UAPI
namespacing and driver library validation has been added to ensure the
driver library and UAPI device enum values match.
The kernel and driver library version have been given a major bump for
the introduction of the device type in the capabilities.
Change-Id: Ib782b0bd7a8641db88521f8be399f709d2d73816
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
In preparation to support the Linux kernel driver directly managing the
NPU without a subsystem. A NPU device type has been added to the
capabilities to be able to specify what kind of NPU is used.
Change-Id: I45345b7d75d234f821d70b858453d3af2a99194e
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
Macros and types have been renamed accordingly to namespace them by
their usage in the new kernel driver source tree structure.
This is done in a separate commit from the restructuring to avoid Git
from seeing some of the moved files as new ones because they have been
both moved and modified at the same time and thus losing the connection
in the history.
Change-Id: Icd4d8e8c76779479b5b46a55bf1d4f78a629c281
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
As a first step to have a clearer separation of the different parts of
the kernel driver, the source files have been placed into separate
directories according to their purpose and the different parts are only
allowed to use headers from another part in the include folder.
Files have been renamed accordingly to namespace them by their purpose.
Change-Id: I75e09ebf0002c99a22b6d4b09d34504d186c32b3
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
There is currently a possible race when canceling an inference, where
the inference may be the one currently running and a response will be
sent later or the inference response is already in the mailbox queue. To
handle these cases, the kernel will no longer process the inference
response for an inference that has been marked as aborted or is in the
process of being aborted.
Change-Id: Ifcd86591b09075f994ed2e903cb11ba7c0ee0418
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
The PMU event counter value is an accumulation of 32-bit values during
the inference and to ensure the total value fits in the rpmsg message
and UAPI, the variable holding the value has been changed to 64-bit.
The driver library, Python wrapper and inference runner have been
changed accordingly to support the 64-bit values.
Change-Id: I09a8e45eb75800c8a787f83abff5a3693148cc15
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
The CMake files for the kernel modules have been updated to treat all
Sparse warnings as errors so that warnings cause the build to fail.
Change-Id: I3682afc3beea7244bb6e0831b2baf48b95611dbc
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
The poll file operation function pointer is expected to return the
__poll_t type but the currently implemented poll inference function
returns an unsigned int. This breaks the type safety and causes Sparse
to generate a warning.
To resolve this, the return type has been changed to __poll_t and the
function now uses with EPOLL constants that share the same type.
Change-Id: I41dadb758c7c4b42b431d96a94c6b2cc9f960013
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
Currently when the rpmsg channel name is copied from the rpmsg device,
the full size of the name array is given to strncpy. This means if there
is no null-terminator in that size, the name will be left unterminated.
To ensure that the name is always null-terminated, the size given to
strncpy is now decreased by one and the name array is zero initialized.
Change-Id: I73b4b597f51a63e5dac23945735f307cb1035e25
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
To not allow the buffer for a network instance to be changed after
creation, the create network UAPI will now take the network model data
as a user buffer. The content of the user buffer is copied into an
internally allocated DMA buffer that cannot be accessed by the user.
This breaks the current API so the Linux kernel NPU driver version and
the driver library version have been given major version bumps. All the
tests, documentation and other applications affected by the changes have
been updated accordingly.
Change-Id: I25c785d75a24794c3db632e4abe5cfbb1c7ac190
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
To allow the NPU kernel driver to allocate and use DMA memory internally
without creating a buffer instance, the DMA memory management has been
split out from the buffer code.
Change-Id: I46fdeee51b5ef786a54b8e7c866d137d91222724
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
The UAPI no longer supports the buffer capacity, offset and resize
functionality. Instead, the UAPI now only accepts a fixed size given at
the creation of the buffer. This change was made because the features
were not used and made the buffer handling more complicated. The user
knows how big buffers they need for their networks so they don't need
resize support or partial buffer usage support by having separate size
and capacity with an offset.
Without these features, the buffer instance no longer needs any IOCTL
call support so it has been removed. However, to still be able to check
the size of a buffer from its file descriptor, seek support has been
implemented so lseek and similar functions can be used to get the size.
The driver library's clear function that previously only reset the size
and offset values of the buffer will now clear the buffer content
instead.
These are breaking changes so the Linux kernel NPU driver version and
the driver library version have been given major version bumps. All the
tests and other applications affected by these changes have been updated
accordingly.
Change-Id: Ifc34cf04724a95853ad23fd7398dd286f73bcdab
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
Currently, the network create UAPI will assume that any network type
that isn't a buffer is an index. This means that the Linux kernel NPU
driver will accept any network type value and the user won't get any
feedback that they have specified an incorrect type.
To resolve this, the Linux kernel NPU driver will now return -EINVAL if
an unknown network type is given and a test has been added to validate
this behavior.
Change-Id: Ib7d9f5d5451897787981aae61a4e0a6650a73e05
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
Sparse is a semantic checker for C that can help find potential issues
in the kernel module code. To make use of the tool, the CMake files for
the kernel modules have been updated to enable the tool to run on all
the kernel module source files before they are built.
Change-Id: I114920448763b69eb0bdfb9ab6ec51ea8b940881
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
Changed several logs level from info to dbg and removed redundant or
uninformative logging in order to reduce verbosity of the kernel
modules.
Change-Id: Ie9ff7f3ae6478007ea58547380b3ddfef5d280b4
Signed-off-by: Ledion Daja <ledion.daja@arm.com>
|
|
In order to avoid struct layout misalignment issues between user and
kernel space, raising from size of enum members being implementation-
defined, the enum members of structs in the UAPI header file have now
been replaced with fixed-width integers.
Change-Id: I44a41c6d6669d116ca239da72676dd0539bf88b9
Signed-off-by: Ledion Daja <ledion.daja@arm.com>
|
|
Change-Id: I6405c2d5e614239af58e6f1e741e03ea8a19ba35
Signed-off-by: Ledion Daja <ledion.daja@arm.com>
|
|
To ensure that no compilation warnings goes unnoticed, the kernel
modules are now built with the -Werror flag so all warnings are treated
as errors.
Change-Id: Id688052bd297e366ab7ca07c508ca697ca03431e
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
Currently, when the NPU driver allocates buffers for the NPU subsystem,
it does not provide DMA addresses that can be used by the subsystem.
Instead, the DMA address is set to the Linux Kernel physical address and
must be translated by the firmware before use.
The correct DMA address is not provided because the device instance
created by the NPU driver has not been DMA configured so it does not
know about the subsystem's address requirements.
To resolve this, the device will now inherit the DMA configuration from
the parent rproc device.
Change-Id: Ibcbf30e812e72ebf8672bbe9dde3a9f9abdaa417
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
Currently, the mailbox uses the rpmsg_send function to send messages,
which will block for up to 15 seconds if there is no TX buffer available
for the message. This is an issue because the device mutex is locked
while waiting and the receive callback for messages uses the same mutex
to prevent concurrent access so no received messages can be handled
while waiting for a TX buffer.
To resolve this, the mailbox has been changed to use the rpmsg_trysend
function, which will return directly if there is no TX buffer available,
together with a wait queue. While waiting in the queue to send the
message, the device mutex is released to not block the receive callback
and other users of the mutex.
Change-Id: I34fbfd21167b49fb83744ab2473ab02632a809ee
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
Instead of locking the device mutex for all IOCTL calls to the NPU
device, the mutex will now only be locked for the calls that have
resources that must be protected from concurrent access. IOCTL calls
that only copy static data do not need to lock the mutex.
The same device mutex is used for all concurrent access protection in
the driver, including the mailbox handling, so removing unnecessary
locking decreases the need to wait for the mutex.
Change-Id: Ic606fe0e1db4aa2e3d4e579ada74418491546468
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
When an inference request fails, the mailbox message registered for the
request is not unregistered, causing the mailbox's IDR to be left with
an entry for the message with a dangling pointer to the freed inference.
The dangling pointer later causes the mailbox clean up to crash when it
attempts to call the fail callback for the IDR entry.
The clean up for a failed inference request has now been fixed so it
unregisters the message from the mailbox.
Change-Id: Ida8015c88afee9cad074289be4d65947854c9e6b
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
To not leak any information from kernel space when IOCTL calls copy data
from kernel space to user space, the source struct in the IOCTL calls is
now initialized.
Change-Id: I0825e82ccdb51ced747e160dd7385fa1ed227eaf
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
The network info IOCTL call doesn't use any information from the user
provided struct so there is no need to copy the struct to kernel space.
Change-Id: I3f58a80ab9377e4ae39fc29068de6fd6bc3d539c
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
In some cases, the IOCTL functions didn't update the return value
variable so the wrong value was returned from the IOCTL calls.
Change-Id: I56529f8fb03cec52dbc555d5ed3d4e68e4bfab70
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
To make it easier to detect why a network info request failed, logs have
been added to the network info response error handling.
Change-Id: I88d98e08501711912d10068344cef62634ac3889
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
To ensure that the description string in the network info response is
correctly null-terminated and that the string will fit into the UAPI
struct. The kernel will now check for the null-termination and verify
that the UAPI struct can hold the expected string size.
Change-Id: I8097a04c6ee4e80f1ab62a66e7323d2462c2c23a
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
The variable used for the return value from the rpmsg callback to handle
mailbox messages isn't initialized when handling some message types.
To resolve this, the variable is now initialized by default.
Change-Id: I7862a6bc3479f03ca06536d94eb4478263e49b70
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
To not leave behind any data in memory from buffers, inferences and
networks, the memory allocated for them is now cleared before it is
freed.
Change-Id: I68c221b190e78f9b4c5022016c38f4ced5e78166
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
The kernel driver's version is now exposed in the UAPI so the UAPI user
can validate that they are compatible with the kernel driver in use.
Change-Id: I52d096dfaedb8c3c1889d27a907626be96779ee7
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
To ensure that the Linux kernel NPU driver and the firmware used are
compatible, the NPU driver will now request and verify the protocol
version when setting up the device during the probing.
Change-Id: I13bca96bf6f98ca90dc79e11ae637c269c2a5eec
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
The current implementation of the protocol version request IOCTL call
doesn't return any version information and the protocol version is not
needed by any userspace program as it's for the communication between
the Linux kernel NPU driver and firmware. The IOCTL call has therefore
been removed from the UAPI and the inference runner has been updated
accordingly.
Change-Id: I9ed09e55d9522996a182f237c8a6db2f45bc40a9
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
The IDR instance must be protected against concurrent access and the
device mutex in the kernel driver should be used to protect it. However,
the device mutex is not locked when an inference is released or when the
mailbox is cleared which means the IDR is not protected in these
instances.
To resolve this, the missing mutex locks have been added and the
functions using the IDR instance have been updated to make it clearer
that the device mutex is expected to be locked when called.
Change-Id: Id0b314db556836c36663d6481806b7c113e55e5f
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
Add missing `WITH Linux-syscall-note` to the ethosu.h UAPI header file
Change-Id: Ie7abf81108d85faab963a262a27f93c02064e4dd
Signed-off-by: Jonny Svärd <jonny.svaerd@arm.com>
|
|
If an error occurs after the kernel driver probe has created a rpmsg
endpoint, the endpoint is not freed during the clean up.
Clean up code for the endpoint has now been added to resolve this.
Change-Id: I9259713f2b7dcf4926e6f317c57f2791ab3463de
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
If the probe function is past the NPU device registration when a failure
occurs, the device instance will be freed twice causing a crash. This
occurs because the device's release callback will free the device
instance when the device is unregistered and then the probe clean up
will attempt to free it again.
To resolve this, the probe will now only directly free the device
instance if the registration fails and otherwise let the device's
release callback handle it.
Change-Id: Iafe87e7ca44b91f8e0e2e870106a4b8c2a69dd8f
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
If the rpmsg channel to communicate with the subsystem is not available
until after the NPU kernel driver has been registered, the driver's
probe function will be called from the rpmsg message receive callback
and block handling of any other messages until the probing is done.
To avoid blocking the callback while performing the probing and to allow
the probe to communicate with the subsystem, the kernel driver will now
request that its probe function is called asynchronously.
Change-Id: Iaeaa4a2322f6a76c0acd202e69b91a8beeb6114b
Signed-off-by: Mikael Olsson <mikael.olsson@arm.com>
|
|
When the kernel driver handles incoming rpmsg messages, it uses the ID
in the message header to find the corresponding mailbox message. The
mailbox messages are stored as a generic message struct that is later
cast to the specific message type.
There is currently no type information stored in the generic message
struct so only the ID is used to distinguish between the mailbox
messages. This means if an incorrect ID is received that matches a
mailbox message with a different type, the kernel driver will cast the
message struct to the wrong type.
Type information has now been added to the generic message struct and
will be checked when trying to find the corresponding mailbox message so
both the ID and type must be correct to find a matching message.
Change-Id: Ifdbceea6ec4ae7078f424a32ce1ff5474bd22fef
|
|
To not have to fetch the capabilities from the device for each time they
are requested with the UAPI, the kernel driver will now fetch and save
the capabilities during the device initialization and use the saved copy
for the UAPI requests.
With this change, the character device creation in the device
initialization has been moved to be the last step so the capabilities
are fetched and saved before the character device is created.
Change-Id: I57956aedb983fbc1765d45906a1c6182e2cd5b96
|
|
When IOCTL calls are handled and when sending messages to the mailbox,
the device mutex is locked. However, when received mailbox messages are
handled, the mutex is not locked so there are possible concurrent access
races in shared states e.g. an inference's status may be updated while
another IOCTL caller is reading it.
To avoid these races, the mutex is now locked while messages received
from the mailbox are handled.
Change-Id: I4d51da542410ab02fb0f890c939269c642176b2c
|
|
The kernel driver will now verify the magic value in an incoming
message's header before any further processing of the message is
performed to avoid trying to handle an invalid message.
Change-Id: I7ae1cac19a6fe4b78316decff4ca2147175ee07a
|
|
When the kernel driver creates the Ethos-U device, it doesn't setup the
coherent DMA mask. This causes the kernel to generate a warning the
first time any of the coherent DMA functions are used and the kernel
will end up using a default DMA mask.
To ensure that the device uses the correct DMA mask and to no longer get
the warning, the kernel driver will now setup the DMA mask for the
device.
Change-Id: I92c67c85be1754970412da92161dbf1ec993bca3
|
|
Change-Id: I64151321ac4a020045de5802beb29fe0013a5c3f
|
|
The Ethos-U kernel driver has been converted into a rpmsg driver and
the ethosu_core_interface.h has been replaced by ethsou_core_rpmsg.h.
Change-Id: If6db8c20cd88e950545eed561ed95c067157d061
|
|
When the Ethos-U kernel driver is probed it creates a /dev/ethosu<nr>
device node, which user space use ioctl to communicate with. When
the driver is removed the Ethos-U resources must live on until all
open file handles have been closed.
The rpmsg device can be removed for a number of reasons, for example
if the firmware is stopped or the remoteproc driver is removed. To
allow the remove to complete without waiting for all file handles to
close, a new 'struct device' is created by the kernel driver. This
device will be used to memory allocations and will live on until the
last file handle has been closed.
Change-Id: I790d8219960f25fe64f58c11a865eb65c7b08974
|
|
The 'struct ethosu_device' has been passed as argument to classes.
This creates a circular dependency dependency and it gives all
classes full visibility to all resources in the device struct. This
patch removes the circular dependency.
Using device_lock() and device_unlock() to for synchronization.
Change-Id: I8322e6530c72d7bd67f48f411b4f14b612be2706
|
|
The Ethos-U kernel driver has been converted from a
platform driver with a custom firmware interface into a
rpmsg driver.
Change-Id: I9ae449f5e79eb02924e6630611d0893e5fec86be
|
|
Removing watchdog and firmware reset as a preparations for the
migrations to rpmsg.
Change-Id: Ic1053e3f4301ecadbde8c59dbaed437625a0a5ea
|
|
Change-Id: If27999db4f18355b571b67e5e5d663a703c3f6a3
|
