diff options
Diffstat (limited to 'driver_library/python/README.md')
| -rw-r--r-- | driver_library/python/README.md | 336 |
1 files changed, 336 insertions, 0 deletions
diff --git a/driver_library/python/README.md b/driver_library/python/README.md new file mode 100644 index 0000000..7fae749 --- /dev/null +++ b/driver_library/python/README.md @@ -0,0 +1,336 @@ +# About Python ethosu_driver + +Python ethosu_driver is an extension for +[Arm Ethos-U driver library](https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ethos-u-linux-driver-stack/). +Python ethosu_driver provides interface similar to Ethos-u Linux driver C++ +Api. + +The Python package is built with public headers from the +[driver_library/include](https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ethos-u-linux-driver-stack/+/refs/heads/master/driver_library/include/) +folder. + +The [SWIG](http://www.swig.org/) tool is used to generate the Ethos-U driver +library Python shadow classes and C wrapper. + +## Python ethosu_driver library installation + +Python ethosu_driver library can be packaged as a source package or a binary +package (wheel). Binary package is platform dependent, the name of the +package will indicate the platform it was built for, e.g.: + +* Linux Aarch 64 bit machine: ethosu_driver-1.0.0-cp37-cp37m-linux_aarch64.whl + +The source package is platform independent but installation will involve +Ethos-U driver library C wrapper compilation on a target machine. +You will need to have g++ compatible with C++ 14 standard and a Python +development library installed on the target machine. + +Python driver binary package is linked statically with C++ Ethos-U driver +library and can operate independently from it when built. +Python driver source package requires static Ethos-U driver library - +libethosu.a - and public header during installation, thus they must be +present on the target machine. + +### Installing from wheel + +Install ethosu_driver from binary by pointing to the wheel file: + +1) If corresponding wheel is available for your platform architecture in the +public repository. + ``` + pip install ethosu_driver + ``` +2) If you have local wheel file. Example: + ``` + pip install /path/to/ethosu_driver-X.X.X-cp37-cp37m-linux_aarch64.whl + ``` + +### Installing from source package + +While installing from sources, you can choose Ethos-U driver library to +be used. By default, library will be searched in standard for your system +locations. You can check them by running: +``` +gcc --print-search-dirs +``` +Headers will be searched in standard include directories for your system. +If Ethos-U driver library has custom location, set environment variables +*ETHOS_U_DRIVER_LIB* and *ETHOS_U_DRIVER_INCLUDE* to point to Ethos-U driver +library (libethosu.a) and header (ethosu.hpp): +``` +export ETHOS_U_DRIVER_LIB=/path/to/lib +export ETHOS_U_DRIVER_INCLUDE=/path/to/headers +``` + +Installing from the public repository. +``` +pip install ethosu_driver +``` +Installing from local file. +``` +pip install /path/to/ethosu_driver-X.X.X.tar.gz +``` + +If ethosu_driver installation script fails to find Ethos-U driver libraries it +will raise an error like this + +`RuntimeError: Ethos-U driver library was not found in +('/usr/lib/gcc/aarch64-linux-gnu/8/', <...> ,'/lib/', '/usr/lib/'). +Please install driver to one of the standard locations or set correct +ETHOS_U_DRIVER_INCLUDE and ETHOS_U_DRIVER_LIB env variables.` + +You can now verify that ethosu_driver library is installed and check +ethosu_driver version using: + +``` +pip show ethosu_driver +``` + +## Building Python ethosu_driver library locally + +### Install SWIG + +We suggest to use SWIG version 3.0.12 or newer. You can check available swig +version for you system here: https://pkgs.org/download/swig. + +For example, install the tool with Ubuntu package manager as follows: +``` +sudo apt install swig +``` +If your system has swig version less than 3.0.12, please, build and install +from sources: + +1. Download SWIG: + ``` + wget https://github.com/swig/swig/archive/refs/tags/v4.0.2.zip + unzip v4.0.2.zip + ``` +2. Build and install SWIG: + ``` + cd swig-4.0.2 + ./autogen.sh + ./configure --prefix=<set your system installation prefix> + make + make install + ``` + +### Building as part of cmake flow + +To build Python ethosu_driver as part of Ethos-U NPU Linux driver stack provide +the following cmake flags: +1) For source distribution + ``` + -DBUILD_PYTHON_SRC=1 + ``` +2) For wheel distribution + ``` + -DBUILD_PYTHON_WHL=1 + ``` +Note: you will need to have a Python instance for your target platform to build +wheel. For example, if you are building for an AArch64 platform, you will need +Python installation for aarch64-linux-gnu tool-chain. + +Build result can be found in `<your cmake build dir>/python/dist`. + +### Building standalone + +Navigate to `driver_libarary/python` and execute: +``` +python3 setup.py clean --all +python3 ./swig_generate.py +python3 setup.py sdist +``` +Build result can be found in `driver_libarary/python/dist`. + +## Python ethosu_driver API overview + +### Getting started + +After the Python driver library is installed with pip and can be accessed +within your work environment, import it in your script: + +```python +import ethosu_driver as driver +``` + +Create a device. You can ping Ethos-U device with `ping` method: + +```python +device = driver.Device("/dev/ethosu0") +device.ping() +``` + +You can create memory buffer with data from a binary file or Python buffer +object: + +```python +# from file: +network_file = "/path/to/model.tflite" +network_buffer = driver.Buffer(device, network_file) + +# from numpy: +ifm_zeros = numpy.zeros(ifm_size, dtype=np.uint8) +ifm_buffer = driver.Buffer(device, ifm_size) +ifm_buffer.from_buffer(ifm_zeros.data) +``` + +To create a network object, provide memory buffer for the model file and +created device: + +```python +network = driver.Network(device, network_buffer) +``` + +Inference object is instantiated with a network object and lists of input +memory buffers and output memory buffers. +For example: + +```python +ifms = [ifm_buffer] + +ofms = [] +for ofm_size in network.getOfmDims(): + ofm_buffer = driver.Buffer(device, ofm_size) + ofms.append(ofm_buffer) + +inference = driver.Inference(network, ifms, ofms) +``` + +To execute the inference and wait for the callback: + +```python +# wait infinitely +inference.wait() + +# wait with a timeout in nano seconds: +inference.wait(timeoutNanos=60e9) +``` + +To read results of the inference, iterate through available outputs and convert +them to numpy array: + +```python +for buffer_out in inference.getOfmBuffers(): +array = np.frombuffer(buffer_out.data(), dtype=np.uint8) +``` + +See inline py docs for more info on driver public API. + +## Inference runner + +Python ethosu_driver library comes with a script `inference_runner` that is +installed to the Python environment `bin` directory and could be invoked +from a command line by the file name: + +```cmd +inference_runner <args> +``` + +Arguments: + +* `--device` : Npu device name. Default: ethosu0. +* `--model` : Tflite model file path. +* `--timeout` : inference timeout in seconds, Default: infinite. +* `--inputs` : list of files containing input feature maps. +* `--output_dir` : directory to store inference results, output feature maps. +Default: current directory. +* `--npy` : Use npy input/output. Default: 0. +* `--profile_counters`: profile counters to measure during inference, accepts +four integers chosen from +[HW Supported Ethos-U PMU +Events](https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ethos-u-core-driver/+/refs/heads/master/include/pmu_ethosu.h). + +Example: + +```cmd +inference_runner --device ethosu0 --model ./mobilenet_v2_vela.tflite --timeout +60 --npy 1 --inputs ./input1.npy --output_dir ./ofms +``` + +## Using inference runner with numpy + +Python ethosu_driver libarary could be installed with numpy support. +Numpy will be automatically downloaded and installed alongside ethosu_driver. +If your machine does not have access to pypi repositories you might need to +install NumPy in advance by following public instructions: +<https://scipy.org/install.html>, or to have it installed from wheel package +built for your platform. + +Now, if you provide inference runner command line parameter `--npy 1` the +script will interpret input files as numpy array exports and will save output +feature maps as numpy arrays. + +## Setup development environment + +Before, proceeding to the next steps, make sure that: + +1. You have Python 3.6+ installed system-side. The package is not compatible +with older Python versions. +2. You have python3.6-dev installed system-side. This contains header files +needed to build ethosu_driver extension module. +3. In case you build Python from sources manually, make sure that the following +libraries are installed and available in you system: +``python3.6-dev build-essential checkinstall libreadline-gplv2-dev +libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev +libbz2-dev`` +4. install SWIG, swig must be version 4.* + +## Setup virtual environment + +Now you can proceed with setting up workspace: + +1. Set environment variables ETHOS_U_DRIVER_LIB (pointing to Ethos-U driver +library) and ETHOS_U_DRIVER_INCLUDE (pointing to Ethos-U driver library +headers) +2. Create development env using script ``init_devenv.sh`` + +## Generating SWIG wrappers + +Before building package or running tests you need to generate SWIG wrappers +based on the interface [files](src/ethosu_driver/swig). +```commandline +python setup.py clean --all +python ./swig_generate.py +``` + +## Running unit-tests + +Tests could be executed only on a system with Ethos-U NPU device. +Pytest is used as unit-test framework, before running the test you need to +install pytest and numpy or include into your rootfs image: + +```commandline +pip install pytest +pip install numpy +``` + +Execute command from the project root dir: +``` +pytest -v +``` + +## Regenerate SWIG stubs inplace + +If you want to check that swig wrappers are compiling correctly, you can issue +extension compilation inplace: + +1) clean old generated files: + + ```bash + python setup.py clean --all + ``` + +2) Run swig wrapper source generation as described in [Generating SWIG +wrappers](#generating-swig-wrappers) +3) Run `build_ext` command: + + ```bash + export ETHOS_U_DRIVER_LIB=/path/to/driver/lib + export ETHOS_U_DRIVER_INCLUDE=/path/to/driver/include + python setup.py build_ext --inplace + ``` + +It will put all generated files under ./src/ethosu_driver/_generated folder. +Command will fail on x86 machine during linkage phase because Ethos-U driver +is built for Arm platform, but compilation stage should pass successfully +(or give you indication of compilation problems). |