aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-10-11Re-factoring of rewrite management & added metricsGergely Nagy
- List available rewrites - Refactor/rename 'Rewrite' class to 'RewritingOptimizer' - Introduce a registry for rewrite functions - Refactor 'Rewriter' to use the registry to look up rewrite functions - Remove mentions of hardcoded "fully_connected" from CLI help and error messages, using the registry instead - Add unit tests - Enable rewrites for all targets: Extract optimization (including rewrite specific code) from the Ethos-U-specific data collector into OptimizingDataCollector. This is reused in other targets' collectors, such as TOSA and Cortex-A. - Add more logging for rewrite - add display of MAE and NRMSE values for the trained result - add total model MAE and NRMSE metric Resolves: MLIA-891, MLIA-899, MLIA-906 Change-Id: Ie798749e1ed60cab14fdb6d9c2271c833960e93f Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com>
2023-10-11Bug-fixes and re-factoring for the rewrite moduleBenjamin Klimczak
- Fix input shape of rewrite replacement: During and after training of the replacement model for a rewrite the Keras model is converted and saved in TensorFlow Lite format. If the input shape does not match the teacher model exactly, e.g. if the batch size is undefined, the TFLiteConverter adds extra operators during conversion. - Fix rewritten model output - Save the model output with the rewritten operator in the output dir - Log MAE and NRMSE of the rewrite - Remove 'verbose' flag from rewrite module and rely on the logging mechanism to control verbose output. - Re-factor utility classes for rewrites - Merge the two TFLiteModel classes - Move functionality to load/save TensorFlow Lite flatbuffers to nn/tensorflow/tflite_graph - Fix issue with unknown shape in datasets After upgrading to TensorFlow 2.12 the unknown shape of the TFRecordDataset is causing problems when training the replacement models for rewrites. By explicitly setting the right shape of the tensors we can work around the issue. - Adapt default parameters for rewrites. The training steps especially had to be increased significantly to be effective. Resolves: MLIA-895, MLIA-907, MLIA-946, MLIA-979 Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com> Change-Id: I887ad165aed0f2c6e5a0041f64cec5e6c5ab5c5c
2023-10-11Implement first rewrite (proof of concept)Ruomei Yan
* Define replacement function fully_connected layer * Define RewriteConfiguration and Rewriter to integrate rewrite module into mlia optimize command * Fix a bug in the ethos_u/data_collection.py file * Fix a bug in join.py * Remove diff_stats and use diff instead, added related changes around this to ensure e2e tests passing * Add unit tests for all changes * Fix bug in diff_stats function * The bug was caused by a dividing by numpy array of all zeros. The previous way of handling it did not consider the all zeros case but only dealt with partially zeros * unit tests added. * Fix the bug in rewrite/core/graph_edit/join.py * Remove the possibility of passing None to append_relabel function because it is immutable * The bug happened when empty dictionary was passed in the append_relabel function and the function overwrites the reference of operator_map which caused the dictionary was not updated after the function call Resolves: MLIA-749, MLIA-864, MLIA-866 Change-Id: I1ab426996232f182345e6e98033d5dcb32aea08c Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com>
2023-10-11Adapt rewrite module to MLIA coding standardsAnnie Tallund
- Fix imports - Update variable names - Refactor helper functions - Add licence headers - Add docstrings - Use f-strings rather than % notation - Create type annotations in rewrite module - Migrate from tqdm to rich progress bar - Use logging module in rewrite module: All print statements are replaced with logging module Resolves: MLIA-831, MLIA-842, MLIA-844, MLIA-846 Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com> Change-Id: Idee37538d72b9f01128a894281a8d10155f7c17c
2023-10-11MLIA-843 Add unit tests for module mlia.nn.rewriteBenjamin Klimczak
Note: The unit tests mostly call the main functions from the respective modules only. Change-Id: Ib2ce5c53d0c3eb222b8b8be42fba33ac8e007574 Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com>
2023-10-11Add a CLI component to enable rewritesRuomei Yan
* Add flags for rewrite (--rewrite, --rewrite-start, --rewrite-end, --rewrite-target) * Refactor CLI interfaces to accept tflite models with optimize for rewrite, keras models with optimize for clustering and pruning * Refactor and move common.py and select.py out of the folder nn/tensorflow/optimizations * Add file nn/rewrite/core/rewrite.py as placeholder * Update/add unit tests * Refactor OptimizeModel in ethos_u/data_collection.py for accepting tflite model case * Extend the logic so that if "--rewrite" is specified, we don't add pruning to also accept TFLite models. * Update README.md Resolves: MLIA-750, MLIA-854, MLIA-865 Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com> Change-Id: I67d85f71fa253d2bad4efe304ad8225970b9622c
2023-10-11MLIA-845 Migrate rewrite codeAnnie Tallund
- Add required files for rewriting of TensorFlow Lite graphs - Adapt rewrite dependency paths and project name - Add license headers Change-Id: I19c5f63215fe2af2fa7d7d44af08144c6c5f911c Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com>
2023-09-29MLIA-984 Pin 'build' package to ~=0.10.0Benjamin Klimczak
The recent update to version 1.0.3 causes issues when building. Change-Id: I74db3b419ece7b744cbb48c8e11096e99709a9cd Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com>
2023-09-26MLIA-469 Support batch size > 1 for optimizationsAnnie Tallund
- Add a PruningPolicy to skip layers that are not supported by the Keras pruning API - Make dataset generation more generic to support use-cases beyond classification Signed-off-by: Annie Tallund <annie.tallund@arm.com> Change-Id: I198dae2b53860f449f2fdbc71575babceed1ffcf
2023-09-08MLIA-961 Update pre-commit hook versionsBenjamin Klimczak
- Update dependencies in .pre-commit.yaml - Fix code issues with new linters Change-Id: I36964ecf1a405dd8faac01b4470b56122a7cad17 Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com>
2023-09-05MLIA-961 Update tox dependenciesBenjamin Klimczak
- Update version dependencies in the tox.ini - Fix linter issues Change-Id: I04c3a841ee2646a865dab037701d66c28792f2a4 Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com>
2023-09-05MLIA-961 Update MLIA dependenciesBenjamin Klimczak
Update Vela, TensorFlow and other dependencies. Change-Id: Id497084391d31548c2f307f5a4b7981981fa25a5 Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com>
2023-08-22MLIA-932 Update RELEASES.md0.7.0-rc.10.7.0Annie Tallund
Signed-off-by: Annie Tallund <annie.tallund@arm.com> Change-Id: I3c3f458dab214a7be00dd50c640a62767389fd9b
2023-08-22MLIA-757: Update Vela and TensorFlow dependenciesJoseph Tummon
Ignoring new 'Conv_hwcn_Weights' layer that is named in TensorFlow 2.11.1 and used as buffer for float32 only. Signed-off-by: Joseph Tummon <Joseph.Tummon@arm.com> Change-Id: Iaa579c76013b1e0dc505466c46f5864a96af8c6d
2023-08-21MLIA-881 Update compatible operators for Cortex-ABenjamin Klimczak
Update the compatibility information of the backend for ArmNN TensorFlow Lite Delegate (classic) to version 23.05. Change-Id: I84693842d1a883f7083a6faf7d5ddcd5ecc34e5d Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com>
2023-07-25MLIA-757: Update Vela and remove NumPy dependencyJoseph Tummon
Two warnings appear when running e2e tests from where TensorFlow declares ranges of NumPy dtype, but this shouldn't affect anything else in MLIA. Change-Id: Idc22f1de352980c70d27cdcba35a51a1e1efaafc Signed-off-by: Joseph Tummon <Joseph.Tummon@arm.com>
2023-05-15Add contribution guidanceAnton Kachatkou
Signed-off-by: Anton Kachatkou <anton.kachatkou@arm.com> Change-Id: Ie74d7f8e7a4df319b72b64dda85cc9b754769dc5
2023-05-15MLIA-755 Remove TOSA Extra DependencyDhruv Chauhan
- Removed TOSA Checker dependency from setup.cfg - Installation process calls directly the installation of TOSA Checker instead of the extra dependency Change-Id: I21e309b9316671959483bd2ef1ecaf644936a4cb
2023-04-12MLIA-849 Update mypy version and fix associated issuesAnnie Tallund
- Remove unused silencing of typing - Amend None type hints where it is default Change-Id: Id972b56dcdce865bf6c9d6aea88bc76baf39133e
2023-04-11MLIA-825 Make backend installation more genericBenjamin Klimczak
Use the backend registry to manage installation routines for different backends as well. This makes the process more generic and allows us to move towards plug & play for backends (i.e. there should be no code changes needed in the general MLIA code to add/support new backends). Change-Id: Ib7c772ec52b2f4d857c2c9871e44dfff97e9d970
2023-04-04MLIA-816 Correctly name TOSA Checker attributeDhruv Chauhan
- JSON output now reports tosa_checker as attribute name, which accurately represents its value as the version of TOSA Checker Change-Id: I8ac5032fea5dae65382db76091d295595a644816
2023-03-24MLIA-711 Extend TensorFlow Lite Compatibility CheckDhruv Chauhan
- Unify the TensorFlow Lite compatibility check across Cortex-A, TOSA and Ethos-U targets - Display tables/messages with parsed information - Do not display raw TensorFlow Lite errors, and return with exit code 0 Change-Id: I9333fdb6cbe592f1ed7395d392412168492a1479
2023-02-21MLIA-813 Change default output directory0.6.0-rc.30.6.0Dmitrii Agibov
- Use directory mlia-output as output directory for MLIA - If parameter --output-dir provided then place directory mlia-output under specified path or otherwise create it in the current working directory Change-Id: I298088c4aa8dbe9f35dee69ecb9ff6e9ea3cac0a
2023-02-16MLIA-812 Show error message for unsupported functionality0.6.0-rc.2Dmitrii Agibov
Change-Id: I68fb8c4e51046e9fc2d91ad8338718ba545209cd
2023-02-15MLIA-814 Specify required Python version exactlyBenjamin Klimczak
There is an issue with Python 3.8.0 that makes it impossible to use MLIA. It is fixed from Python 3.8.1 onwards. https://github.com/python/cpython/issues/82019 Change-Id: I48c0a5f103fb29561be483875ba706928b7e77d1
2023-02-15MLIA-798 Update/check RELEASES.mdRuomei Yan
Change-Id: I5eff40ac2050b77f580460c2d5233ed6e96adeb7
2023-02-15MLIA-813 Change default output directoryDmitrii Agibov
- If no output directory provided then create one in the current working directory - Update documentation and tests Change-Id: Id1f2acef14e1bd2edffbfa6139a961a5c5018211
2023-02-13MLIA-709 Update compatibility data for Cortex-A0.6.0-rc.1Annie Tallund
- Add operator compatibility data for Cortex-A via for ArmNN TensorFlow Lite delegate 22.11 - Extend the Cortex-A target profile to include the version of the ArmNN TensorFlow Lite delegate to be used. - Some re-factoring work to support multiple versions and the new target profile parameter. Change-Id: Iae91bb0757ea3909be975af68b34d0ca2be47c43
2023-02-13MLIA-775 Refactor metadata related classesRuomei Yan
- define Metadata base class with dictionary data and abstract method - mlia, tosa, model and metadatadisplay classes are all inherited from base class - update unit tests - update function report_metadata into more generalized format Change-Id: Id49e15283eebdca705045eda81db637d82f85453
2023-02-13MLIA-811 Fix issue with CLI parser in e2e testsDmitrii Agibov
- Create new CLI parser instance whenever needed as it contains static data and could not be reused Change-Id: I8a67bb585e95d972304eca809ff28b822a3151b6
2023-02-10MLIA-769 Add "pretty names" for targets / backendsBenjamin Klimczak
- Provide "pretty names" to print information for targets and backends. - Use 'target_config' instead of 'target' if a target profile is used. - Fix minor issue in output regarding the output directory. Change-Id: Ib38231f30b4d609a0d1e8f9c52b2fb547c69cb6a
2023-02-10MLIA-769 Replace use of 'device' with 'target'Benjamin Klimczak
Term 'device' can be ambiguous and is replaced with 'target'. Change-Id: I5e5108d033a13b98e4c2997713e1c32bce63ae62
2023-02-10MLIA-769 Expand use of target/backend registriesBenjamin Klimczak
- Use the target/backend registries to avoid hard-coded names. - Cache target profiles to avoid re-loading them Change-Id: I474b7c9ef23894e1d8a3ea06d13a37652054c62e
2023-02-10MLIA-592 Add documentation for using custom target profilesAnnie Tallund
Change-Id: I7dda9c2b99a1464d9682074245584d8c518a9c13
2023-02-10MLIA-594 Save target profile configurationAnnie Tallund
Save the target profile file in the output directory. Change-Id: I886e52cb922c5425e749b154bd67a5d294ce0201
2023-02-08MLIA-595 Remove old backend configuration mechanismDmitrii Agibov
- Remove old backend configuration code - Install backends into directory ~/.mlia - Rename targets/backends in registry to make it consistent across codebase. Change-Id: I9c8b012fe863280f1c692940c0dcad3ef638aaae
2023-02-08MLIA-770 List all available backendsAnnie Tallund
- Rely on target and backend registry for support information - Make above information less Ethos(TM)-U specific Change-Id: I8dbfb84401016412a3d719a84eb592f21d79c46b
2023-02-08MLIA-591 Create interface for target profilesBenjamin Klimczak
New class 'TargetProfile' is used to load and verify target profiles. Change-Id: I76373a923e2e5f55c4e95860635afe9fc5627a5d
2023-02-08MLIA-590 Support path to custom target profilesAnnie Tallund
- Start using TOML format for target profile - Add support for loading custom target profile files Change-Id: I6be019d4341e93115440ccdbdb6dafdc1c85b966
2023-02-08MLIA-737 Show connection between target / backendBenjamin Klimczak
- The help text of MLIA now shows a table of supported targets, backends and advice. - The table is only shown with the help message and not when MLIA is run normally. Change-Id: I3234ce91e943de4b08b9471bd95a474df34755f7
2023-02-06MLIA-461 Add parameter for the output directoryDmitrii Agibov
- Add CLI parameter --output-dir - Rename ExecutionContext property working_dir into output_dir - Remove logic for default command as it is no longer needed Change-Id: I6387f6b688520ba1fc69a80167587297353620f6
2023-02-03MLIA-782 Remove --output parameterDiego Russo
* Remove --output parameter from argument parser * Remove FormattedFilePath class and its presence across the codebase * Move logging module from cli to core * The output format is now injected in the execution context and used across MLIA * Depending on the output format, TextReporter and JSONReporter have been created and used accordingly. * The whole output to standard output and/or logfile is driven via the logging module: the only case where the print is used is when the --json parameter is specified. This is needed becase all output (including third party application as well) needs to be disabled otherwise it might corrupt the json output in the standard output. * Debug information is logged into the log file and printed to stdout when the output format is plain_text. * Update E2E test and config to cope with the new mechanism of outputting json data to standard output. Change-Id: I4395800b0b1af4d24406a828d780bdeef98cd413
2023-01-31MLIA-785 Enable export into json for enumsDmitrii Agibov
Change-Id: I8e4d5d04f6b1b252dae872ea76d2bd8c41f4b376
2023-01-30MLIA-784 Make CLI options order independentDmitrii Agibov
- Change logic for parameter "-b/--backend" to avoid dependency between CLI options Change-Id: Ic98cb4eb6340cf249fcaa8ec55d17d21b5664b81
2023-01-16MLIA-741/2 Report test resultsRuomei Yan
- add version extraction function in compat.py - create Metadata, MLIAMetadata, TOSAMetadata and MetadataDisplay classes - update the reporting functions so tosa and mlia version will be displayed in output json - update unit test test_configure_and_get_tosa_advisor to mock the get_events function - update the copyright information of all changed/added files - handle exception and report to json when program crashes - write new context managers for capturing stderr and stdout - support reporting stderr to json output - support reporting model checksum and model name to json output - made changes in test_e2e.py handling {model_name} replacement in --output - add unit tests Change-Id: I6629fd1c5754378e6accd488217c83d87c7eb6f1
2023-01-10MLIA-650 Implement new CLI changesRaul Farkas
Breaking change in the CLI and API: Sub-commands "optimization", "operators", and "performance" were replaced by "check", which incorporates compatibility and performance checks, and "optimize" which is used for optimization. "get_advice" API was adapted to these CLI changes. API changes: * Remove previous advice category "all" that would perform all three operations (when possible). Replace them with the ability to pass a set of the advice categories. * Update api.get_advice method docstring to reflect new changes. * Set default advice category to COMPATIBILITY * Update core.common.AdviceCategory by changing the "OPERATORS" advice category to "COMPATIBILITY" and removing "ALL" enum type. Update all subsequent methods that previously used "OPERATORS" to use "COMPATIBILITY". * Update core.context.ExecutionContext to have "COMPATIBILITY" as default advice_category instead of "ALL". * Remove api.generate_supported_operators_report and all related functions from cli.commands, cli.helpers, cli.main, cli.options, core.helpers * Update tests to reflect new API changes. CLI changes: * Update README.md to contain information on the new CLI * Remove the ability to generate supported operators support from MLIA CLI * Replace `mlia ops` and `mlia perf` with the new `mlia check` command that can be used to perform both operations. * Replace `mlia opt` with the new `mlia optimize` command. * Replace `--evaluate-on` flag with `--backend` flag * Replace `--verbose` flag with `--debug` flag (no behaviour change). * Remove the ability for the user to select MLIA working directory. Create and use a temporary directory in /temp instead. * Change behaviour of `--output` flag to not format the content automatically based on file extension anymore. Instead it will simply redirect to a file. * Add the `--json` flag to specfy that the format of the output should be json. * Add command validators that are used to validate inter-dependent flags (e.g. backend validation based on target_profile). * Add support for selecting built-in backends for both `check` and `optimize` commands. * Add new unit tests and update old ones to test the new CLI changes. * Update RELEASES.md * Update copyright notice Change-Id: Ia6340797c7bee3acbbd26601950e5a16ad5602db
2023-01-04MLIA-589 Create an API to get target informationBenjamin Klimczak
Change-Id: Ieeaa9188ea1e29e2ccaad7475d457bce71e3140d
2022-12-30MLIA-761 Constrain dependenciesBenjamin Klimczak
Constrain dependencies in the setup.cfg to avoid incompatibilities when dependencies are updated. Change-Id: I6e26a769bc677827da3ffd90a9b7ad510e3b4d4f
2022-12-19MLIA-756 Fix Vela bug in MLIA caused by numpyRaul Farkas
Set a constraint for numpy in MLIA to be < 1.24. This is needed to fix the temporary issue that Numpy is causing in Vela 3.6 (i.e. "AttributeError: module 'numpy' has no attribute 'bool'") Change-Id: Ie9c99a7a328b5db4c89bccc9429ab93d6a88fe9e
2022-12-16MLIA-460 Revisit backend managementDmitrii Agibov
- Provide command for backend installation in case if backend is not available - Fix issue with connection timeout during downloading - Show installation tools output only in verbose mode Change-Id: Ic0e495ba19879cc2cda4fd0bce20b57ba896cfeb
generated by cgit v1.2.1 (git 2.23.0) at 2024-06-02 08:06:04 +0000