aboutsummaryrefslogtreecommitdiff
path: root/tests/test_cli_commands.py
AgeCommit message (Collapse)Author
2024-03-27fix: Update rewrite target nameMadeleine Dunn
- Rename "fully_connected" to "fully-connected" - This will resolve issues with upstreaming rewrite library changes Signed-off-by: Madeleine Dunn <madeleine.dunn@arm.com> Change-Id: I2f24ae4917a556fd0bd44f0db6ee4e0f7a68cd24
2024-03-14feat: Enable rewrite parameterisationNathan Bailey
Enables user to provide a toml or default profile to change training settings for rewrite optimization Resolves: MLIA-1004 Signed-off-by: Nathan Bailey <nathan.bailey@arm.com> Change-Id: I3bf9f44b9a2062fb71ef36eb32c9a69edcc48061
2024-02-23refactor: Migrate from Vela's internal code to CSV summaryNathan Bailey
Removes vela defines from vela compiler.py and performance.py Replaces calls to vela code with data from vela summary csv Resolves: MLIA-1024 Signed-off-by: Nathan Bailey <nathan.bailey@arm.com> Change-Id: I569878f2936767f70c0255919ca40d1969275529
2024-02-09feat: Integrate Vela's per-layer performance estimatesNathan Bailey
Resolves: MLIA-1055, MLIA-1056, MLIA-1057 Signed-off-by: Nathan Bailey <nathan.bailey@arm.com> Change-Id: Id573cec94e4a69117051dcd5175f383c0955d890
2023-10-11Re-factoring of rewrite management & added metricsGergely Nagy
- List available rewrites - Refactor/rename 'Rewrite' class to 'RewritingOptimizer' - Introduce a registry for rewrite functions - Refactor 'Rewriter' to use the registry to look up rewrite functions - Remove mentions of hardcoded "fully_connected" from CLI help and error messages, using the registry instead - Add unit tests - Enable rewrites for all targets: Extract optimization (including rewrite specific code) from the Ethos-U-specific data collector into OptimizingDataCollector. This is reused in other targets' collectors, such as TOSA and Cortex-A. - Add more logging for rewrite - add display of MAE and NRMSE values for the trained result - add total model MAE and NRMSE metric Resolves: MLIA-891, MLIA-899, MLIA-906 Change-Id: Ie798749e1ed60cab14fdb6d9c2271c833960e93f Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com>
2023-10-11Implement first rewrite (proof of concept)Ruomei Yan
* Define replacement function fully_connected layer * Define RewriteConfiguration and Rewriter to integrate rewrite module into mlia optimize command * Fix a bug in the ethos_u/data_collection.py file * Fix a bug in join.py * Remove diff_stats and use diff instead, added related changes around this to ensure e2e tests passing * Add unit tests for all changes * Fix bug in diff_stats function * The bug was caused by a dividing by numpy array of all zeros. The previous way of handling it did not consider the all zeros case but only dealt with partially zeros * unit tests added. * Fix the bug in rewrite/core/graph_edit/join.py * Remove the possibility of passing None to append_relabel function because it is immutable * The bug happened when empty dictionary was passed in the append_relabel function and the function overwrites the reference of operator_map which caused the dictionary was not updated after the function call Resolves: MLIA-749, MLIA-864, MLIA-866 Change-Id: I1ab426996232f182345e6e98033d5dcb32aea08c Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com>
2023-10-11Add a CLI component to enable rewritesRuomei Yan
* Add flags for rewrite (--rewrite, --rewrite-start, --rewrite-end, --rewrite-target) * Refactor CLI interfaces to accept tflite models with optimize for rewrite, keras models with optimize for clustering and pruning * Refactor and move common.py and select.py out of the folder nn/tensorflow/optimizations * Add file nn/rewrite/core/rewrite.py as placeholder * Update/add unit tests * Refactor OptimizeModel in ethos_u/data_collection.py for accepting tflite model case * Extend the logic so that if "--rewrite" is specified, we don't add pruning to also accept TFLite models. * Update README.md Resolves: MLIA-750, MLIA-854, MLIA-865 Signed-off-by: Benjamin Klimczak <benjamin.klimczak@arm.com> Change-Id: I67d85f71fa253d2bad4efe304ad8225970b9622c
2023-02-10MLIA-769 Expand use of target/backend registriesBenjamin Klimczak
- Use the target/backend registries to avoid hard-coded names. - Cache target profiles to avoid re-loading them Change-Id: I474b7c9ef23894e1d8a3ea06d13a37652054c62e
2023-02-08MLIA-591 Create interface for target profilesBenjamin Klimczak
New class 'TargetProfile' is used to load and verify target profiles. Change-Id: I76373a923e2e5f55c4e95860635afe9fc5627a5d
2023-02-08MLIA-590 Support path to custom target profilesAnnie Tallund
- Start using TOML format for target profile - Add support for loading custom target profile files Change-Id: I6be019d4341e93115440ccdbdb6dafdc1c85b966
2023-01-10MLIA-650 Implement new CLI changesRaul Farkas
Breaking change in the CLI and API: Sub-commands "optimization", "operators", and "performance" were replaced by "check", which incorporates compatibility and performance checks, and "optimize" which is used for optimization. "get_advice" API was adapted to these CLI changes. API changes: * Remove previous advice category "all" that would perform all three operations (when possible). Replace them with the ability to pass a set of the advice categories. * Update api.get_advice method docstring to reflect new changes. * Set default advice category to COMPATIBILITY * Update core.common.AdviceCategory by changing the "OPERATORS" advice category to "COMPATIBILITY" and removing "ALL" enum type. Update all subsequent methods that previously used "OPERATORS" to use "COMPATIBILITY". * Update core.context.ExecutionContext to have "COMPATIBILITY" as default advice_category instead of "ALL". * Remove api.generate_supported_operators_report and all related functions from cli.commands, cli.helpers, cli.main, cli.options, core.helpers * Update tests to reflect new API changes. CLI changes: * Update README.md to contain information on the new CLI * Remove the ability to generate supported operators support from MLIA CLI * Replace `mlia ops` and `mlia perf` with the new `mlia check` command that can be used to perform both operations. * Replace `mlia opt` with the new `mlia optimize` command. * Replace `--evaluate-on` flag with `--backend` flag * Replace `--verbose` flag with `--debug` flag (no behaviour change). * Remove the ability for the user to select MLIA working directory. Create and use a temporary directory in /temp instead. * Change behaviour of `--output` flag to not format the content automatically based on file extension anymore. Instead it will simply redirect to a file. * Add the `--json` flag to specfy that the format of the output should be json. * Add command validators that are used to validate inter-dependent flags (e.g. backend validation based on target_profile). * Add support for selecting built-in backends for both `check` and `optimize` commands. * Add new unit tests and update old ones to test the new CLI changes. * Update RELEASES.md * Update copyright notice Change-Id: Ia6340797c7bee3acbbd26601950e5a16ad5602db
2022-11-29Rename modulesDmitrii Agibov
- Rename module "mlia.devices" into "mlia.target" - Rename module "mlia.target.ethosu" into "mlia.target.ethos_u" - Rename module "mlia.target.cortexa" into "mlia.target.cortex_a" - Rename and update tests Change-Id: I6dca7c8646d881f739fb6b5914d1cc7e45e63dc2
2022-11-29Move backends functionality into separate modulesDmitrii Agibov
- Move backend management/executor code into module backend_core - Create separate module for each backend in "backend" module - Move each backend into corresponding module - Split Vela wrapper into several submodules Change-Id: If01b6774aab6501951212541cc5d7f5aa7c97e95
2022-11-16MLIA-649 Support tosa-checker as a backendDmitrii Agibov
- Add new type of the backend based on python packages - Add installation class for TOSA checker - Update documentation - Extend support of the parameter "force" in the "install" command Change-Id: I95567b75e1cfe85daa1f1c3d359975bb67b2504e
2022-11-15MLIA-649 Strip mlia backend management into a new commandRuomei Yan
* add entry point for mlia-backend in setup.cfg and main.py * add --force option for install from path: uninstall existing backend in ML Inference Advisor and install from given path * add uninstall and list program parameters: uninstall has backend_name as input arg, install has backend_name as a mandatory argument * add unit tests in test_cli_commands.py, test_cli_main.py, test_tools_metadata_common.py, test_tools_metadata_corstone.py * updated README.md * remove --download option for installing backend * add new lines for the display section when we do mlia-backen list * add case insensitive support for backend names in command line argument Change-Id: Icb89d8957fa6be4b767710e24fa074f26472674b
2022-10-04MLIA-673 Use inclusive language in the source code and filenamesDmitrii Agibov
- Update configuration for inclusive language linter - Fix reported issues Change-Id: If0f8b6e20c17d8ee1c6179c61040fc351437f036
2022-09-09MLIA-386 Simplify typing in the source codeDmitrii Agibov
- Enable deferred annotations evaluation - Use builtin types for type hints whenever possible - Use | syntax for union types - Rename mlia.core._typing into mlia.core.typing Change-Id: I3f6ffc02fa069c589bdd9e8bddbccd504285427a
2022-07-26MLIA-551 Rework remains of AIET architectureBenjamin Klimczak
Re-factoring the code base to further merge the old AIET code into MLIA. - Remove last traces of the backend type 'tool' - Controlled systems removed, including SSH protocol, controller, RunningCommand, locks etc. - Build command / build dir and deploy functionality removed from Applications and Systems - Moving working_dir() - Replace module 'output_parser' with new module 'output_consumer' and merge Base64 parsing into it - Change the output consumption to optionally remove (i.e. actually consume) lines - Use Base64 parsing in GenericInferenceOutputParser, replacing the regex-based parsing and remove the now unused regex parsing - Remove AIET reporting - Pre-install applications by moving them to src/mlia/resources/backends - Rename aiet-config.json to backend-config.json - Move tests from tests/mlia/ to tests/ - Adapt unit tests to code changes - Dependencies removed: paramiko, filelock, psutil - Fix bug in corstone.py: The wrong resource directory was used which broke the functionality to download backends. - Use f-string formatting. - Use logging instead of print. Change-Id: I768bc3bb6b2eda57d219ad01be4a8e0a74167d76