aboutsummaryrefslogtreecommitdiff
path: root/ethosu/vela/tflite_writer.py
AgeCommit message (Collapse)Author
2023-12-22MLBEDSW-8497: [MLCE] Avoid modifying FC with dynamic weightsJohan Alfven
- If a npu op is followed by a convolution op with dynamic weights the optimized file ends up containing a duplicated tensor called _cpu. - Another problem is also that an empty bias tensor is added in the reader. - The fix is to ignore these cpu ops both in the reader and the writer. Change-Id: I476b4f6062e26cca4ba589df694a99ef79b0f6d4 Signed-off-by: Johan Alfven <johan.alfven@arm.com>
2023-12-20MLBEDSW-8157: Update to TensorFlow 2.15William Isaksson
Updates to TensorFlow 2.15. No StableHLO operators were added to Vela since these are subject to change and have almost no runtime support. - FlatBuffers version was unchanged. Change-Id: I9a506a2dcc2e0bc2498742e857bbb6d69b19ac1b Signed-off-by: William Isaksson <william.isaksson@arm.com> Signed-off-by: Rickard Bolin <rickard.bolin@arm.com>
2023-08-22MLBEDSW-7949: [MLCE] Remove duplicate cpu tensorsJohan Alfven
- If a npu op is followed by a convolution op than runs on the cpu, the optimized file ends up containing a duplicated tensor called _cpu. Functionality wise not a problem but the graph will look strange in a graph viewer. - This error was introduced when removing duplicate weights tensors but the above use case was not considered in that patch. - The fix is to make sure that only the weight and bias tensor are modified. Change-Id: I576f13650f1f9d3d50a421ab7100fc8b5ab62657 Signed-off-by: Johan Alfven <johan.alfven@arm.com>
2023-08-04MLBEDSW-7681: Add Vela version to output fileWilliam Isaksson
Adds Vela version to description and metadata Change-Id: I75fccd1a05a396612a249b8ec1662d8cae940ee6 Signed-off-by: William Isaksson <william.isaksson@arm.com>
2023-05-15MLBEDSW-7613: Crash when compiling model with resource variablesJohan Alfven
Fixed serializing of attribute container and shared_name that accidently got lost when fixing the crash for a faulty LSTM model. Change-Id: Ibd11da65735112bed4b1c8bcc4ef048bc093ebc4 Signed-off-by: Johan Alfven <johan.alfven@arm.com>
2023-05-04MLBEDSW-7504: Vela does not keep op version numberwilisa01
We now read operator code version, store it in operator and write it out to optimized file. Signed-off-by: wilisa01 <william.isaksson@arm.com> Change-Id: Idba672531d2e2a0203a85d3ffca9cf65ace85b47
2023-04-28MLBEDSW-7503: Avoid changing buffer index for models with only CPU opsJohan Alfven
- When compiling a model that only contains CPU ops, Vela unnecessary adds an empty buffer. - This extra buffer is added due to that the fast scratch tensor always occupies index 1. - Since scratch and fast_scratch does not have any constant data they can use buffer 0. Change-Id: I25e1fb124deed7069641bde1f571b522c5bf763a Signed-off-by: Johan Alfven <johan.alfven@arm.com>
2023-04-24MLBEDSW-7501: Vela unnecessary adds reshaped weights tensorsJohan Alfven
- Weights are internally cloned and reshaped/transposed when running on the NPU. This happens already in the reader. If the op is passed through to the CPU there are code that writes backs these clones but with another round of reshape/transpose. This adds extra tensors in the optimized file compared to the original file if the original tensors are subgraph inputs. - If the op is passed trough to the CPU the clones should not be written to the file. Solved this by setting the src_tensor when making the clone. Change-Id: I9f55d542c099882882920bffe8e15b43b2ca2c8d Signed-off-by: Johan Alfven <johan.alfven@arm.com>
2023-04-21MLBEDSW-7373: Vela sometimes write empty buffers in incorrect formatTim Hall
- Fixed an issue whereby a zero length buffer was written out instead of an empty buffer - Added a warning message to highlight when this type of semantically incorrect empty buffer is read from an input network Change-Id: Iac3bc71a2dbfda53737bbeb6e7f895552f0f13d0 Signed-off-by: Tim Hall <tim.hall@arm.com>
2023-04-21MLBEDSW-7408: MLCE: Crash when serialising model LSTMTim Hall
- Added checking and reporting of missing operator attributes when reading and writing TFLite file - Added a TFLite semantic check to ensure that all required attribute fields of builtin operators are read - Added some sanity checks for RESHAPE operators that run on the Ethos-U - Stopped CPU operators from having their attributes modified Change-Id: I05700681acdb09554f5945819717c08a9457295c Signed-off-by: Tim Hall <tim.hall@arm.com>
2023-02-14MLBEDSW-7316: Fix crash for networks with resource variablesJohan Alfven
- The problem was that networks with resource variables have not been thought of. The major problem was the graph traversal where these ops were not visited resulting in an empty subgraph that resulted in the crash. - Fixed the problem by attaching virtual tensors to the ops simulating subgraph output. These tensors are only used to get the graph traversal to work. - Fixed serializing of attribute container and shared_name - Fixed subgraph index for operator CallOnce - All resource variable ops are pushed to the CPU Change-Id: I815f9c81baf7a3fbb686e895980b462f58208b6e Signed-off-by: Johan Alfven <johan.alfven@arm.com>
2022-11-16MLBEDSW-6620: Update copyright notice and yearsRickard Bolin
- Update copyright notices to use SPDX format and add OSS mail as contact. - Update years on files where it had been missed. Signed-off-by: Rickard Bolin <rickard.bolin@arm.com> Change-Id: I7e9715ea4e17b76252728c708e46df12ad67ab1f
2022-11-01MLBEDSW-7077: Store original tensor shape in optimized fileJohan Alfvén
- CPU side always needs to work we the original tensor shape. Due to a bypass memory optimization the IFM, produced by CPU, was stored with the wrong shape in the optimized file. - Store the original tensor shape so it can be correctly written to the optimized file. Change-Id: I666dbcb0acd806ad208c0f925a51dfc25421688b Signed-off-by: Johan Alfven <johan.alfven@arm.com>
2022-10-19MLBEDSW-6880: Add support for multiple subgraphsJohan Alfvén
- Vela failed to compile networks with multiple subgraphs because only cascaded passes in the root subgraph were used when extracting the live ranges. The fix is to extract the subgraph range live on Ops that have connected subgraphs. - The tf_writer did not handle multiple subgraphs in a correct way resulting in corrupt buffer data in the optimized tflite file. The buffer index must be unique for every tensor. -Added support to handle multiple subgraphs for the OfflineMemoryAllocation meta data. The change will not change behavior for single graphs. Signed-off-by: Johan Alfven <johan.alfven@arm.com> Change-Id: I2328dfc1f07e2e4faf43a75423ea95423096ffa3
2022-10-03MLBEDSW-6955: Update to TensorFlow 2.10erik.andersson@arm.com
- Updated to TensorFlow 2.10 and FlatBuffers 2.0.7 - Changed absolute to relative imports in the auto-generated code - Updated Vela's TFLite writer to support FlatBuffer builder's internal number of elements count - Removed use of deprecated numElems argument to FlatBuffer builder's EndVector() Signed-off-by: erik.andersson@arm.com <erik.andersson@arm.com> Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: If447778134db81ae0ac374c7397e1140082372fd
2021-11-05MLBEDSW-5209 Vela: output diff depthwise with non-zero zero pointsFredrik Svedberg
Fixed by adjusting zero points for ops with int8 IFM and asymmetric weights since the reference does not support asymmetric weights for int8 IFM and ignores the zero points. Signed-off-by: Fredrik Svedberg <fredrik.svedberg@arm.com> Change-Id: I2a206a01a471a53aa864a6a3616aa23d2a5a23c8
2021-07-26MLBEDSW-4892: Fix crash affecting biases without quantization.James Peet
Remove quant_values attribute from Tensor class. It only needs a single values attribute, holding either quantized or unquantized values as appropriate. Change-Id: Ie96f80ac58061b6077e0f7048dc60209fdfbcafa Signed-off-by: James Peet <james.peet@arm.com>
2021-07-09MLBEDSW-4839: Fix issues with Elementwise IFM/OFM overlapTim Hall
- Fixed typo with not using ifm.mem_type - Fixed bug with using ifm1 properties when only ifm2 is a potential match - Removed restriction on not considering SHL and SHR for overlap - Removed some dead reshape code Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: Id9bcc3c2b3ee9ac7b6276187d3e2f513b4acd4b5
2021-07-09MLBEDSW-4840 Move setting of input indices to tflite readerPatrik Gustavsson
Mapping to internal input indexing has been added to tflite_reader.py and tosa_reader.py. And the other way around in tflite_writer.py. Signed-off-by: Patrik Gustavsson <patrik.gustavsson@arm.com> Change-Id: I4d8596e747cfa7c4203884c4e785eb1977e2bcc1
2021-07-05MLBEDSW-3890 handling scratch tensorSamuel Panijel
vela: Possible issue with handling scratch tensor on non-ethosu custom op Fixing a case where a tensor input name ends with "scratch". 4 test cases passing this change: 1) non-optimized tflite - input tensor name is _split_1_scratch 2) optimized tflite - input tensor name is _split_1_scratch 3) optimized tflite - input tensor name is _split_1_scratch and custom operation name is non_ethus_u 4) non-optimized tflite - input tensor name is _split_1_scratch_fast Change-Id: Ia515805825b7f9a646607c5075b7ea3a0cf6aad8 Signed-off-by: Samuel Panijel <samuel.panijel@arm.com>
2021-02-17[MLBEDSW-3813] Fix LSTM operator pass throughFredrik Svedberg
Fixed pass through of LSTM operator. Change-Id: I23140c69ab6cdc83f6bb8129256b4cc6a7c5ffac Signed-off-by: Fredrik Svedberg <fredrik.svedberg@arm.com>
2021-02-05MLBEDSW-3771: Updated to TF 2.4 flatbuffer schemaTim Hall
Updated tflite loader and mappings from tensorflow 2.3 to tensorflow 2.4 Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: I55884000ee139baf639bb0377008e0534f72fe94
2021-01-28[MLBEDSW-3891] Fix reading back in an ethos-u custom opFredrik Svedberg
Fixed assertion when reading back in an ethos-u custom op. Signed-off-by: Fredrik Svedberg <fredrik.svedberg@arm.com> Change-Id: I275ec9187ffead1e96f2522ecbd658328fa4ef69
2021-01-25MLBEDSW-3847: MLCE: Vela not handling multiple custom operators correctlyTim Hall
- Fixed bug with multiple 3rd party custom operators not inserting the correct custom_code. Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: I470a964867e60d4d71f01592dd33d4ad1aa2d441
2020-12-14MLBEDSW-2066 Improve Exception messagesMichael McGeagh
Minor refactoring to use fstrings. Improve Error classes to correctly inherit the base class. Use existing exception classes instead of plain exceptions where it makes sense. Signed-off-by: Michael McGeagh <michael.mcgeagh@arm.com> Change-Id: I0941c04e91010da1db77299517a8e2d896371e77
2020-12-08MLBEDSW-2836 Change sets to tuplesMichael McGeagh
Replace conditional checks against sets with tuples. If not requiring uniqueness, or complex set operations, it is quicker to use tuples instead. Signed-off-by: Michael McGeagh <michael.mcgeagh@arm.com> Change-Id: Ie8732c8d46067244963936c53f0ec81adda50372
2020-11-16MLBEDSW-3483, KeyError "fused_activation_function"Louis Verhaard
Bug fix for a regression: Vela could crash for operators placed on CPU. Signed-off-by: Louis Verhaard <louis.verhaard@arm.com> Change-Id: I99dcfdb4d3029ad86ffd2c8b3fd2547554794b79
2020-11-13MLBEDSW-839: Code generation using external API2.0.0.rc1Louis Verhaard
Added external API to generate register command streams. Existing code generation has been refactored to make use of this API. Change-Id: Ibb4c2b167809869f16470b14da24f08a65c82b7b Signed-off-by: Louis Verhaard <louis.verhaard@arm.com>
2020-11-04MLBEDSW-1974: Set Scratch buffers sizeJacob Bohlin
Set the actual size of the Scratch and Fast Scratch buffer and remove both Scratch buffers from the subgraph inputs. Signed-off-by: Jacob Bohlin <jacob.bohlin@arm.com> Change-Id: I9e4213f48289d9136cdd4cd43c668d37c6af8530
2020-10-14Revert "MLBEDSW-3219: Suppress CPU info for Const/Placeholder"patrik.gustavsson
This reverts commit 04986c0016e59993563490fe67052371fc0e1ad2. Reason for revert: Merged by mistake Change-Id: I150ad9ba7074ad1e80f21180aeba56a454d9f748
2020-10-14MLBEDSW-3219: Suppress CPU info for Const/PlaceholderLouis Verhaard
Suppress info print that Const/Placeholder/SubgraphInput are not supported on the NPU. Change-Id: I689d25481df0cd10487484c9f639e4253df081ee Signed-off-by: Louis Verhaard <louis.verhaard@arm.com>
2020-10-12MLBEDSW-3220: missing "fused_activation_function" caused crashLouis Verhaard
Vela could crash in operator serialization if "fused_activation_function" was not set. Change-Id: I7f2364b0849fd371dee87e26c6d33d44ce8cec26 Signed-off-by: Louis Verhaard <louis.verhaard@arm.com>
2020-10-08MLBEDSW-3148: Refactor OperationLouis Verhaard
- op.type is now an enum instead of a string - Removed unused operator codes - Refactored some attributes like npu_block_type, fused_activation_function - Refactored operator index calculation - Refactored a number of operator sets Change-Id: I641f65ee375794b7aec42abc0664251ae37d78e8 Signed-off-by: Louis Verhaard <louis.verhaard@arm.com>
2020-09-22MLBEDSW-2813: Handle non-const weights and check shapesAndreas Nevalainen
- Added check for non-constant weights in supported operators - Added check ifm & ifm2 shapes - Handle None tensors for CPU operators - Handle missing attributes for Cast operator Signed-off-by: Andreas Nevalainen <andreas.nevalainen@arm.com> Change-Id: I2f16d3d44d0c6da5237550b39273cdb9cc3c7607
2020-09-07MLBEDSW-2874 Fix writing out empty tflite filesMichael McGeagh
If a tflite file with no ops but just the input/output tensor is given, vela wrote an empty optimised tflite file with no tensors given. This fixes that by allowing all placeholder tensors to also be serialised on write. Signed-off-by: Michael McGeagh <michael.mcgeagh@arm.com> Change-Id: If79817100869e712a75264889f401e38de0b1e7a
2020-08-21MLBEDSW-2646: Refactor unknown operator serialisation1.2.0.rc1Tim Hall
- Minor cleanup of register command stream generator too Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: I0514622402ee9b0557769dd7c7decfddecc87ffa
2020-08-20MLBEDSW-2783 Vela crashed on empty tflite fileMichael McGeagh
There may be cases where after optimisations, there are no operators contained within the subgraph. Upon serialising and writing out the vela optimised tflite file, it would crash for such a corner case. This fixes it allowing it to not crash but instead write out the empty tflite file. Signed-off-by: Michael McGeagh <michael.mcgeagh@arm.com> Change-Id: Ia879d1ffdbab21706b15e99aa107fb2d8d4dd3de
2020-08-12MLBEDSW-2383 Preserve previous metadataMichael McGeagh
The input tflite file potentially has metadata attached to it, which was lost when writing the vela optimised tflite file out. This patch preserves any metadata found. Signed-off-by: Michael McGeagh <michael.mcgeagh@arm.com> Change-Id: I7b4e941696d21b81802fd4398cd405323778bedf
2020-07-14MLBEDSW-2641: Fix crash for network with only CPU opsJacob Bohlin
Signed-off-by: Jacob Bohlin <jacob.bohlin@arm.com> Change-Id: Ia9c70d62c6abc827cbdf73a8bb37afd595796741
2020-07-07MLBEDSW-2551 Add support for more mem-cfgsPatrik Gustavsson
Added support for one more memory configuration- Change-Id: Iac19992386e3e9b80bd519acb1b0a399c47d736f Signed-off-by: Patrik Gustavsson <patrik.gustavsson@arm.com>
2020-07-02MLBEDSW-2340: Make the tensor address default NoneCharles Xu
Signed-off-by: Charles Xu <charles.xu@arm.com> Change-Id: I53d9d56acee57cff208dccb4822c1f1a461c416d
2020-06-25MLBEDSW-2306 Added more supported mem-cfgsPatrik Gustavsson
Additional supported memory configurations: -Permanent_storage = DRAM -Tensor arena either in DRAM or SRAM Signed-off-by: Patrik Gustavsson <patrik.gustavsson@arm.com> Change-Id: I20beb7151e306bfdba540e7c0b2a7b478b4d94e1
2020-06-18MLBEDSW-2528: MLCE-219: Custom operator pass throughTim Hall
- Fixed custom operator pass through - Added error printing functions for operators and tensor - Minor cleanup of custom exception handling Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: Idf295df1e4c544381dc480244d880c32fb285e38
2020-06-18vela: Add support for CPU only networksTim Hall
- Fix various problems when no operators run on Ethos-U55 Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: I44a1a914fabb7ca26c921a02753da8abeecd9c7b
2020-06-18Add reorder-python-import pre-commit hookDiego Russo
Also updated README.md Change-Id: I118309c61f4d00e8508d6b888c606995490fba39 Signed-off-by: Diego Russo <diego.russo@arm.com>
2020-06-18Add pre-commit support for sanity checksDiego Russo
Use pre-commit framework [1] to run black and flake8 before the commit. black and flake8 are managed by the pre-commit framework and they can be run manually by the user using `pre-commit run` command. Fix the code base with the help of black and flake8. Fix import statements according to PEP8 guidelines [1] Both tools have the following settings (specified in the pre-commit configuration file): * line length: 120 characters * directory to exclude: ethosu/vela/tflite/ and ethosu/vela/ethos_u55_regs Updated README.md on how to install pre-commit and how to run sanity checks. Pipenv files have been updated including new dependencies for pre-commit. [1]: https://www.python.org/dev/peps/pep-0008/#imports [2]: https://github.com/pre-commit/pre-commit Change-Id: I304d9fffdf019d390ffa396a529c8a7c2437f63d Signed-off-by: Diego Russo <diego.russo@arm.com>
2020-04-29Add Vela codebase0.1.0Tim Hall
- Added modules ethosu.vela and ethosu.mlw_codec. - Added README and various configuration files. Change-Id: I3690f8c8f5966306ecddaeb2793c30ca9c6e2eee
generated by cgit v1.2.1 (git 2.23.0) at 2024-06-01 14:11:54 +0000