aboutsummaryrefslogtreecommitdiff
path: root/ethosu/vela/weight_compressor.py
AgeCommit message (Collapse)Author
2024-03-07TOSA fixesOscar Andersson
- Fix TOSA imports - Handle weights connected to Identity nodes - Scaling info was missing in Fully Connected - Disable rescaling fusing for conv-like ops - Explicit scaling was missing for conv-like ops - Handle Const->Identity->Transpose chains - Handle Const->Identity->Reshape chains Change-Id: I063af1f187b6b56105ccf5e8e8b2eb0d3a39dd3b Signed-off-by: Oscar Andersson <oscar.andersson@arm.com>
2024-01-26MLBEDSW-8575 Tests fails on conv networksFredrik Svedberg
Fixed a problem where the compiler incorrectly called the mlw_codec to create an empty weight stream for the second weight core. Also added code to the mlw_codec to detect this as an value error rather than a memory error. Change-Id: I463846cecb1178f8fbf04dc3e39bd6965cb8ddfc Signed-off-by: Fredrik Svedberg <fredrik.svedberg@arm.com>
2023-05-17MLBEDSW-7223: Fusing Pad and AvgPool causes diffTim Hall
- Fixed an issue with the fusing of PAD and AVERAGE_POOL_2D whereby the rounding away from zero didn't work because it requires the zero point to be at zero but the input padding required it to be set to the desired zero point. This affected both int8 and int16. The solution was to remove it by using the bias prior to the scaling - Refactored the rounding away from zero mode Change-Id: I8f2df69df06d2a9722315c346646e5a901cb2c3b Signed-off-by: Tim Hall <tim.hall@arm.com>
2023-05-15MLBEDSW-7428: Remove unused rescale_for_fafRickard Bolin
Remove unused parameter rescale for faf Change-Id: Id388d307f3eb0d27bce813ab58e3c9a5f4ba89ae Signed-off-by: Rickard Bolin <rickard.bolin@arm.com>
2023-05-04MLBEDSW-7542: Fix output diff caused by wrong scaling in Conv2dJohan Alfven
- The reference calculates the scale slightly different between Conv2d and FullyConnect. Recently a fix was submitted to address this issue. However, internally a Conv2d can be changed to a FullyConnect but then the scale must still be calculated following the Conv2d reference. - The fix is to check the original type if FullyConnect scale should be used or not. Change-Id: I5a9fb49126f0df63712b73fb5520fdc604cee378 Signed-off-by: Johan Alfven <johan.alfven@arm.com>
2023-04-04MLBEDSW-7442: Removed ofm quantization for ArgMaxJohan Alfven
- Quantization for the OFM was added for the ArgMax operator as a workaround in order to avoid a crash in the weight compressor. This quantization is now removed. - The weight compressor expects that all tensors have a quantization. Updated code to use scale = 1.0 and zero point = 0 for tensor without quantization. Change-Id: I6816dce2db55f7d795d19f88d7fbe7ee419347fc Signed-off-by: Johan Alfven <johan.alfven@arm.com>
2023-03-13MLBEDSW-7427 Fix scale calculations for FullyConnectedFredrik Svedberg
Fixed scale calculations for FullyConnected to match the reference. Also removed unused low_precision_scaling. Change-Id: I4b766febff4a0010acd3de708bb49be458d22bf3 Signed-off-by: Fredrik Svedberg <fredrik.svedberg@arm.com>
2023-02-09MLBEDSW-6982: Move to setup.cfg and pyproject.toml3.7.0.rc1Raul Farkas
- Move all static information from setup.py to newly added pyproject.toml - Add setup.cfg used for static information that cannot be added to pyproject.toml due to it still being in beta. - Modify mlw_codec to to throw a real python exception when importing NumPy arrays instead of just printing them to stdout. - Surround mlw_codec import with try catch statement to catch NumPy C API mismatch errors and throw them again with a more detailed message. - Update README.md with documentation about known issue with changing used NumPy version after installing ethos-u-vela. Change-Id: I1eeee5536be7c1744e30d6088f7069fbb1403e06 Signed-off-by: Raul Farkas <raul.farkas@arm.com>
2022-11-16MLBEDSW-6620: Update copyright notice and yearsRickard Bolin
- Update copyright notices to use SPDX format and add OSS mail as contact. - Update years on files where it had been missed. Signed-off-by: Rickard Bolin <rickard.bolin@arm.com> Change-Id: I7e9715ea4e17b76252728c708e46df12ad67ab1f
2022-09-23MLBEDSW-6686: Resize bilinear HPC with tile paddingRickard Bolin
- Added support for Resize Bilinear with half pixel centers for int8 and uint8. - Utilizes the new "TILE" padding mode. - Utilizes ofm stride multipliers and modified tile base offsets to write OFMs interleaved. Signed-off-by: Rickard Bolin <rickard.bolin@arm.com> Change-Id: I37fa77c022a368f05fda0ead75d8696c9205f833
2022-09-21MLBEDSW-4338 Randomized int16 PAD output diffFredrik Svedberg
The issue was that the AveragePool in these test cases were translated to DepthwiseConv2DBias and int16 convolutions always runs with reduced scale. Fixed so that reduced scale is not used in this case. Signed-off-by: Fredrik Svedberg <fredrik.svedberg@arm.com> Change-Id: Ice956eabbb37c8aa1991464870006971c6ecec43
2022-05-16MLBEDSW-6263: Use separate tensors for double bufferingRickard Bolin
Uses separate tensors for the individual weight buffers in case of weight double buffering. Each weight buffer tensor gets its own individual live range. This patch is a clone of a previously reverted patch, but with some additional bug fixes applied. Signed-off-by: Rickard Bolin <rickard.bolin@arm.com> Change-Id: I868c70d15821eb9f1399186f2da6e7345f6ee343
2022-05-04Revert "MLBEDSW-6263: Use separate tensors for double buffering"Tim Hall
This reverts commit cc5f4de1c35ba44fca7ff6295c6ae846f8242344. Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: I0fa5babfe9ad9ec668720d04fe1c16d9a9092131
2022-03-30Update version of Black to 22.3.0Jonas Ohlsson
Update version of Black to 22.3.0 due to updated dependencies. Updates to fix reported issues due to new version. Signed-off-by: Jonas Ohlsson <jonas.ohlsson@arm.com> Change-Id: I60056aae452093ce8dcea1f499ecced22b25eef1
2022-03-30MLBEDSW-6263: Use separate tensors for double bufferingLouis Verhaard
Uses separate tensors for the individual weight buffers in case of weight double buffering. Each weight buffer tensor gets its own individual live range. Change-Id: I724a8c61a7045615fbd2ed9535663076ac8edd13 Signed-off-by: Louis Verhaard <louis.verhaard@arm.com>
2022-03-21MLBEDSW-3367 Add mypy to pre-commitJonas Ohlsson
Add mypy to pre-commit and clean up all reported errors. Signed-off-by: Jonas Ohlsson <jonas.ohlsson@arm.com> Change-Id: If7dc869f5fecdb0e2db40f14e7d9db21aa33df71
2021-09-03TOSA: Added support for ADD, SUB and MULPatrik Gustavsson
Added support for ADD, SUB and MUL Signed-off-by: Patrik Gustavsson <patrik.gustavsson@arm.com> Change-Id: I52acdc126b16e2cf4096bcf7a77023ea7d204998
2021-07-26MLBEDSW-4892: Fix weight compression of asymmetrically dilated kernels.James Peet
Signed-off-by: James Peet <james.peet@arm.com> Change-Id: I5bf39aa4f1fb48bcb0423edc4cd1d01f59aac1db
2021-07-26MLBEDSW-4892: Fix crash affecting biases without quantization.James Peet
Remove quant_values attribute from Tensor class. It only needs a single values attribute, holding either quantized or unquantized values as appropriate. Change-Id: Ie96f80ac58061b6077e0f7048dc60209fdfbcafa Signed-off-by: James Peet <james.peet@arm.com>
2021-07-08MLBEDSW-4838 Added basic TOSA support.Patrik Gustavsson
Added basic TOSA support, enabling Vela to read and compile a .tosa file corresponding to CONV2D + Rescale + Clamp, and writing it to an optimized .tflite file. The optimized .tflite file, will in this case, hold a commandstream where the Rescale and Clamp has been fused into the CONV2D. The optimized tflite file is not output from Vela. -Added support to read .tosa file into Vela internal structure. - Added tosa_reader.py, tosa_mapper.py and helper files stored under tosa/ - Support for this limited to ~10 ops -Added reader_util.py for functions common for TOSA and TFLite -Added tosa_graph_optimiser.py -Added support to fuse Rescale into convolution -Modified handling for padding -Added support to fuse Clamp to previous op -Added graph_optimiser_util.py -Moved functions common for TOSA/TFLite graph optimization to this file. -Renamed graph_optimiser.py to tflite_graph_optmiser.py -Added separate tosa_supported_operators.py -Added supported_operator_util.py -For functions in common for TOSA/TFLite Signed-off-by: Patrik Gustavsson <patrik.gustavsson@arm.com> Change-Id: Ic3c540504ec8c5eb4771397fdc6882050ecf33ab
2021-06-25MLBEDSW-4819: MLCE: weight_compressor int has no attribute astypeTim Hall
- Added type checking so that the correct type conversion can be used Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: Ia83f46029fac7bad63844c090b87d23c2072b105
2021-06-09mlw_codec: Fixed alignment warningMauricio Briceno
- Restructured pointer API to prevent alignment warnings - Changed weight tensor data type to np.int16 Change-Id: I310c1ca733bf98724c84e8b2194becb4be3e7eea
2021-06-08MLBEDSW-4602: Fix Deepspeech scale & bias reuse issue.Tim Hall
- Deepspeech reuses identical weights and biases throughout the network. Since biases are now interleaved with weights there is a scaling issue when the ifm scales differ between operations using the same weight and scale tensor. - This commit uses interleaved weights/scales on their first use but separates scales to source memory on subsequent use (if the ifm scale is different). Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: I7aae163438160a919cae04e235966e75355a6148
2021-05-27MLBEDSW-4034: New Scheduler Size or Performance OptimisationTim Hall
- Merged dev/scheduler at 83639f90e8c828f70de6e29142355a940224959b Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: I0050529d4b42da93768c7264296434dd877fb5b4
2021-05-07weight_compressor: added mlw_reorder_encode3.0.0.rc1Mauricio Briceno
- Moves reordering to C - Runtime is greatly minimized for encoding weights Change-Id: Ifff01e7b1ea6d5cec68310a155c3b80aa1a38545 Signed-off-by: Mauricio Briceno <mauricio.briceno@arm.com>
2021-05-06[MLBEDSW-4254] Improve weight information in summaryFredrik Svedberg
Improved weight information showed in summary if --verbose-weights option is used. Signed-off-by: Fredrik Svedberg <fredrik.svedberg@arm.com> Change-Id: Iac142f2a813bf1c05aa9da3f8a384466e2914d06
2021-02-25MLBEDSW-1499: Add MEAN operatorDwight Lidman
This commit adds support for the MEAN operator, with some caveats. Signed-off-by: Dwight Lidman <dwight.lidman@arm.com> Change-Id: I165cb26cb5aefd68e70d2cfc68291ccf7b778921
2021-02-25MLBEDSW-4064: Update copyright headerserik.andersson@arm.com
All files which have been updated in 2021 and contain a copyright header have had their headers updated. Signed-off-by: erik.andersson@arm.com <erik.andersson@arm.com> Change-Id: Ia682111a719d16e690433398ccfb69c7e93c1cd1
2021-02-02MLBEDSW-3927: Fix wrong compression ratioDiqing Zhong
Change-Id: I06feeb98fb48badf06097f377a9504e6f4eeae91 Signed-off-by: Diqing Zhong <diqing.zhong@arm.com>
2020-12-14MLBEDSW-2066 Improve Exception messagesMichael McGeagh
Minor refactoring to use fstrings. Improve Error classes to correctly inherit the base class. Use existing exception classes instead of plain exceptions where it makes sense. Signed-off-by: Michael McGeagh <michael.mcgeagh@arm.com> Change-Id: I0941c04e91010da1db77299517a8e2d896371e77
2020-12-08MLBEDSW-2836 Change sets to tuplesMichael McGeagh
Replace conditional checks against sets with tuples. If not requiring uniqueness, or complex set operations, it is quicker to use tuples instead. Signed-off-by: Michael McGeagh <michael.mcgeagh@arm.com> Change-Id: Ie8732c8d46067244963936c53f0ec81adda50372
2020-11-23MLBEDSW-3424: Expose API through separate fileLouis Verhaard
All external APIs are now exposed by api.py. Signed-off-by: Louis Verhaard <louis.verhaard@arm.com> Change-Id: I33f480e424692ac30e9c7d791f583199f31164a7
2020-11-20vela: Rename Yoda to Ethos-U65Tim Hall
- Also changed to use Ethos-U where appropriate Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: Ie45ba2bb3935b305abe897b78b498681296cb7c1
2020-11-13MLBEDSW-839: Code generation using external API2.0.0.rc1Louis Verhaard
Added external API to generate register command streams. Existing code generation has been refactored to make use of this API. Change-Id: Ibb4c2b167809869f16470b14da24f08a65c82b7b Signed-off-by: Louis Verhaard <louis.verhaard@arm.com>
2020-11-11MLBEDSW-3222: Bias tensors in fast storageAndreas Nevalainen
For IFM streamed cascades bias tensors are read several times. Moves these tensors to fast storage and add DMA commands. Change-Id: I630f6275986c1b5e3f126c925b11e22500fb1128 Signed-off-by: Andreas Nevalainen <andreas.nevalainen@arm.com>
2020-10-08MLBEDSW-3148: Refactor OperationLouis Verhaard
- op.type is now an enum instead of a string - Removed unused operator codes - Refactored some attributes like npu_block_type, fused_activation_function - Refactored operator index calculation - Refactored a number of operator sets Change-Id: I641f65ee375794b7aec42abc0664251ae37d78e8 Signed-off-by: Louis Verhaard <louis.verhaard@arm.com>
2020-10-02MLBEDSW-3060 Adjust check if weights fit in sramPatrik Gustavsson
When deciding if weights fit sram: A compression of the weights has been added when a weight compression test limit makes it impossible to fit weights in a double buffer in sram. The worst compression ratio from compression, is used to decide if weights can be fit in sram. Signed-off-by: Patrik Gustavsson <patrik.gustavsson@arm.com> Change-Id: I9458769866b3f9fc15659185aae09658ed10fb38
2020-09-23MLBEDSW-3070: Fix addressing of weightsLouis Verhaard
Assign different equivalence ids to weights with same values but different compression, to ensure correct addressing. Signed-off-by: Louis Verhaard <louis.verhaard@arm.com> Change-Id: I13aabad71520e4f4a78fb2d6a81740bdd4d1256c
2020-09-08MLBEDSW-2935: LUT fusing with preceding operatorLouis Verhaard
Allows fusing of LUT with a preceding operator regardless of input/output scale. Change-Id: Ia378adbb3fe61d71299feb085f7313377e0efa39 Signed-off-by: Louis Verhaard <louis.verhaard@arm.com>
2020-08-31MLBEDSW-2804: Removed invalid assert for shift rangeJacob Bohlin
Signed-off-by: Jacob Bohlin <jacob.bohlin@arm.com> Change-Id: I857aeb7aeb34f4b8ea47e6ac954cead268335e32
2020-08-18MLBEDSW-2589: Skip weight compression for CPU opsDwight Lidman
This commit fixes a bug where CPU ops were getting passed on as NPU ops in weight_compressor.py due to Operation.find_npu_op() incorrectly returning any op with an 'npu_block_type' attribute (which every op has) as an NPU op. Signed-off-by: Dwight Lidman <dwight.lidman@arm.com> Change-Id: I7a758f8d1b1237907816bc1be7b77aff765ae688
2020-08-12[MLBEDSW-2749] removed the decorator for typecheckManupa Karunaratne
*the decorator is causing the verification tests to fail when using TF 2.1, but not with TF 2.2, hence removing it for now. Change-Id: I07357c0fef383d9a65278fe99ad8e4d3f7dc6d9b Signed-off-by: Manupa Karunaratne <manupa.karunaratne@arm.com>
2020-08-12MLBEDSW-2684: Fix weight compression scale calculations for FCJacob Bohlin
Fixed an issue with Fully Connected weights' shape used for compression scale calculations causing incorrect performance estimates. Signed-off-by: Jacob Bohlin <jacob.bohlin@arm.com> Change-Id: Id3a5c187ad3e942b8e3d4c690b3dbba3c6fda922
2020-08-06[EXTAPI] exposing encode of biases to be consumed by an external APIManupa Karunaratne
*Renamed pack_bias_and_scale to encode_bias to be consumed externally *added unit test for the API Change-Id: I71829f3fcb390c475795848f0be3d132d3e158ee Signed-off-by: Manupa Karunaratne <manupa.karunaratne@arm.com>
2020-07-27[EXTAPI] refactor weight compression to be used by an external consumerManupa Karunaratne
*lint *added unit tests *added typecheck *added docstring for the api Change-Id: Ibd4bc40d4381ac40ad2ea3d500b26c4ec565ab07 Signed-off-by: Manupa Karunaratne <manupa.karunaratne@arm.com>
2020-07-10MLBEDSW-2634 Avoid encoding empty weights streamPatrik Gustavsson
Avoid encoding empty weights stream. Signed-off-by: Patrik Gustavsson <patrik.gustavsson@arm.com> Change-Id: I120ede14f19705e169c5f03ed344036a58b5f84f
2020-07-08MLBEDSW-2625 Alignment of tensor for bias and scalePatrik Gustavsson
Fix for alignment of tensor for bias and scale Change-Id: I303a225a536f169909cec9ba4d5cee088110bb94 Signed-off-by: Patrik Gustavsson <patrik.gustavsson@arm.com>
2020-07-07MLBEDSW-2615: MLCE: Ethos-U55 Flash usage increases due to Yoda BetaTim Hall
- Fixed bug with the size of the scale and bias tensor Signed-off-by: Tim Hall <tim.hall@arm.com> Change-Id: I4d267d4c918a5c834ebdff82de4f021717e95203
2020-07-07MLBEDSW-2548: Fix for Double Buffer size estimateJacob Bohlin
This will give a worst case estimate of the Double Buffer size in the Scheduler and it will no longer be able to choose strategies that end up with a buffer that doesn't fit in SRAM. Signed-off-by: Jacob Bohlin <jacob.bohlin@arm.com> Change-Id: I763731f63c7672679f3b8cd6db65dad03b946ae5
2020-06-25MLBEDSW-2556: Odd core/block depth weight interleaving updateTim Hall
- If blockdepth or core count resulted in empty or non-existent substreams, the command generator generated an error. This commit changes the command stream generator to only program cores that have streams and are enabled for the configuration. Change-Id: I4e724b19de14d3a12e886ec6b17d0038593dfb59 Signed-off-by: Tim Hall <tim.hall@arm.com>
generated by cgit v1.2.1 (git 2.23.0) at 2024-06-15 00:14:00 +0000