aboutsummaryrefslogtreecommitdiff
path: root/src/core/CL/kernels/CLGEMMMatrixMultiplyKernel.cpp
AgeCommit message (Collapse)Author
2019-08-02COMPMID-2539: Add bias addition check in CLGEMM validationGian Marco Iodice
Change-Id: Ib33574662d2b62ce80dd7f74a656199ed64225bc Signed-off-by: Gian Marco Iodice <gianmarco.iodice@arm.com> Reviewed-on: https://review.mlplatform.org/c/1676 Reviewed-by: Michele Di Giorgio <michele.digiorgio@arm.com> Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Tested-by: Arm Jenkins <bsgcomp@arm.com>
2019-07-23COMPMID-1979: Fuse Activation Function in CLGEMM - part 3Gian Marco Iodice
Fused beta*bias in in the old cl gemm kernels Fused activation function in the old cl gemm kernels Change-Id: I695fb9189e6d4792010abd256784624982d17d79 Signed-off-by: Gian Marco Iodice <gianmarco.iodice@arm.com> Reviewed-on: https://review.mlplatform.org/c/1587 Reviewed-by: Giuseppe Rossini <giuseppe.rossini@arm.com> Comments-Addressed: Arm Jenkins <bsgcomp@arm.com> Tested-by: Arm Jenkins <bsgcomp@arm.com>
2019-06-11COMPMID-2380: Create utility functions for is_one and is_zero with floatGian Marco Iodice
Change-Id: If5b968e19cf830d5472395a1b43bf72a456fd331 Signed-off-by: Gian Marco Iodice <gianmarco.iodice@arm.com> Reviewed-on: https://review.mlplatform.org/c/1322 Reviewed-by: Michele Di Giorgio <michele.digiorgio@arm.com> Comments-Addressed: Arm Jenkins <bsgcomp@arm.com> Tested-by: Arm Jenkins <bsgcomp@arm.com>
2019-02-07COMPMID-1706: Fuse the bias addition within CLGEMMMichele Di Giorgio
Change-Id: I378f2023f4fa010f195f76716ac07aa86279bfae Signed-off-by: Michele Di Giorgio <michele.digiorgio@arm.com> Reviewed-on: https://review.mlplatform.org/280 Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Gian Marco Iodice <gianmarco.iodice@arm.com>
2019-01-25COMPMID-1866: Revisit padding and window on CLDepthwiseConvolutionNHWCMichele Di Giorgio
Change-Id: I1fd20ebb5f3295722c833a599ea82798ceff6e01 Reviewed-on: https://review.mlplatform.org/471 Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Gian Marco Iodice <gianmarco.iodice@arm.com>
2019-01-14Issue COMPMID-1835: Remove CLGEMMInterleave4x4Kernel and replace with ↵giuros01
CLGEMMReshapeLHSMatrixKernel Change-Id: Id6a1bd78f9b1698b64a004e4adebc41002b15745 Reviewed-on: https://review.mlplatform.org/496 Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Gian Marco Iodice <gianmarco.iodice@arm.com>
2018-12-21COMPMID-1836: Remove CLGEMMTranspose1xWKernel and replace with ↵giuros01
CLGEMMReshapeRHSMatrixKernel Change-Id: Ic5a4f32657a155380684dcd4b44fbb608ef40cb4 Reviewed-on: https://review.mlplatform.org/418 Reviewed-by: Gian Marco Iodice <gianmarco.iodice@arm.com> Tested-by: Arm Jenkins <bsgcomp@arm.com>
2018-11-20COMPMID-1801 : (Nightly) CLWinogradConvolutionLayer FP16 mismatchesVidhya Sudhan Loganathan
FP mixed precision support added to GEMM kernel used for fp16 winograd conv on Midgard GPUs Change-Id: I1619beb025fc484a1ac9d3e528d785edabbc7ee6
2018-11-16COMPMID-1266 : Add support for FP16 in CLWinogradConvolutionLayer: 5x5 kernelsVidhya Sudhan Loganathan
Introduced F32 accumulation for F16 winograd gemm and output transform WinogradConvolution will be available for F16 only if fast math flag is enabled Change-Id: I215593c205236a0f9669218437bb40b184ec6a4f
2018-11-08COMPMID-1736: Fixed out-of-bound write in CLIm2ColGian Marco Iodice
The issue was related to CLIm2Col when the number of input channels was less than the number of elements processed by each thread. The bug has been fixed in the validate_and_configure_window() function setting the correct number of elements accessed in the output tensor. Also fixed an issue GEMM3D when we have a single output channel Change-Id: I094292d0c7662599c4a4c3916ec5f5821df5faef
2018-11-02COMPMID-1451: Reverting changes for CLGEMM and CLGEMMLowp previuosly done ↵Isabella Gottardi
(384496) Mirroring CLGEMM behaviour to CLGEMMLowp Change-Id: I308b54e2c0de131a5322b77e83e7454db498d692 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/153175 Reviewed-by: Gian Marco Iodice <gianmarco.iodice@arm.com> Tested-by: bsgcomp <bsgcomp@arm.com>
2018-11-02COMPMID-1607 - (Nightly) CLGEMMLowpMatrixMultiplyCore errors and mismatchesIsabella Gottardi
Change-Id: I5f2e6843526cb154176a5b113627d4f36c3a8edd Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/150967 Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Tested-by: bsgcomp <bsgcomp@arm.com>
2018-11-02COMPMID-1607 - (Nightly) CLGEMMLowpMatrixMultiplyCore errors and mismatchesIsabella Gottardi
COMPMID-1608 - (Nightly) CLGEMMConvolutionLayer QASYMM8 errors and mismatches COMPMID-1609 - (Nightly) CLFullyConnectedLayer QASYMM8 mismatches Change-Id: I84c0d4f468be892f437f9f38b964dc7dfb66663a Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/150869 Reviewed-by: Michele DiGiorgio <michele.digiorgio@arm.com> Tested-by: bsgcomp <bsgcomp@arm.com>
2018-11-02COMPMID-708 Fix AccessWindowTransposeGiorgio Arena
Change-Id: I68f65b6dea7889d71b4a10021f59e6f0ab82903b Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/145590 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-1478: Stop relying on static default OpenCL objects in cl2.hppAnthony Barbier
This causes problems when ACL is used as a shared library on Android. Fixes some problems related to creation / destruction order between the Graph's CL backend and core / runtime Change-Id: I716d63fd42f4586df1ffbb6fa97e4db06d3a781b Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/143228 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Michele DiGiorgio <michele.digiorgio@arm.com> Reviewed-by: Gian Marco Iodice <gianmarco.iodice@arm.com>
2018-11-02COMPMID-1188 - Fixed performance degradation with GEMM3DGian Marco Iodice
The previous implementation of GEMM3D degradated the performance when the input had to be reinterpreted as 3D. However if both input and output have to be reinterpreted as 3D, we can skip the offset calculation for that specific case and run the multi GEMM approach Change-Id: I0d5d48add2c6ccdebfbb268ea199dd181101f3aa Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/142872 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-1276 - Allow GEMM to work with 3D input tensorGian Marco Iodice
Skipped im2col in CLGEMMConvolutionLayer for 1x1 convolutions with NHWC data layout Change-Id: I894e6b952ed8605e8f3ffc0ffc25c24730d4664c Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/141909 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com> Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com>
2018-11-02COMPMID-1384: graph_mobilenet fails for NHWC on OpenCLGeorgios Pinitas
Makes GEMM3D account top padding when jumping among planes. Change-Id: Ia7c16cfa5498de106774ce42cbc4716e9f43195b Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/139612 Reviewed-by: Giorgio Arena <giorgio.arena@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com> Tested-by: Jenkins <bsgcomp@arm.com>
2018-11-02COMPMID-970 : Remove QS8 / QS16 supportVidhya Sudhan Loganathan
Removed fixed point related code. Change-Id: I487acf138dace3b0450e0d72ca7071eaec254566 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/137678 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-911: Allow GEMM to work with 3D tensorsIsabella Gottardi
Change-Id: I8c4823a0d909e19e9ef548f00b9ae98c66de61dd Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/123569 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-817: Tuner: Port kernels to new design.Georgios Pinitas
Change-Id: Iaabb1153c2abe0400ec79d51a21347debe92d642 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/134062 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-655 : Check FP16 is supported by the GPUVidhya Sudhan Loganathan
Change-Id: I507b04680a4e88426b682bd0be03bccb560ec78d Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/132589 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-1052 - Rework validate method in CLGEMMGian Marco Iodice
Change-Id: Iece5bd6478b5fac5164abff30c1e63e8a77291a9 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/130374 Reviewed-by: Anthony Barbier <anthony.barbier@arm.com> Tested-by: Jenkins <bsgcomp@arm.com>
2018-11-02COMPMID-922 - CLGEMM FP16 optimizations - part2Gian Marco Iodice
This patch improves of ~30 % GEMM fp16 when the reshape is required The results have been reported at the following confluence page: https://confluence.arm.com/display/MLENG/GEMM+FP16+performance%3A+ACL+18.05 Change-Id: I8233095a7e9ab06f1f915782a25dd41653b49140 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/128254 Reviewed-by: Anthony Barbier <anthony.barbier@arm.com> Tested-by: Jenkins <bsgcomp@arm.com>
2018-11-02COMPMID-1117: TransposeAccessWindow leads to high paddingGeorgios Pinitas
Switches CLGEMMMatrixMultiplyKernel and CLGEMMTranspose1xWKernel to use AccessWindowStatic Change-Id: I21533d4218215d5b8f84b23c603062678eccb1ed Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/130244 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Gian Marco Iodice <gianmarco.iodice@arm.com>
2018-11-02COMPMID-1026 - Add support for 4x4 output tile in CLWinogradConvolutionLayerGian Marco Iodice
The performance achieved can be found at the following confluence page: https://confluence.arm.com/display/MLENG/GEMM-based+convolution+vs+Winograd-based+convolution+on+OpenCL Change-Id: I4b690cfdd4eb4ff0cd17b14fdd49ccaa1d1dc85c Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/127729 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com>
2018-11-02COMPMID-922 - CLGEMM FP16 optimizations - part1Gian Marco Iodice
This patch improves of ~20% GEMM fp16. The results has been reported at the following confluence page: https://confluence.arm.com/display/MLENG/GEMM+FP32+performance%3A+ACL+18.05 I am aware with few cases we have a bit of degradation. However this cases are memory bound anyway (Fully connected layer cases) Change-Id: I183cbb7fba55a0b5eb86532c4dca5efe096096b0 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/128044 Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-1031: Use LWS hints for G51, G51BIG, G51LIT, and TNOXSam Laynton
Change-Id: Ie07d9225faaef778bdcfdcb56ae42ec95962e48d Signed-off-by: Sam Laynton <sam.laynton@arm.com> Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/126735 Reviewed-by: Michalis Spyrou <michalis.spyrou@arm.com> Tested-by: Jenkins <bsgcomp@arm.com>
2018-11-02COMPMID-1032 - Fixing bug in CLGEMM when is_interleaved_transposed=trueGian Marco Iodice
The bug concerned the collapse of the window in CLGEMMMatrixMultiplyKernel Change-Id: I5043bf37b72eeb615ebe7fb3f2c8e72d006bf341 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/126262 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-935 - Implementing Convolution with Winograd on OpenCL (part 4)Gian Marco Iodice
Implemented Winograd Output Transform (2x2,3x3) on OpenCL Implemented CLWinogradConvolutionLayer on OpenCL Change-Id: I6a113fc5f052ca07f878d2b800d2ab003f84af65 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/125148 Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Tested-by: Jenkins <bsgcomp@arm.com>
2018-11-02COMPMID-886 Don't use LWS hints by default for GPU post Mali-G72Michalis Spyrou
Change-Id: I64cb2d7f9513d69aebd9307a803b1b2c9c0e04c3 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/121929 Reviewed-by: Anthony Barbier <anthony.barbier@arm.com> Tested-by: Jenkins <bsgcomp@arm.com>
2018-11-02COMPMID-935 - Implementing Convolution with Winograd on OpenCL (Part 1)Gian Marco
This patch enables GEMM to execute multiple batches in parallel https://confluence.arm.com/display/MLENG/Winograd%3A+batched+GEMM Change-Id: I66222db041dd35e82af11fbb262fd1ebd3ca4b2f Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/120866 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-979: Add NHWC data layout to the tensor's metadata (Part 2)Isabella Gottardi
Change-Id: I24aa35a85834abf0c9954aba714aeae654615b44 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/122646 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-754: Add validation to kernels.Georgios Pinitas
Adds validation method to: - CLConvolutionLayer Change-Id: I95516e20cfb71c1e603c60fc6491ac695883a856 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/117355 Reviewed-by: Anthony Barbier <anthony.barbier@arm.com> Tested-by: Jenkins <bsgcomp@arm.com>
2018-11-02COMPMID-748 - Integrating optimized SGEMM for bifrostGian Marco
This patch introduces a new GEMM capable to improve the mac utilisation of 10% compared to the GEMM without reshape. However this implementation is not faster in all cases as we need to take into account the time for reshaping the matrices. For this reason an heuristic solution to select the optimal GEMM to use has been added to the function. More information about the heuristic implementation can be found at COMPMID-852. With this new patch, GoogleNet, MobileNet, VGG16 and SqueezeNet can improved the performance of 1.5x. More information about the performance uplift can be found here: https://confluence.arm.com/display/MLENG/GEMM+FP32+performance%3A+ACL+18.02 Change-Id: I024563c06b9aed02a211a974e452bae5c233b04c Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/117140 Reviewed-by: Pablo Tello <pablo.tello@arm.com> Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-759 - CLGEMM optimization for McVail benchmarksGian Marco
This patch introduces an optimization for CLGEMM on Bifrost architectures which can bring to 40% of FMA utilization on config 3 of McVail. The new CLGEMM does not require any reshape of matrix A and matrix B. This patch also adds the auto-config in CLConvolutionLayer and CLGEMM and extends the interface for NEGEMM and CLGEMM. Change-Id: Ibb354eda45e9ca64b14a99700fb21dff5989dda9 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/113716 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Michalis Spyrou <michalis.spyrou@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-617: Adds CLFullyConnectionLayer validation supportGeorgios Pinitas
Change-Id: I4d2eb9872a3165fdcaa7784596e441cbe563dbc2 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/112577 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Ioan-Cristian Szabo <ioan-cristian.szabo@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-661: Vectorize im2col and add lws heuristics for convolution kernels #46Anthony Barbier
Change-Id: Idaab987384d6a12a114f609abd50446fd94536b2 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/110879 Tested-by: BSG Visual Compute Jenkins server to access repositories on http://mpd-gerrit.cambridge.arm.com <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-661: Optimize FC layer with 2 new Bifrost kernels and LWS tuning (#33)Anton Lokhmotov
Change-Id: Ie56ac88dff5ff339572cec562e8cd62dc7f0aa8b Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/109805 Tested-by: BSG Visual Compute Jenkins server to access repositories on http://mpd-gerrit.cambridge.arm.com <bsgcomp@arm.com> Reviewed-by: Gian Marco Iodice <gianmarco.iodice@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-524 - Implemented CLTuner objectGian Marco
Change-Id: Idbdbecca1fc299ed042936119d90e2bed8db0938 Reviewed-on: http://mpd-gerrit.cambridge.arm.com/87101 Reviewed-by: Anthony Barbier <anthony.barbier@arm.com> Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com>
2018-11-02COMPMID-477 - Optimized batched case in CLConvolutionLayerGian Marco Iodice
Change-Id: I4ef18f49f1da0cb816aaa0762466b940792c15ed Reviewed-on: http://mpd-gerrit.cambridge.arm.com/84162 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-417: Cleanup CL FullyConnectedLayerMoritz Pflanzer
Change-Id: Ic7191be1f136c6aad4037cf2ec4bc6d7d0e440d3 Reviewed-on: http://mpd-gerrit.cambridge.arm.com/83713 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-09-17COMPMID-417: DepthConvert NEON for QS8/QS16.Georgios Pinitas
Change-Id: Ieb120bccf146045b3a0001ceb3893d4e67fd19df Reviewed-on: http://mpd-gerrit.cambridge.arm.com/79763 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Steven Niu <steven.niu@arm.com>
2018-09-17COMPMID-434 - Port CLGEMM to support 16 bit fixed pointGian Marco Iodice
Change-Id: I30aef3c7ecd1ee740c2a7f2ce65a63c7dcd66e49 Reviewed-on: http://mpd-gerrit.cambridge.arm.com/79630 Reviewed-by: Anthony Barbier <anthony.barbier@arm.com> Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com>
2018-09-17COMPMID-411 - Port CLGEMM to support 8 bit fixed pointGian Marco Iodice
Change-Id: I6c8bd69ae9715e4d83d128b2162fc15aa5561afb Reviewed-on: http://mpd-gerrit.cambridge.arm.com/78804 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Reviewed-by: Moritz Pflanzer <moritz.pflanzer@arm.com>
2018-09-17COMPMID-344 Updated doxygenAnthony Barbier
Change-Id: I32f7b84daa560e460b77216add529c8fa8b327ae
generated by cgit v1.2.1 (git 2.23.0) at 2024-05-23 09:09:30 +0000