aboutsummaryrefslogtreecommitdiff
path: root/tests/framework/Framework.cpp
AgeCommit message (Collapse)Author
2023-05-18Printing out the rerun command of each failed testcaseRamy Elgammal
- After a testcase fail, print the arm_compute_validation command with filter specified and seed to rerun. Resolves: COMPMID-5916 Signed-off-by: Ramy Elgammal <ramy.elgammal@arm.com> Change-Id: I0c8214e7d3b61f01ce16f50499c29bfe08b6a885 Reviewed-on: https://review.mlplatform.org/c/ml/ComputeLibrary/+/9673 Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Pablo Marquez Tello <pablo.tello@arm.com> Reviewed-by: Viet-Hoa Do <viet-hoa.do@arm.com> Comments-Addressed: Arm Jenkins <bsgcomp@arm.com> Benchmark: Arm Jenkins <bsgcomp@arm.com>
2023-05-10Re-enable dyanmic weights in Neon™ depthwise convolutionRamy Elgammal
- Call Neon™ depthwise convolution validation inside in its configure() method. Resolves: COMPMID-6188 Signed-off-by: Ramy Elgammal <ramy.elgammal@arm.com> Change-Id: Ib2ae4d995ff2bbc92ce4496d4ab93cf09113e3e9 Reviewed-on: https://review.mlplatform.org/c/ml/ComputeLibrary/+/9594 Comments-Addressed: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Viet-Hoa Do <viet-hoa.do@arm.com> Reviewed-by: Jakub Sujak <jakub.sujak@arm.com> Tested-by: Arm Jenkins <bsgcomp@arm.com> Benchmark: Arm Jenkins <bsgcomp@arm.com>
2021-12-24Replacing non-inclusive terms with proper termsramelg01
Partially-Resolves: COMPMID-4854 Signed-off-by: Ramy Elgammal <ramy.elgammal@arm.com> Change-Id: Ic9757c89878b9b5a89680b5344de657f676c7bf2 Reviewed-on: https://review.mlplatform.org/c/ml/ComputeLibrary/+/6859 Comments-Addressed: Arm Jenkins <bsgcomp@arm.com> Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Gunes Bayir <gunes.bayir@arm.com>
2021-07-19Add layer data to JSON outputFreddie Liardet
Add layer data information to SchedulerTimer JSON output. Resolves: COMPMID-4423 Change-Id: Ife78dee8afc0910cf47b135bc6809cc170ec4ed3 Signed-off-by: Freddie Liardet <frederick.liardet@arm.com> Reviewed-on: https://review.mlplatform.org/c/ml/ComputeLibrary/+/5923 Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Comments-Addressed: Georgios Pinitas <georgios.pinitas@arm.com> Comments-Addressed: Arm Jenkins <bsgcomp@arm.com> Tested-by: Arm Jenkins <bsgcomp@arm.com>
2021-03-02Set up configure-only flag for validation. First trial with DepthwiseConvoltionGiorgio Arena
This is needed in order to validate OpenCL kernel run-time compilation, without necessarily running or validating the kernels' execution - Add a run-time option for our validation suite to only configure one target function, without allocating, running or validating - Avoid to map/unmap tensors in CLAccessor if no allocation/validation is required - Create a new Fixture macro that accepts fixtures split into configure/allocate_and_run/reference, and do the last two only if required - Adjust fixture and validation files for the first trial function(s) (DepthwiseConvolutionLayer) Signed-off-by: Giorgio Arena <giorgio.arena@arm.com> Change-Id: I56fa1ce5ef4ac0c86bcabda686cc277ef5ec69c8 Reviewed-on: https://review.mlplatform.org/c/ml/ComputeLibrary/+/5048 Comments-Addressed: Arm Jenkins <bsgcomp@arm.com> Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Manuel Bottini <manuel.bottini@arm.com> Reviewed-by: Sang-Hoon Park <sang-hoon.park@arm.com>
2020-12-01Update default C++ standard to C++14Georgios Pinitas
(3RDPARTY_UPDATE) Resolves: COMPMID-3849 Signed-off-by: Georgios Pinitas <georgios.pinitas@arm.com> Change-Id: I6369f112337310140e2d6c8e79630cd11138dfa0 Reviewed-on: https://review.mlplatform.org/c/ml/ComputeLibrary/+/4544 Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Michalis Spyrou <michalis.spyrou@arm.com> Comments-Addressed: Arm Jenkins <bsgcomp@arm.com>
2020-07-09COMPMID-3324: Adjusting capitalization of Arm copyright claim to reflect Arm ↵Michele Di Giorgio
preferred presentation Change-Id: Ib7dcfcbb24b408999dfae366b9da396485aacf78 Signed-off-by: Michele Di Giorgio <michele.digiorgio@arm.com> Reviewed-on: https://review.mlplatform.org/c/ml/ComputeLibrary/+/3525 Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Tested-by: Georgios Pinitas <georgios.pinitas@arm.com> Comments-Addressed: Arm Jenkins <bsgcomp@arm.com>
2020-03-12COMPMID-3069: Improve compilation time by removing regex from test framework ↵Matthew Bentham
headers Regex is used as an implementation detail by TestFilter and libnpy, is an expensive header to parse, and also instantiates static objects. Move TestFilter out of Framework.h by using a partial definition and a unique_ptr instead of storing the TestFilter by value. Move npy.h out of AssetsLibrary.h by moving part of a template definition into AssetsLibrary.cpp Knocks about 15% off compilation time of small test cases (for me, knocked .7s off 5s compilation of HogDetector.cpp) Signed-off-by: Matthew Bentham <matthew.bentham@arm.com> Change-Id: I1dce18855d0752ec25b2165fddbc6861a4c55a76 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/c/VisualCompute/ComputeLibrary/+/229181 Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Tested-by: bsgcomp <bsgcomp@arm.com> Reviewed-on: https://review.mlplatform.org/c/ml/ComputeLibrary/+/2856 Tested-by: Arm Jenkins <bsgcomp@arm.com> Comments-Addressed: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Michele Di Giorgio <michele.digiorgio@arm.com>
2020-03-11COMPMID-3069: Remove unnecessary uses of ToolchainSupport.hMatthew Bentham
Change-Id: I9c09d1002043fd2f927493a85924298d54b1ad9c Signed-off-by: Georgios Pinitas <georgios.pinitas@arm.com> Reviewed-on: https://review.mlplatform.org/c/ml/ComputeLibrary/+/2854 Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Michele Di Giorgio <michele.digiorgio@arm.com>
2020-01-10COMPMID-2986: Extend test framework options.Georgios Pinitas
Add the following option to the test framework: - delay: Allows run delay between tests in seconds. Signed-off-by: Georgios Pinitas <georgios.pinitas@arm.com> Change-Id: Ib1b7a1d1ad83c9a99b23f353b1800db075b32bce Reviewed-on: https://review.mlplatform.org/c/2525 Reviewed-by: Michele Di Giorgio <michele.digiorgio@arm.com> Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Gian Marco Iodice <gianmarco.iodice@arm.com> Comments-Addressed: Arm Jenkins <bsgcomp@arm.com>
2019-11-20COMPMID-2929: Resolve segfault at end of benchmark suite execution.Georgios Pinitas
Disable usage of RuntimeContext for CL in the test and benchmark suite as ordering of construction/destruction has been altered causing issues. Change-Id: I86702350fdabeb6d3a9547c405ee82358255d574 Signed-off-by: Georgios Pinitas <georgios.pinitas@arm.com> Reviewed-on: https://review.mlplatform.org/c/2314 Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Giuseppe Rossini <giuseppe.rossini@arm.com> Comments-Addressed: Arm Jenkins <bsgcomp@arm.com>
2019-11-01COMPMID-2549: Add GLES Runtime Context interfaces.Georgios Pinitas
* Creates interfaces and concrete classes * Ports GCActivationalLayer * Adapts test framework and relevant tests Change-Id: Ide36cd65ebf185958db3c4a5bebd630fcb2f39b3 Signed-off-by: Georgios Pinitas <georgios.pinitas@arm.com> Reviewed-on: https://review.mlplatform.org/c/2199 Reviewed-by: Pablo Marquez <pablo.tello@arm.com> Tested-by: Arm Jenkins <bsgcomp@arm.com> Comments-Addressed: Arm Jenkins <bsgcomp@arm.com>
2019-10-15COMPMID-2205: CL runtime context.Pablo Tello
CL Interfaces implemented. Concrete classes implemented. One test (ActivationLayer) ported to the new interface. Change-Id: I283808bec36ccfc2f13fe048c45cbbee698ce525 Signed-off-by: Pablo Tello <pablo.tello@arm.com> Reviewed-on: https://review.mlplatform.org/c/1998 Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Comments-Addressed: Arm Jenkins <bsgcomp@arm.com>
2019-09-09COMPMID-2204: RuntimeContext interface for NEON functions.Georgios Pinitas
This patch creates the interfaces for the runtime context for NEON. Only the Neon backend implements the context which currently only holds an instance of the scheduler. The NEActivationLayer function has been updated to use the new context interface and the corresponding validation tests ported. Change-Id: I32e7e6aa888796dcbbfc5039b1e7f784a24f47da Signed-off-by: Pablo Tello <pablo.tello@arm.com> Reviewed-on: https://review.mlplatform.org/c/1851 Comments-Addressed: Arm Jenkins <bsgcomp@arm.com> Tested-by: Arm Jenkins <bsgcomp@arm.com> Reviewed-by: Michele Di Giorgio <michele.digiorgio@arm.com>
2018-11-08COMPMID-1777: Add option to make instruments output timestamps instead of ↵Anthony Barbier
duration Change-Id: Iafc1d6cd8003de64a3439ad807f4002036c73a73
2018-11-02COMPMID-1695: (Nightly) bad_alloc in CLIm2ColGeorgios Pinitas
Clear CLContext in a more regular basis to make the driver release memory back to the system. Change-Id: I0df847766f57719433bbaeada45fe630e38c9541 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/155435 Reviewed-by: Giuseppe Rossini <giuseppe.rossini@arm.com> Tested-by: bsgcomp <bsgcomp@arm.com>
2018-11-02COMPMID-1631: Fixed test framework memory corruption.Pablo Tello
Change-Id: I2c4dcedcd3b56e41174eebbbacd47be4e968d34d Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/152767 Tested-by: bsgcomp <bsgcomp@arm.com> Reviewed-by: Vidhya Sudhan Loganathan <vidhyasudhan.loganathan@arm.com> Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com>
2018-11-02COMPMID-1622 : (Nightly) Seg fault on androidVidhya Sudhan Loganathan
call to print_cpu_info moved to main Change-Id: I6d82649964542df4e944bc79e4c16f0813976295 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/152695 Tested-by: bsgcomp <bsgcomp@arm.com> Reviewed-by: Pablo Tello <pablo.tello@arm.com>
2018-11-02COMPMID-1622 : (Nightly) Seg fault on androidVidhya Sudhan Loganathan
conditionally compile the std::cout that was causing the fault Change-Id: I7f50151ab88f19ed6eec1be11ca975614653e359 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/151762 Reviewed-by: Pablo Tello <pablo.tello@arm.com> Tested-by: bsgcomp <bsgcomp@arm.com>
2018-11-02COMPMID-162: output cpus list and capabilities in validation suite.Pablo Tello
Change-Id: I6f71f2da851454e8fbbdfc9223592dea9ad03bac Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/151014 Tested-by: bsgcomp <bsgcomp@arm.com> Reviewed-by: Giuseppe Rossini <giuseppe.rossini@arm.com> Reviewed-by: Isabella Gottardi <isabella.gottardi@arm.com>
2018-11-02COMPMID-1246: Check OpenCL is available before re-creating the CL Context ↵Anthony Barbier
and Queue Change-Id: I0b0be0a86d43d87ea8225f6d5e5b02b0f80e6e3a Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/145034 Reviewed-by: Michele DiGiorgio <michele.digiorgio@arm.com> Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Tested-by: Jenkins <bsgcomp@arm.com>
2018-11-02COMPMID-1478: Stop relying on static default OpenCL objects in cl2.hppAnthony Barbier
This causes problems when ACL is used as a shared library on Android. Fixes some problems related to creation / destruction order between the Graph's CL backend and core / runtime Change-Id: I716d63fd42f4586df1ffbb6fa97e4db06d3a781b Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/143228 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Michele DiGiorgio <michele.digiorgio@arm.com> Reviewed-by: Gian Marco Iodice <gianmarco.iodice@arm.com>
2018-11-02COMPMID-1182: printf doesn't workGeorgios Pinitas
Change-Id: I013d57f6e2becbd6d2d7700ce5fbbeca670443c4 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/133735 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Pablo Tello <pablo.tello@arm.com>
2018-11-02COMPMID-1119 - Bugfix SIGKILLGian Marco Iodice
The problem seems caused by the OpenCL driver that does not release the allocated memory. In order to solve this problem the OpenCL context is destroyed every 5000 tests to force the release of the memory Change-Id: I2135f49d7ff92c7761ec8dba6819db1590e19691 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/131459 Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-1077 Add OpenCL Memory Usage instrumentAnthony Barbier
Change-Id: Ic5a24f22bec9d6ca71486097f6ad70fb0d83da2b Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/128674 Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Tested-by: Jenkins <bsgcomp@arm.com>
2018-11-02COMPMID-1081: Introduced test-wide instrumentsAnthony Barbier
Change-Id: I5831241f3fc503717cc51136453c2bf96d4b420b Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/128484 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com>
2018-11-02COMPMID-798 Add instrumentation to NEON kernelsAnthony Barbier
Change-Id: I9dbb090cac731d68bd98a7d1c8ab0e1cb0a5c911 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/116746 Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Reviewed-by: Pablo Tello <pablo.tello@arm.com> Tested-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02APPBROWSER-359: Sync tensor in GC benchmark testsJoel Liang
Change-Id: I22c1aa92e70d6143bbcec90e9e7de9f1ce1c1e55 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/114635 Tested-by: Jenkins <bsgcomp@arm.com> Reviewed-by: Pablo Tello <pablo.tello@arm.com>
2018-11-02COMPMID-694: merge opencl_timer flag with the opencl flagAnthony Barbier
Is there any reason why we might have OpenCL enabled but wouldn't want to make the opencl_timer instrument available ? Change-Id: I946d643028fc394a607478415e4f84c4e8fa5bc4 Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/110466 Tested-by: BSG Visual Compute Jenkins server to access repositories on http://mpd-gerrit.cambridge.arm.com <bsgcomp@arm.com> Reviewed-by: Kevin Petit <kevin.petit@arm.com> Reviewed-by: Ioan-Cristian Szabo <ioan-cristian.szabo@arm.com> Reviewed-by: Michalis Spyrou <michalis.spyrou@arm.com> Reviewed-by: Gian Marco Iodice <gianmarco.iodice@arm.com> Reviewed-by: Giorgio Arena <giorgio.arena@arm.com>
2018-11-02COMPMID-694: Fix condition which decides whether or not to profile a runAnthony Barbier
Change-Id: I4853a4d83a13b162d5759434a4e800c8f9e05cce Reviewed-on: https://eu-gerrit-1.euhpc.arm.com/110108 Tested-by: BSG Visual Compute Jenkins server to access repositories on http://mpd-gerrit.cambridge.arm.com <bsgcomp@arm.com> Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com>
2018-11-02COMPMID-622 Let the user choose the units for the instrumentsGiorgio Arena
Change-Id: Ic6ac4cd6df6970593a5e2e6310b6d61951c88898 Reviewed-on: http://mpd-gerrit.cambridge.arm.com/93887 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-556: Don't measure performance for the first runAnthony Barbier
This messes up badly the opencl_timer if we measure the first run and are using a CLTuner at the same time. Basically all the tuner combinations appear in the JSON file and this makes it really hard to post process Change-Id: I59377eb752ca8ddc95b884ab04b1ef3f65389931 Reviewed-on: http://mpd-gerrit.cambridge.arm.com/95860 Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com>
2018-11-02COMPMID-631: Merge branches/gles_compute branchAnthony Barbier
Last commit: commit b25c5f68042b0c81bf611d59a1bb8535e1c42497 Author: Xinghang Zhou <xinghang.zhou@arm.com> Date: Wed Oct 25 18:48:10 2017 +0800 Synced validation's tolerances of GCSoftmax from cl side Change-Id: Ibe72054205c1c8721845d679a31af7ed0a7c5cf6 Reviewed-on: http://mpd-gerrit.cambridge.arm.com/93283 Reviewed-by: Anthony Barbier <anthony.barbier@arm.com> Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com>
2018-11-02COMPMID-663 Adding instrument to return timestamps of clGetEventProfilingInfoAnthony Barbier
Change-Id: I1037054615593205f07e25fb9b16fecd13407c2c Reviewed-on: http://mpd-gerrit.cambridge.arm.com/95142 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com>
2018-11-02COMPMID-621 Support several printers for testsGiorgio Arena
Change-Id: I2e8e98aab1ec14c2a1bf17db2facc908b241b783 Reviewed-on: http://mpd-gerrit.cambridge.arm.com/93308 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-417 Allow the tests to run even when assets are not presentAnthony Barbier
Change-Id: Ief165b1d583a70cbe35aae93f05ddfe962196323 Reviewed-on: http://mpd-gerrit.cambridge.arm.com/89503 Reviewed-by: Georgios Pinitas <georgios.pinitas@arm.com> Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com>
2018-11-02COMPMID-417: Fix error loggingMoritz Pflanzer
Previously every failed expectation was shown as expected error evene if the test wasn't marked as expected failure. Change-Id: Ie72a69ae17e6f9e625560f9b011f5c0457c0a0d7 Reviewed-on: http://mpd-gerrit.cambridge.arm.com/87948 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Pablo Tello <pablo.tello@arm.com>
2018-11-02COMPMID-482: Add mali countersMoritz Pflanzer
Change-Id: I1782c3d92f7fea5a73ed89868d8c3ce04ffcf518 Reviewed-on: http://mpd-gerrit.cambridge.arm.com/85020 Reviewed-by: Anthony Barbier <anthony.barbier@arm.com> Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com>
2018-11-02COMPMID-417: Fix validationMoritz Pflanzer
Change-Id: I7a745037136bc6e02d177f65fe4f4cd43873b98e Reviewed-on: http://mpd-gerrit.cambridge.arm.com/87406 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-516 Change the CL CNN validation functions to use relativesteniu01
tolerance error Change-Id: Iec6347af26ea2a83c911f5fe10e6048e8a2a47ba Reviewed-on: http://mpd-gerrit.cambridge.arm.com/85381 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Moritz Pflanzer <moritz.pflanzer@arm.com>
2018-11-02COMPMID-482: Refactor PMU countersMoritz Pflanzer
Change-Id: I9b254ce693363ecbbd7c188d211c85471134a91e Reviewed-on: http://mpd-gerrit.cambridge.arm.com/84328 Reviewed-by: Anthony Barbier <anthony.barbier@arm.com> Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com>
2018-11-02COMPMID-345 Fix the failure counter wrongly logging issuesteniu01
Change-Id: Ic917b0361e602fadb8dbff69c6bec5582d6b261d Reviewed-on: http://mpd-gerrit.cambridge.arm.com/85956 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
2018-11-02COMPMID-415: Rename and move testsMoritz Pflanzer
The boost validation is now "standalone" in validation_old and builds as arm_compute_validation_old. The new validation builds now as arm_compute_validation. Change-Id: Ib93ba848a25680ac60afb92b461d574a0757150d Reviewed-on: http://mpd-gerrit.cambridge.arm.com/86187 Tested-by: Kaizen <jeremy.johnson+kaizengerrit@arm.com> Reviewed-by: Anthony Barbier <anthony.barbier@arm.com>
generated by cgit v1.2.1 (git 2.23.0) at 2024-05-19 22:44:44 +0000