From 716b1be5f92f5d025ed0ee15775b903728a8e73b Mon Sep 17 00:00:00 2001 From: Gian Marco Iodice Date: Wed, 10 Feb 2021 17:33:27 +0000 Subject: Update OpenCL GEMM benchmark script - Update cl_gemm_benchmark.sh script. Now it also possible to split the benchmark among different platforms - Update documentation Resolves COMPMID-3848 Change-Id: Ifbf71cee5ff83aead76e20dcbf7dc5ce41526594 Signed-off-by: Gian Marco Iodice Reviewed-on: https://review.mlplatform.org/c/ml/ComputeLibrary/+/5047 Tested-by: Arm Jenkins Reviewed-by: Giorgio Arena --- examples/gemm_tuner/README.md | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) (limited to 'examples/gemm_tuner/README.md') diff --git a/examples/gemm_tuner/README.md b/examples/gemm_tuner/README.md index 73bddc9239..aae803eabb 100644 --- a/examples/gemm_tuner/README.md +++ b/examples/gemm_tuner/README.md @@ -9,7 +9,7 @@ The details of these strategies can be found in the documentations of the corres **CLGEMMMatrixMultiplyReshapedOnlyRHSKernel**. The Tuner consists of 2 scripts and 3 binaries: -* benchmark_gemm_examples.sh and GemmTuner.py under examples/gemm_tuner, and +* cl_gemm_benchmark and GemmTuner.py under examples/gemm_tuner, and * benchmark_cl_gemm_native, benchmark_cl_gemm_reshaped_rhs_only and benchmark_cl_gemm_reshaped under build/tests/gemm_tuner (you'll need to build the library first) @@ -42,7 +42,7 @@ what kernel and subsequently what configurations for that kernels are the most p ### Step2: Push relevant files to the target device All the files that need to be present on the target device are: -* benchmark script: \/examples/gemm_tuner/benchmark_gemm_examples.sh +* benchmark script: \/examples/gemm_tuner/cl_gemm_benchmark * shapes and configs csv files: gemm_shapes.csv, gemm_configs_native.csv, gemm_configs_reshaped_only_rhs.csv, gemm_configs_reshaped.csv * Example benchmark binaries: \/build/tests/gemm_tuner/benchmark_cl_gemm* @@ -51,15 +51,25 @@ With these files on device, we can collect benchmark data using the script. Assu to a folder called *gemm_tuner*. While logged onto our device: ``` # Native -./benchmark_gemm_examples.sh -s native -e ./gemm_tuner -g ./gemm_shapes.csv -c ./gemm_configs_native.csv -o results/native +./cl_gemm_benchmark -s native -e ./gemm_tuner -g ./gemm_shapes.csv -c ./gemm_configs_native.csv -o results/native # Reshaped Only RHS -./benchmark_gemm_examples.sh -s reshaped_rhs_only -e ./gemm_tuner -g ./gemm_shapes.csv -c ./gemm_configs_reshaped_only_rhs.csv -o results/reshaped_only_rhs +./cl_gemm_benchmark -s reshaped_rhs_only -e ./gemm_tuner -g ./gemm_shapes.csv -c ./gemm_configs_reshaped_only_rhs.csv -o results/reshaped_only_rhs # Reshaped -./benchmark_gemm_examples.sh -s reshaped -e ./gemm_tuner -g ./gemm_shapes.csv -c ./gemm_configs_reshaped.csv -o results/reshaped +./cl_gemm_benchmark -s reshaped -e ./gemm_tuner -g ./gemm_shapes.csv -c ./gemm_configs_reshaped.csv -o results/reshaped ``` You can repeat the 3 commands above to have a bit redundancy in your benchmark data (as you can imagine, measurement is noisy), but you may need to change the output folder for each repeat +It is also possible to split the benchmark phase among different platforms using the **-i** and **-n** options to specificy the starting experiment and the number of benchmark to run. + +# Reshaped benchmark on 3 different platforms +## Platform 1 +./cl_gemm_benchmark -s reshaped -e ./gemm_tuner -g ./gemm_shapes.csv -c ./gemm_configs_reshaped.csv -o results/reshaped -i 0 -n 8 +## Platform 2 +./cl_gemm_benchmark -s reshaped -e ./gemm_tuner -g ./gemm_shapes.csv -c ./gemm_configs_reshaped.csv -o results/reshaped -i 8 -n 8 +## Platform 3 +./cl_gemm_benchmark -s reshaped -e ./gemm_tuner -g ./gemm_shapes.csv -c ./gemm_configs_reshaped.csv -o results/reshaped -i 16 -n 8 + ### Step4: Generate the heuristics 1. After benchmarking, we pull the benchmark data, the *results* folder, from the target device to our host machine 2. We use the GemmTuner.py script to give us the heuristics @@ -77,7 +87,7 @@ passing a lower value to *-t \* to the GemmTuner.py script. * Android or Linux OS * Bash shell * Built Compute Library with benchmark examples binaries - * benchmark_gemm_examples.sh script + * cl_gemm_benchmark script * gemm shape file A csv file containing the **GEMMParam search list**. This is the list of GEMMParams/gemm shapes that we're @@ -202,13 +212,13 @@ passing a lower value to *-t \* to the GemmTuner.py script. ## Usage The usage of the 2 scripts: -1. benchmark_gemm_examples.sh +1. cl_gemm_benchmark - Run the shell script (**benchmark_gemm_examples.sh**) on your **target device**. Note that all the built benchmark + Run the shell script (**cl_gemm_benchmark**) on your **target device**. Note that all the built benchmark examples: build/tests/gemm_tuner/benchmark_cl_gemm*, have to be present on your target device prior to running. The benchmark results will be saved to json files in an output directory. ``` - Usage: benchmark_gemm_examples.sh [-h] -s \ -e \ -g \ + Usage: cl_gemm_benchmark [-h] -s \ -e \ -g \ -c \ [-d \] [-o \] Options: @@ -265,4 +275,4 @@ The usage of the 2 scripts: milliseconds. Recommended value: <= 0.1 ms -D, --debug Enable script debugging output - ``` \ No newline at end of file + ``` -- cgit v1.2.1