Age | Commit message (Collapse) | Author |
|
In Execute Network the Tflite runtime execution does not support an
asynchronous mode. Inform the user if they have specified an
asynchronous parameter and TfLite executor.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: I9c5ecb91954c2293a013d110555ecff4aabbfc23
|
|
* Detect kTfLiteApplicationError from the TfLite runtime and allow
inference to continue with a BIG warning.
* Fix handling of output tensors in the TfLiteExecutor.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: If99dab7f0ac068fe4d17f338306c7bc5b128250a
|
|
with "--tflite-executor parser" option
* Allow serialize to dot for the parser delegate
Signed-off-by: Tracy Narine <tracy.narine@arm.com>
Change-Id: Id9edef143f0e0cc02dd7b1bfe87d36035bf44c6d
|
|
* All of the unit tests were including the TfL schema even though it
is not used. This prevented building with delegate and not TfL parser.
* Fixing some template compile errors that were introduced when
FileComparisonExecutorTests.cpp was compiled without TfL parser.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: I86742d57a6520ab6bd8336a683fee29de326fff5
|
|
* Adding clock times before and after inferences which SA team requested
Signed-off-by: Kevin May <kevin.may@arm.com>
Change-Id: Idc37085a25ce573cb74340f1be7313a616f561e8
|
|
This script will attempt to execute a tflite model through all available
backends reporting performance and accuracy checks.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: Id60626eddc48b48c497a7f52f3fbd10aa036d997
|
|
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com>
Change-Id: I0a111fa8ef62632c3c975537a727d75531a44b7a
|
|
* Add Signed 64 bit support for Output and Debug Layers
Signed-off-by: John Mcloughlin <john.mcloughlin@arm.com>
Change-Id: I991c2d5f1067b16d0fac362e7406305fbe90d034
|
|
* Moved deprecated functionality to its own heading
* Updated documentation for many of the parameters
* Added information on execution time versus inference time
Signed-off-by: Tracy Narine <tracy.narine@arm.com>
Change-Id: If017e49e7f6c60a72fe948c1cfa1a6f6a5c4a692
|
|
* Check if preferred backends have FP16 support before enable fp16-turbo-mode
* Unit tests
* Replaced global gpuAccCapabilities with getter method construction
* Replaced deprecated function call in SL shim
Signed-off-by: Narumol Prangnawarat <narumol.prangnawarat@arm.com>
Signed-off-by: Ryan OShea <ryan.oshea3@arm.com>
Change-Id: If29b62b330ca8987de8acf6408db11daf25ca0b5
|
|
* Added optimization to remove reshapes for Neon and Ref Backends
by using overridden TensorInfos
* Added ability to delete Subgraphs during Optimization
* Fixed naming error in NeonEndToEndTests and CLEndToEndTests
* Added LayerNameAndTypeCheck for testing.
* Fixed error where layers were not marked as altered when removed in
CLBackend
Signed-off-by: Mike Kelly <mike.kelly@arm.com>
Change-Id: I1ac25cd4ec9821470d961831ae2c8d24882276cc
|
|
Even though we're using C++ 17 we still do a Debian buster build. This
prevents using some new libraries. In this case std::filesystem.
Change-Id: Icbdcac8a53d5931f25f8aa77c7422aa0b8ae94d4
Signed-off-by: Francis Murtagh <francis.murtagh@arm.com>
|
|
Even though we're using C++ 17 we still do a Debian buster build. This
prevents using some new libraries. In this case std::filesystem.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: Icdf3460cb5dd0e3ed2f07fccc5b89bd8975d488d
|
|
working with `-T delegate` flag
* Removed original SetInternalProfilingParams
* Added Undefined Profiling DetailsMethod for delegateOptions
* Replaced with if / else block to handle DetailsOnly and DetailsWithEvents
Signed-off-by: Declan-ARM <decmce01@arm.com>
Change-Id: Id0c91892732c5c6b59696a43c6a73a22cbfffa09
|
|
* Implement the "-C" command line option of executenetwork.
* Add a FileComparisonExecutorFile which will read tensors from
a previously written text file and compare them to the execution
output.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: I8380fd263028af13d65a67fb6afd89626d1b07b8
|
|
If you pass a file name without an extension as the -m parameter to
execute network it won't be able to determine the parser to use. This
results in a segmentation fault.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: I91821a5abd50b65b8d60409aa9a807013e0b0e6a
|
|
Signed-off-by: Narumol Prangnawarat <narumol.prangnawarat@arm.com>
Change-Id: Id404893c47d43dafa743f4b9524001072b426509
|
|
Signed-off-by: Narumol Prangnawarat <narumol.prangnawarat@arm.com>
Change-Id: I3f0e224c90a4eea9945183028c9de1b61e75e510
|
|
Signed-off-by: Narumol Prangnawarat <narumol.prangnawarat@arm.com>
Change-Id: Ibdded86713368ecfdf31c4118dfe8a3404d1e3b8
|
|
!android-nn-driver:9431
Signed-off-by: Jim Flynn <jim.flynn@arm.com>
Change-Id: I58143445b5c5cf2aafd0838156c9543adce21e6a
|
|
Signed-off-by: John Mcloughlin <john.mcloughlin@arm.com>
Change-Id: Id4bdc31e3e6f18ccaef232c29a2d2825c915b21c
|
|
The initial model load and tensor allocation operations against the
TfLiteInterpreter were not checking return codes resulting in
segmentation faults.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: I812785f0af9012c97570065d200f72eaf781165a
|
|
The -A -B -C options in execute network were attempting to calculate
the RMS error over output tensors. However, the calculation was mixing
tensor elements and bytes when doing the calculation. This patch
changes the calculation to use a per byte RMS error calculation.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: If30230a16cfed1a8804b4d54ed1abcd371f26664
|
|
ProfilingOptions is not used in DelegateOptions. Instead the parameters
are passed in through the RuntimeOptions. This is done in ExecuteNetwork
and TfliteExecutor.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: Iaab3d4ef277c47e1ff82a51ba2648f5f51ec3e2c
|
|
* When the output of a network is a boolean from a comparison layer
ExecuteNetwork was missing the data type when writing the output tensor
Signed-off-by: Ryan OShea <ryan.oshea3@arm.com>
Change-Id: Id9c1609462395a68e8c1842c77a4a033a10f74e8
|
|
* When the tfLiteExecutor attempts to populate the input tensors it did
not check whether the tensor was constant. This was causing
segmentation faults.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: I80a4cc788de4ffe08afb2df9185d04fcb8b27c3a
|
|
* Check if BuildExecutor returns null in ExecuteNetwork.
* Check if tflite BuildFromFile returns null in TfliteExecutor.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: I42b6e5f26dfd127dd16b6b322184900846317c41
|
|
one works fine
* All ArmNNExecutors now share a single IRuntime.
* All armnn_delegates now share a single IRuntime.
* Increased delegate major version.
Signed-off-by: Mike Kelly <mike.kelly@arm.com>
Change-Id: I95cbdc32655ec0beb476dbb2d60f1a0209df8f04
|
|
Signed-off-by: Cathal Corbett <cathal.corbett@arm.com>
Change-Id: Ic6bbbaa04aabaa5c3fd525acd5121f07d3392120
|
|
* Build ExecNet lib dependencies as object libs except libarmnn
* Disable PIPE when building static ExecNet
* Remove multiple definition from AsyncExecutionCallback
* Disable DynamicBackend for ExecNet Static build
* Disable inference tests for TfLiteParser and ONNX during static ExecNet
* Remove Tensorflow Parser if condition
* Add Disable thread macro to InferenceModel
* Don't compile dynamic backend symbols in Runtime.cpp for Baremetal and
Exenet Static
Signed-off-by: Ryan OShea <ryan.oshea3@arm.com>
Change-Id: If41c063eab5f05b3df0a6e064924a36a177f116a
|
|
Two problems here:
* First the Delegate was using the parameter options after the execution
of std::move on it.
* In ExecuteNetworkParams 3 GPU backend options were instead being set as
optimizer options.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: I61c7fad8a5819a0a4aec0243899019a342c5cc5f
|
|
Move call to 'SetupInputAndOutputs' to after LoadedNetwork is available.
Change-Id: I101e297d1d7b2517011d4ef3f1a4927566845474
Signed-off-by: Matthew Bentham <matthew.bentham@arm.com>
|
|
- Remove Bf16ToFp32 Conversion Layer
- Remove Fp32ToBf16 Conversion Layer
- Remove B16 Conversion tests
* Throw exception if m_ReduceFp32ToBf16 optimzer option is set to true
* Provide comments to enable fast math in order to use bf16
* Update docs to inform users to enable fast math for bf16
Execute Network Changes
* Require bf16_turbo_mode to also have fast_math_enabled set to true
- Remove setting m_ReduceFp32ToBf16 optimizer option
Signed-off-by: Ryan OShea <ryan.oshea3@arm.com>
Change-Id: Ibaa6da9d29c96a1ce32ff5196b0847fde9f04a1c
|
|
Signed-off-by: Kevin May <kevin.may@arm.com>
Change-Id: If837e4bec7940b53d18d0da32f3e736215dd2a03
|
|
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com>
Change-Id: I2fd0f6aac6ffff695b17df7455f252f6013c0d43
|
|
* The intention is to keep the flexibility given by the ExNet before the refactor.
* When iteration > inputFiles, we repeat the usage in order
* When iteration < inputFiles, we just discard extra files.
Signed-off-by: Adam Jalkemo <adam.jalkemo@arm.com>
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com>
Change-Id: I2fbe69f8affe0e3a5cc86fc1748164967f0c2d64
|
|
not match"
This reverts commit 6c95836e894f88c4bab6b22f974341f0dd2dddaa.
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com>
Change-Id: I8be2147feb557a0849de5785fb63b464abc7dbb9
|
|
Signed-off-by: Cathal Corbett <cathal.corbett@arm.com>
Change-Id: Ib30fc633a10b6ff8090b50314278fe5dc46fb250
|
|
* Add functionality to print output tensors to file in tempdir
* UnitTests
Signed-off-by: Keith Davis <keith.davis@arm.com>
Change-Id: Idfb4c186544187db1fecdfca11c662540f645439
|
|
Signed-off-by: Jim Flynn <jim.flynn@arm.com>
Change-Id: I3573078206272c3a72a2b3acf8781ab458ea6c90
|
|
* When a model with multiple outputs was used and output to file, e.g.
with "-w ./boxes,./classes,./scores,./detection", the results where
not saved in the correct files.
* Applies only to the ArmNNExecutor.
Change-Id: I2899322622a4c3fd1d0ddc75b100b81669417660
|
|
* I had issues when folder name contained "armnn" and
a .tflite model was used, as the wrong parser was selected.
* Now only the extension, and not the full string, is
considered when selecting parser.
Change-Id: If7964d2ce5535f7d25762d2a2d7e810bf1a1ed43
|
|
* Some CL kernels are not run after the first inference and this breaks
the profiler which is expecting a measurement for every kernel each run
* Add a function HasKernelMeasurements() to ascertain if the Event is
returning kernel measurements and if so insert 0.0 values for any missing
kernel measurements.
* Fix ExecuteNetwork to only print a json object after all inferences
have completed
Signed-off-by: Kevin May <kevin.may@arm.com>
Change-Id: I99f2bb0db847f5a52ab4c5705b072155c6b6f333
|
|
* Signed32 missing from CompareAndPrintOutput
Signed-off-by: Keith Davis <keith.davis@arm.com>
Change-Id: If3c93fb0d73c566ddcf439fceaa6d629029df18f
|
|
In ArmNNExecutor::ArmNNExecutor the call to m_Runtime->LoadNetwork was
ignoring the Status result and continuing to execute with a failed
network. In addition throwing an exception from the constructor resulted
in a segmentation fault.
* Modify IExecutor to allow the constructor to mark itself as failed.
* Modify ArmNNExecutor to mark itself as failed when LoadNetwork returns
an error.
* Modify ExecuteNetwork to check the value of m_constructionFailed.
Signed-off-by: Colm Donelan <colm.donelan@arm.com>
Change-Id: Idf222cb2b66e1051875dc67046734f2b00b288d1
|
|
* model was declared in the TfLiteExecutor constructor, instead of intializing m_Model
* Working with this model that has 4 output we saw the the output names were not correct, this got fixed too
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com>
Change-Id: I48f194ad4ba6af43d43e6eea336eb87ffee02dcc
|
|
Signed-off-by: Cathal Corbett <cathal.corbett@arm.com>
Change-Id: If00d8dab2846c484a1969fb152cb9f8bd16e1b3e
|
|
* dot file to be generated when -v is given. It was only being generated when using the delegate as executor
* output name read from m_Params.m_OutputNames instead of m_TfLiteInterpreter
* typo: "delage" instead of "delegate"
* QAsymmS8 templated as int8, instead of uint8
Change-Id: Ie13ae0f7e6395c0ebcb5ecda32e72082dee8aa6c
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com>
Change-Id: Iac97a23927ba42290ebeb3446bbd36da15045e07
|
|
This reverts commit 1a7f033768acb27da11503bd29abb468d2e77f9e.
List of fixes to be able to add this code again:
* "emplacing_back" the vector inputTensors into the vector m_InputTensorsVec outside the for loop
* GetIOInfo() uses IOptimizedNetwork instead of INetwork, where the infered shapes are not saved
* Add missing data type Signed32 to SetupInputsAndOutputs()
* PrintOutputTensors() prints the actual output without dequantizing
* Add profilingDetailsMethod as input in networkProperties in ArmNNExecutor constructor
* Fix typos
Change-Id: I91de166f87228282db3efa27431fe91458834442
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com>
Change-Id: Ic6634d48892d11e5f146cdf285e1e333e93e9937
Signed-off-by: Francis Murtagh <francis.murtagh@arm.com>
|
|
* "Asynchronous Execution with std::launch:async..."
* "Asynchronous Execution with Arm NN thread pool..."
Signed-off-by: Teresa Charlin <teresa.charlinreyes@arm.com>
Change-Id: I93f6ae92fd5599d1042f0dfced7e90ef85e20463
|