aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorerik.andersson@arm.com <erik.andersson@arm.com>2021-08-25 16:42:19 +0200
committererik.andersson@arm.com <erik.andersson@arm.com>2021-11-26 13:01:46 +0100
commit480e31b477f877d254f95d2984c1a5b60e068450 (patch)
treed9f9f8bbad556c29940a0957f0b9937ba8234504
parent837c31c2912386e4cdfac36f019dd73620970715 (diff)
downloadethos-u-vela-480e31b477f877d254f95d2984c1a5b60e068450.tar.gz
MLBEDSW-4996: Community bug reporting on ML Platform
Created "BUGS.md" which details to the Vela Community how to issue bug report using the Maniphest Bug Tracker. Also added a reference to it in "README.md". Signed-off-by: erik.andersson@arm.com <erik.andersson@arm.com> Change-Id: I0120a890c8447907e32de6b10a24eceade09df7d
-rw-r--r--BUGS.md173
-rw-r--r--README.md9
2 files changed, 180 insertions, 2 deletions
diff --git a/BUGS.md b/BUGS.md
new file mode 100644
index 0000000..c1c499e
--- /dev/null
+++ b/BUGS.md
@@ -0,0 +1,173 @@
+# Vela Community Bug Reporting
+
+If the Vela community encounters a bug, then we highly encourage the community
+to report it. When issuing a report, use the
+[Maniphest bug tracker](https://developer.mlplatform.org/maniphest/)
+that is part of Phabricator on ML Platform. This site enables any Vela user to
+discuss and share any Vela related issue with the community.
+
+The only prerequisite for this process is a GitHub account, which will be used
+when creating an account on ML platform.
+
+If you also have a fix for the bug you have identified, then we encourage
+you to follow the [contributions guide](CONTRIBUTIONS.md).
+
+The bug reporting process in its entirety is detailed below.
+
+## Maniphest bug tracker
+
+ML Platform uses a bug tracking system known as
+[Maniphest](https://developer.mlplatform.org/maniphest/),
+which is part of the Phabricator tool set.
+
+On the Maniphest platform, one can add active bugs using the Tasks and Bugs
+tab which will then be addressable by the Vela community.
+
+The system is accessible directly via the link above. Alternatively, one can
+navigate to [ML Platform](https://mlplatform.org) and select the "Developers"
+drop down menu followed by "Development Tracker", which will bring you to
+Phabricator. In order to initiate a bug report, select "Tasks and Bugs".
+
+Anyone can browse the Maniphest bug tracker, since it is a public forum.
+However, in order to file your own ticket you need to be logged in
+to an account on ML Platform. These can be created by clicking "Register" at
+<https://developer.mlplatform.org/auth/> and then linking a GitHub ID.
+
+### Proprietary or Sensitive Information
+
+As mentioned previously, ML Platform and the Maniphest Bug tracking system
+are public forums. As such, anyone can see information posted to them.
+Therefore, any proprietary information you wish to share should be done using
+Arm's ML Model Review described below.
+
+## Bug Reporting Process
+
+The entire process of handling a bug is summarized in the following steps.
+
+1. Discovering the bug
+ * A user notices a discrepancy in the functionality of Vela.
+ This can be as simple as Vela crashing upon runtime,
+ where an error message is printed out for the user to read.
+ However, in certain cases the bug is much harder to spot.
+ For example, Vela might run gracefully but optimize a network incorrectly.
+ Thus the error will manifest in a silent error, such as incorrect outputs.
+ In this case, as much information about the system and settings
+ should be noted. This will be used later on when handling the root cause.
+ A sufficient documentation of the problem is crucial when dealing with
+ these silent errors.
+
+2. Sharing the existence of the bug
+ * In order to resolve the issue, its existence needs to be shared
+ with the Vela community. Even though the user could fix it by themselves
+ in the first step, it is important that is is shared with the community.
+ This step will be detailed in the following sections.
+
+3. The community tackles the problem
+ * When the bug has been properly detailed, the Vela community will try to
+ find a solution. During this phase, further information may be requested
+ or shared through the corresponding bug ticket. If a solution is found,
+ the ticket will be updated with the relevant information.
+
+As mentioned, the second step is detailed below.
+
+### Opening the form
+
+To add a new bug report, navigate to the
+[Bug tracker](https://developer.mlplatform.org/maniphest/).
+
+Click the button "+ Create Task" or navigate to
+[the following page](https://developer.mlplatform.org/maniphest/task/edit/form/default/)
+
+### Title
+
+Write a title that describes the problem in a short and meaningful way.
+An example of a good title is "AssertionError in weight_compressor.py
+when optimizing DS-CNN". An example of a poor title is "Vela crashes on Linux".
+
+### Assigned To
+
+Leave this field blank, the report will be assigned when work is started on it.
+
+### Status
+
+If you were able to resolve the bug before submission and this ticket is simply
+a notification to the Vela Community, select the "Resolved" option. Also,
+follow the [contributions guide](CONTRIBUTIONS.md) in order to apply your fix.
+If the bug has not yet been resolved, select the "Open" option instead.
+
+### Description
+
+This section should contain as much information as possible,
+in order to allow someone else to reproduce the issue.
+
+Please provide the following details:
+
+- Vela command line and options.
+
+- Configuration files. If these contain information that you
+do not wish to make public then please upload them using Arm's
+[ML Model Review](https://www.arm.com/resources/contact-us/ml-model-review)
+process described below.
+
+- Version of Vela used.
+
+- Error messages or other console output from Vela.
+
+- Link to a publicly accessible copy of the input network.
+If this is not possible then you will need to upload the network using Arm's
+[ML Model Review](https://www.arm.com/resources/contact-us/ml-model-review)
+process described below.
+
+- License files related to your network, if applicable.
+
+Attached to the description you can find a toolbox, which contains a button to
+upload files. This shall not be used to upload any network files, instead use
+[ML Model Review](https://www.arm.com/resources/contact-us/ml-model-review).
+
+### Visible To
+
+ML Platform and the Manifest Bug Tracking System are public forums.
+As such, anyone can see information posted to them.
+
+Thus, any proprietary information you wish to share should be done using Arm's
+[ML Model Review](https://www.arm.com/resources/contact-us/ml-model-review).
+described below.
+
+Hence, the content of the report should be freely viewable to the public
+and thus the "Public" option should be selected in the "Visility" field.
+
+### Editable by
+
+In this field, restrict the editing privileges to yourself and members of the
+Vela Optimizer group. This is done by selecting "Subscribers" in the
+"Object Policies" entry of the "Editable by" drop down menu.
+
+Proceed by navigating down to the "Subscribers" field and typing in both your
+username and "Vela Optimizer". To clarify, your username and "Vela Optimizer"
+shall be separate entries in the "Subscribers" field.
+
+### Tags
+
+Finally, add "Vela Optimizer" in the "Tags" field. This ensures that the bug
+is directed to the Vela team, enabling a rapid triage of the problem.
+
+### Submitting the report
+
+Submit the report by clicking the "Create New Task" button.
+
+A confirmation e-mail will be sent out to the linked e-mail address.
+Any further changes to the report will be sent out in the same way.
+
+## Arm's [ML Model Review](https://www.arm.com/resources/contact-us/ml-model-review) process.
+
+As mentioned earlier, proprietary or sensitive information should not be shared
+on this page.
+Instead, any required sensitive content should be sent directly to Arm via
+Arm's ML Model Review process.
+
+For instance, if you can't find any publicly hosted examples of your network,
+e.g. a network hosted on Model Zoo, upload your own network to
+[ML Model Review](https://www.arm.com/resources/contact-us/ml-model-review).
+
+It is a simple process that requires you to complete a short online form
+after which you will be sent instructions on how to upload your files. \ No newline at end of file
diff --git a/README.md b/README.md
index f8e57aa..40d9945 100644
--- a/README.md
+++ b/README.md
@@ -23,7 +23,7 @@ embedded system.
The tool will also generate performance estimates (EXPERIMENTAL) for the
compiled model.
-The tool has limited functionality for compiling a
+The tool has limited functionality for compiling a
[TOSA](https://git.mlplatform.org/tosa/specification.git/) neural network
(EXPERIMENTAL).
@@ -171,7 +171,7 @@ vela --optimise Performance my_model.tflite
```
6) Compile a network while optimising for the fastest inference time possible,
-with an upper bound for the SRAM usage. The memory limit is set in bytes, i.e.
+with an upper bound for the SRAM usage. The memory limit is set in bytes, i.e.
run the following example if one requires a limit of 300KB.
```bash
@@ -242,6 +242,11 @@ operators supported in this release.
Please see [Vela Testing](TESTING.md).
+## Bug Reporting
+
+Please see [Vela Community Bug Reporting](BUGS.md) for a description of how to
+report bugs.
+
## Resources
Additional useful information: