blob: 8475813a173df7696af8944aabc6290c7511566a (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
|
<!---
SPDX-FileCopyrightText: Copyright 2022, Arm Limited and/or its affiliates.
SPDX-License-Identifier: Apache-2.0
--->
# Docker™ image
This directory contains different utilities to build/test the TOSA Checker.
## How to build the TOSA Checker manylinux wheel with a Docker™ image
To create a Docker™ image for the TOSA Checker to build it for Python® 3.9 on various Linux® distributions, please run the following command:
```console
docker build . -t tc-cp39-manylinux --build-arg PYTHON_VERSION=3.9 -f Dockerfile
```
The TensorFlow™ source code is automatically downloaded and is located in the `/tensorflow_src` directory.
The command to run the container is:
```console
docker run -it -v <tosa_checker source code on your machine>:/tosa_checker tc-cp39-manylinux
```
Now call the following command to build a `tosa_checker` Python® wheel inside of the container:
```console
cd tosa_checker
python3 setup.py --tensorflow_src_dir /tensorflow_src bdist_wheel
```
The `tosa_checker` wheel can be found in the `/dist` directory.
Generate the new manylinux wheel from the `tosa_checker` wheel:
```console
auditwheel repair dist/<tosa_checker>.whl -w dist/
```
The `tosa_checker` manylinux wheel can now be found in the `dist/` directory.
Install the `tosa_checker` manylinux wheel:
```console
pip install dist/<tosa_checker-manyliux>.whl
```
## How to use the TOSA Checker Docker™ image with security countermeasures
A Docker™ image is provided for builds of the TOSA Checker with security countermeasures
that are used in the project's continuous integration system. The following countermeasures are provided
in this image:
* [Address Sanitizer (ASAN)](https://clang.llvm.org/docs/AddressSanitizer.html)
* [Undefined Behavior Sanitizer (UBSAN)](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html)
* [Bandit](https://pypi.org/project/bandit/)
This section will explain how to use this Docker™ image to build the TOSA checker with
sanitizers and lint the Python source code with Bandit.
### Building the TOSA Checker with sanitizers
To build the Docker™ image run the command below:
```console
docker build . -t tc-cp39-countermeasures --build-arg PYTHON_VERSION=3.9 -f CI.Dockerfile
```
*Note: In this example, the image is built with Python 3.9. This can be changed using the PYTHON_VERSION argument.*
After this, run the container as follows, mounting the source code to the container:
```console
docker run -it -v <tosa_checker source code on your machine>:/tosa_checker tc-cp39-countermeasures
```
Following this, build the TOSA checker using the following command:
```console
cd tosa_checker
python3.9 setup.py --tensorflow_src_dir /tensorflow_src --sanitizer <sanitizer option> bdist_wheel
```
Choose between `asan` or `ubsan` as the sanitizer option. The `tosa_checker` wheel can be found in the `dist/` directory.
The TOSA Checker wheel can then be installed as follows:
```console
python3.9 -m pip install dist/<tosa_checker>.whl
```
To then run the unit test of the TOSA Checker, the requirements for this must be installed:
```console
cd tests
python3.9 -m pip install -r requirements.txt
```
Then, if you're using the ASAN option:
```console
export LD_PRELOAD=$(clang -print-file-name=libclang_rt.asan-x86_64.so)
ASAN_OPTIONS=detect_leaks=0 python3.9 -m pytest --capture=no .
```
For the UBSAN option, run the following command:
```console
UBSAN_OPTIONS=print_stacktrace=1 python3.9 -m pytest --capture=no .
```
### Running the Bandit linter
Firstly build and run the Docker™ image:
```console
docker build . -t tc-cp39-countermeasures --build-arg PYTHON_VERSION=3.9 -f CI.Dockerfile
docker run -it -v <tosa_checker source code on your machine>:/tosa_checker tc-cp39-countermeasures
```
After this, a HTML report can be generated with Bandit as follows:
```console
cd tosa_checker
python3.9 -m bandit --configfile .bandit.yaml -r . -f html -o report.html
```
## Trademarks and Copyrights
* Python® is a registered trademark of the PSF.
* Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.
* Ubuntu® is a registered trademark of Canonical.
* TensorFlow™ is a trademark of Google® LLC.
* Docker™ is a trademark of Docker, Inc.
|
