diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 64 |
1 files changed, 58 insertions, 6 deletions
@@ -42,7 +42,9 @@ Information on reporting security issues can be found in ## License -ML Inference Advisor is licensed under [Apache License 2.0](LICENSES/Apache-2.0.txt). +ML Inference Advisor is licensed under [Apache License 2.0](LICENSES/Apache-2.0.txt) +unless otherwise indicated. This project contains software under a range of +permissive licenses, see [LICENSES](LICENSES/). ## Trademarks and copyrights @@ -181,6 +183,14 @@ documentation, e.g. in the candidates from the rewrite library, with or without training using a small portion of the training data, to achieve local performance gains. +The following rewrites are supported: + +* fully-connected - replaces a subgraph with a fully connected layer +* fully-connected-sparsity - replaces a subgraph with a pruned 2:4 sparse fully connected layer +* fully-connected-clustering - replaces a subgraph with a clustered fully connected layer +* conv2d-sparsity - replaces a subgraph with a pruned 2:4 sparse conv2d layer +* conv2d-clustering - replaces a subgraph with a clustered conv2d layer + **Note:** A ***Keras model*** (.h5 or SavedModel) is required as input to perform pruning and clustering. A ***TensorFlow Lite model*** is required as input to perform a rewrite. @@ -209,19 +219,51 @@ mlia optimize ~/models/ds_cnn_large_fp32.tflite \ --rewrite-end MobileNet/fc1/BiasAdd ``` -### optimization Profiles +### Optimization Profiles Training parameters for rewrites can be specified. -There are a number of predefined profiles: +There are a number of predefined profiles for rewrites shown below: | Name | Batch Size | LR | Show Progress | Steps | LR Schedule | Num Procs | Num Threads | Checkpoints | Augmentations | | :----------: | :--------: | :--: | :-----------: | :---: | :---------: | :-------: | :---------: | :---------: | :-------------: | | optimization | 32 | 1e-3 | True | 48000 | "cosine" | 1 | 0 | None | "gaussian" | +| Name | Batch Size | LR | Show Progress | Steps | LR Schedule | Num Procs | Num Threads | Checkpoints | Num Clusters | Cluster Centroids Init | +| :-------------------------------------: | :--------: | :--: | :-----------: | :---: | :---------: | :-------: | :---------: | :---------: | :----------: | :--------------------------------: | +| optimization-fully-connected-clustering | 32 | 1e-3 | True | 48000 | "cosine" | 1 | 0 | None | 16 | "CentroidInitialization.LINEAR" | + +| Name | Batch Size | LR | Show Progress | Steps | LR Schedule | Num Procs | Num Threads | Checkpoints | Sparsity M | Sparsity N | +| :-----------------------------------: | :--------: | :--: | :-----------: | :---: | :---------: | :-------: | :---------: | :---------: | :--------: | :--------: | +| optimization-fully-connected-pruning | 32 | 1e-3 | True | 48000 | "cosine" | 1 | 0 | None | 2 | 4 | + +| Name | Batch Size | LR | Show Progress | Steps | LR Schedule | Num Procs | Num Threads | Checkpoints | Num Clusters | Cluster Centroids Init | Activation | +| :-------------------------------------: | :--------: | :--: | :-----------: | :---: | :---------: | :-------: | :---------: | :---------: | :----------: | :--------------------------------: | :--------: | +| optimization-conv2d-clustering | 32 | 1e-3 | True | 48000 | "cosine" | 1 | 0 | None | 16 | "CentroidInitialization.LINEAR" | "relu" | + +| Name | Batch Size | LR | Show Progress | Steps | LR Schedule | Num Procs | Num Threads | Checkpoints | Sparsity M | Sparsity N | Activation | +| :-----------------------------------: | :--------: | :--: | :-----------: | :---: | :---------: | :-------: | :---------: | :---------: | :--------: | :--------: | :--------: | +| optimization-conv2d-pruning | 32 | 1e-3 | True | 48000 | "cosine" | 1 | 0 | None | 2 | 4 | "relu" | + +These are summarized below: + +* optimization - Provides training parameters for rewrites +* optimization-fully-connected-clustering - Provides training parameters for rewrites and cluster specific parameters for the fully-connected-clustering rewrite +* optimization-fully-connected-pruning - Provides training parameters for rewrites and pruning specific parameters for the fully-connected-sparsity rewrite +* optimization-conv2d-clustering - Provides training parameters for rewrites and cluster specific parameters for the conv2d-clustering rewrite +* optimization-conv2d-pruning - Provides training parameters for rewrites and pruning specific parameters for the conv2d-sparsity rewrite + +Note for convolutional rewrites (e.g. optimization-conv2d-pruning). The activation function for the rewrite can be selected in the optimization profile from the following list: + +* "relu" - Standard ReLU activation function +* "relu6" - ReLU6 activation function i.e. ReLU activation function capped at 6 +* "none" - No activation function + +The user can also specify custom augmentations as part of the training parameters. An example of this can be found in the following optimization profile: + | Name | Batch Size | LR | Show Progress | Steps | LR Schedule | Num Procs | Num Threads | Checkpoints | Augmentations - gaussian_strength | Augmentations - mixup_strength | | :------------------------------: | :--------: | :--: | :-----------: | :---: | :---------: | :-------: | :---------: | :---------: | :-------------------------------: | :----------------------------: | -| optimization_custom_augmentation | 32 | 1e-3 | True | 48000 | "cosine" | 1 | 0 | None | 0.1 | 0.1 | +| optimization-custom-augmentation | 32 | 1e-3 | True | 48000 | "cosine" | 1 | 0 | None | 0.1 | 0.1 | The augmentations consist of 2 parameters: mixup strength and gaussian strength. @@ -245,7 +287,7 @@ mlia optimize ~/models/ds_cnn_large_fp32.tflite \ --dataset input.tfrec \ --rewrite-target fully-connected \ --rewrite-start MobileNet/avg_pool/AvgPool \ - --rewrite-end MobileNet/fc1/BiasAdd_ + --rewrite-end MobileNet/fc1/BiasAdd ``` #### Custom optimization Profiles @@ -261,7 +303,17 @@ apply for each optimization. ``` bash # for custom profiles -mlia ops --optimization-profile ~/my_custom_optimization_profile.toml +mlia optimize --optimization-profile ~/my_custom_optimization_profile.toml +``` + +When providing rewrite-specific parameters e.g. for clustering, the rewrite name should be specified in the toml: + +For example, the following provides rewrite-specific parameters for the fully-connected-clustering rewrite + +``` bash +[rewrite.fully-connected-clustering] +num_clusters = 16 +cluster_centroids_init = "CentroidInitialization.LINEAR" ``` # Target profiles |