Skip to content

Commit

Permalink
lint
Browse files Browse the repository at this point in the history
  • Loading branch information
horheynm committed May 24, 2024
1 parent a61ad67 commit 933c474
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion examples/llama7b_sparse_quantized/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,5 @@ model.save_pretrained(compressed_output_dir, save_compressed=True)

### Custom Quantization
The current repo supports multiple quantization techniques configured using a recipe. Supported strategies are `tensor`, `group` and `channel`.
The above recipe (`2:4_w4a16_recipe.yaml`) uses channel-wise quantization specified by ` strategy: "channel"` in its config group.
The above recipe (`2:4_w4a16_recipe.yaml`) uses channel-wise quantization specified by `strategy: "channel"` in its config group.
To use quantize per tensor, change strategy from `channel` to `tensor`. To use group size quantization, change from `channel` to `group` and specify its value, say 128, by including `group_size: 128`. Group size quantization example is shown in `2:4_w4a16_group-128_recipe.yaml`

0 comments on commit 933c474

Please sign in to comment.