Skip to content

Commit

Permalink
only AUTO will try other quant linears (ModelCloud#797)
Browse files Browse the repository at this point in the history
* only AUTO will try other linears

* add comments

* update comments
  • Loading branch information
CSY-ModelCloud authored Dec 6, 2024
1 parent cd44c6e commit a2cbcfa
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion gptqmodel/utils/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,10 @@ def make_quant(
try:
result = create_quant_layer(linear, bits, desc_act, dynamic, group_size, module, names, sym)
return result
except NotImplementedError:
except NotImplementedError as e:
# only fallback to other quant linears when backend is auto.
if backend not in [BACKEND.AUTO, BACKEND.AUTO_TRAINABLE]:
raise e
continue

raise ValueError("no support quant linear was found for this module.")
Expand Down

0 comments on commit a2cbcfa

Please sign in to comment.