Code

Add marlin int4 kernel#315

Code

Closed

dacorvo wants to merge 9 commits intomainhuggingface/optimum-quanto:mainfrom add_marlin_int4_kernelhuggingface/optimum-quanto:add_marlin_int4_kernelCopy head branch name to clipboard

+1,668-4

Commits

Commits on Sep 26, 2024

• feat(library): add original marlin fp16i4 kernel

  dacorvo

  committed
  a756fef

  Copy full SHA for a756fef

• feat: modify marlin fp16 int4 kernel to use scaled zeropoint

  Show description for 6c087ca
  

  shcho1118

  authored and

  dacorvo

  committed
  6c087ca

  Copy full SHA for 6c087ca

• fix(marlin): avoid kernel crash on H100

  Show description for 247aeac
  dacorvo

  committed
  247aeac

  Copy full SHA for 247aeac

• feat(library): add Marlin gemm_f16_i4 op

  dacorvo

  committed
  4b5b17e

  Copy full SHA for 4b5b17e

• feat: add MarlinInt4PackedTensor

  

  shcho1118

  authored and

  dacorvo

  committed
  77a4df4

  Copy full SHA for 77a4df4

• feat(marlin): add scales/shifts permutations

  dacorvo

  committed
  409d26e

  Copy full SHA for 409d26e

• test: add test_gemm_marlin_fp16_int4

  Show description for 0366a03
  

  shcho1118

  authored and

  dacorvo

  committed
  0366a03

  Copy full SHA for 0366a03

• perf: add Marlin to w4a16 benchmark

  

  shcho1118

  authored and

  dacorvo

  committed
  ed58970

  Copy full SHA for ed58970

• feat(qtensor): add MarlinQBitsTensor

  dacorvo

  committed
  5564b8f

  Copy full SHA for 5564b8f