Added other dependencies and clarification about HF models#11

Open

randerzander wants to merge 13 commits intoartidoro:mainfrom randerzander:main

+32-19

Commits on May 25, 2023

Added other dependencies and clarification about HF models
randerzander
committed
Check for LlamaTokenizerFast rather than infer type from path name. Fix cases where non-standard llama model path names gets bypassed in tokenizer check. The tokenizer is init with use_fast=True an…
Qubitium
committed
Update README.md
artidoro
authored

Commits on May 27, 2023

Commits on May 30, 2023