Skip to content

Commit

Permalink
Add eval script (#102)
Browse files Browse the repository at this point in the history 
* Allow inference to produce only new tokens

Signed-off-by: Alex-Brooks <[email protected]>

* Add script for f1/accuracy model evaluation

Signed-off-by: Alex-Brooks <[email protected]>

* Stubbed reimplementation for multiclass multilabel metrics

Signed-off-by: Alex-Brooks <[email protected]>

* Update eval scriipt for multioutput

Signed-off-by: Alex-Brooks <[email protected]>

* Code fmt and linting

Signed-off-by: Alex-Brooks <[email protected]>

* Add precision and recall metrics

Signed-off-by: Alex-Brooks <[email protected]>

* docstrings, zero division for prec/recall

Signed-off-by: Alex-Brooks <[email protected]>

* Formatting, [hack] add sklearn to deps

Signed-off-by: Alex-Brooks <[email protected]>

* Explcitly set nan behavior for f1

Signed-off-by: Alex-Brooks <[email protected]>

---------

Signed-off-by: Alex-Brooks <[email protected]>
  • Loading branch information
@alex-jw-brooks
alex-jw-brooks authored Apr 10, 2024
1 parent cdcddce commit b04db32
Show file tree
Hide file tree
Showing 3 changed files with 455 additions and 2 deletions.
3 changes: 3 additions & 0 deletions requirements.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@ peft>=0.8.0
datasets>=2.15.0
fire

# Dependencies needed only to run scripts.
# TODO: Figure out a better place to put these.
scikit-learn>=1.0, <2.0
Loading

0 comments on commit b04db32

Please sign in to comment.