Skip to content

Issues: NVIDIA/garak

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

316 Open 306 Closed
316 Open 306 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

detector: StringDetector matchtype fullmatch detectors work on code that inherits from or manages Detector new plugin Describes an entirely new probe, detector, generator or harness
#1084 opened Jan 17, 2025 by leondz
Crash due to zscore not initialised error when using the Azure generator as DAN probes are skipped bug Something isn't working
#1082 opened Jan 16, 2025 by samisouabni
2
Add caching to GH actions environment for HF items housekeeping Repository and project maintenance
#1080 opened Jan 15, 2025 by leondz 25.02 Efficiency
Add dry run option that includes stats architecture Architectural upgrades generators Interfaces with LLMs
#1079 opened Jan 15, 2025 by leondz 25.02 Efficiency
Output estimated run cost at top of run architecture Architectural upgrades
#1078 opened Jan 15, 2025 by leondz 25.02 Efficiency
Order prompts within probes for max bag attack success rate architecture Architectural upgrades probes Content & activity of LLM probes
#1077 opened Jan 15, 2025 by leondz 25.02 Efficiency
Estimate # prompts, # input tokens per probe architecture Architectural upgrades
#1071 opened Jan 11, 2025 by leondz 25.02 Efficiency
Enable stopping when call (/token) budget hit architecture Architectural upgrades
#1070 opened Jan 11, 2025 by leondz 25.02 Efficiency
Automate updating the calibration process housekeeping Repository and project maintenance
#1069 opened Jan 10, 2025 by leondz 25.02 Efficiency
Harness that orders probes for expected attack success rate (ASR) according to bag architecture Architectural upgrades
#1068 opened Jan 10, 2025 by leondz 25.02 Efficiency
2
Add “real” multi-turn support to chat-enabled generators architecture Architectural upgrades generators Interfaces with LLMs
#1067 opened Jan 10, 2025 by leondz 25.02 Efficiency
probe: Adapt sandwich attack to auto-find effective languages architecture Architectural upgrades probes Content & activity of LLM probes
#1066 opened Jan 10, 2025 by leondz
2 tasks
25.02 Efficiency
1
Bayesian optimisation base class architecture Architectural upgrades probes Content & activity of LLM probes
#1065 opened Jan 10, 2025 by leondz 25.02 Efficiency
Pattern for probes that can react to generator/detector responses architecture Architectural upgrades probes Content & activity of LLM probes
#1064 opened Jan 10, 2025 by leondz 25.02 Efficiency
docs: model type and name mixed up documentation Improvements or additions to documentation
#1063 opened Jan 10, 2025 by dennisrall
1
support for python 3.13 architecture Architectural upgrades
#1062 opened Jan 8, 2025 by leondz
7 tasks
2
Include information about why detectors are skipped detectors work on code that inherits from or manages Detector quality-accuracy This affects result quality/reliability reporting Reporting, analysis, and other per-run result functions ux Interface & interaction improvements
#1061 opened Jan 7, 2025 by erickgalinkin 25.02 Efficiency
@erickgalinkin
probe: text augmentations
#1056 opened Jan 3, 2025 by rafaelsandroni
Detector: Shields, to determine if an LLM moderator is functioning
#1055 opened Jan 2, 2025 by Eric-Hacker
package garak for kali housekeeping Repository and project maintenance
#1036 opened Dec 5, 2024 by leondz
@jmartin-tech
3
error: 'latin-1' codec can't encode character '\u2026' bug Something isn't working
#1035 opened Dec 4, 2024 by wiktorw767
5
probes: fileformats probe fails when targeting a local modal path bug Something isn't working
#1034 opened Dec 3, 2024 by jmartin-tech
Fall back when online services can't be reached. bug Something isn't working
#1033 opened Dec 2, 2024 by erickgalinkin
1
align prompt count per probe architecture Architectural upgrades probes Content & activity of LLM probes
#1032 opened Dec 2, 2024 by leondz 25.02 Efficiency
reporting: enable logging & skipping of unrecognised probes, detectors in digest generation reporting Reporting, analysis, and other per-run result functions
#1024 opened Nov 25, 2024 by leondz
ProTip! Follow long discussions with comments:>50.