v0.35.0

PromptingTools v0.35.0

Diff since v0.34.0

Added

• Added a utility function to RAGTools reciprocal_rank_fusion, as a principled way to merge multiple rankings. See ?RAGTools.Experimental.reciprocal_rank_fusion for more information.

Commits

Merged pull requests:

• Add Reciprocal Rank Fusion (#173) (@svilupp)

v0.34.0

PromptingTools v0.34.0

Diff since v0.33.2

Added

• RankGPT implementation for RAGTools chunk re-ranking pipeline. See ?RAGTools.Experimental.rank_gpt for more information and corresponding reranker type ?RankGPTReranker.

Commits

Merged pull requests:

• Add RankGPT (#172) (@svilupp)

v0.33.2

PromptingTools v0.33.2

Diff since v0.33.1

Fixed

• Add back accidentally dropped DBKS keys

Commits

Merged pull requests:

• Add back API keys (#171) (@svilupp)

v0.33.1

PromptingTools v0.33.1

Diff since v0.33.0

Fixed

• Fixed loading RAGResult when one of the candidate fields was nothing.
• Utility type checks like isusermessage, issystemmessage, isdatamessage, isaimessage, istracermessage do not throw errors when given any arbitrary input types (previously they only worked for AbstractMessage types). It's a isa check, so it should work for all input types.
• Changed preference loading to use typed global instead of const, to fix issues with API keys not being loaded properly on start. You can now also call PromptingTools.load_api_keys!() to re-load the API keys (and ENV variables) manually.

Commits

Merged pull requests:

• Rag Tools fix + relaxing const for API key loading (#170) (@svilupp)

v0.33.0

PromptingTools v0.33.0

Diff since v0.32.0

Added

• Added registry record for Anthropic Claude 3.5 Sonnet with ID claude-3-5-sonnet-20240620 (read the blog post). Aliases "claude" and "claudes" have been linked to this latest Sonnet model.

Commits

Merged pull requests:

• Add Anthropic Sonnet 3.5 (#167) (@svilupp)

v0.32.0

PromptingTools v0.32.0

Diff since v0.31.1

Updated

• Changed behavior of RAGTools.rerank(::FlashRanker,...) to always dedupe input chunks (to reduce compute requirements).

Fixed

• Fixed a bug in verbose INFO log in RAGTools.rerank(::FlashRanker,...).

Commits

Merged pull requests:

• Update FlashRank to use only unique documents (#166) (@svilupp)

v0.31.1

PromptingTools v0.31.1

Diff since v0.31.0

Updated

• Improved the implementation of RAGTools.unpack_bits to be faster with fewer allocations.

Commits

Merged pull requests:

• Improve unpack_bits (#165) (@svilupp)

v0.31.0

PromptingTools v0.31.0

Diff since v0.30.0

Breaking Changes

• The return type of RAGTools.find_tags(::NoTagger,...) is now ::Nothing instead of CandidateChunks/MultiCandidateChunks with all documents.
• Base.getindex(::MultiIndex, ::MultiCandidateChunks) now always returns sorted chunks for consistency with the behavior of other getindex methods on *Chunks.

Updated

• Cosine similarity search now uses partialsortperm for better performance on large datasets.
• Skip unnecessary work when the tagging functionality in the RAG pipeline is disabled (find_tags with NoTagger always returns nothing which improves the compiled code).
• Changed the default behavior of getindex(::MultiIndex, ::MultiCandidateChunks) to always return sorted chunks for consistency with other similar functions. Note that you should always use re-rankering anyway (see FlashRank.jl).

Commits

Merged pull requests:

• Update RAG performance (Breaking changes) (#164) (@svilupp)

v0.30.0

PromptingTools v0.30.0

Diff since v0.29.0

Fixed

• Fixed a bug on Julia 1.11 beta by adding REPL stdlib as a direct dependency.
• Fixed too restrictive argument types for RAGTools.build_tags method.

Commits

Merged pull requests:

• Update deps for 1.11 (REPL) (#163) (@svilupp)

v0.29.0

PromptingTools v0.29.0

Diff since v0.28.0

Added

• Added a package extension for FlashRank.jl to support local ranking models. See ?RT.FlashRanker for more information or examples/RAG_with_FlashRank.jl for a quick example.

Commits

Merged pull requests:

• Add FlashRank.jl package extension (#162) (@svilupp)

Closed issues:

• Allow changing OpenAI key after first load (#161)