PromptingTools.jl

Appearance

Reference for RAGTools ​

# PromptingTools.Experimental.RAGTools — Module.
julia
RAGTools

Provides Retrieval-Augmented Generation (RAG) functionality.

Requires: LinearAlgebra, SparseArrays, Unicode, PromptingTools for proper functionality.

This module is experimental and may change at any time. It is intended to be moved to a separate package in the future.

source


# PromptingTools.Experimental.RAGTools.AbstractCandidateChunks — Type.
julia
AbstractCandidateChunks

Abstract type for storing candidate chunks, ie, references to items in a AbstractChunkIndex.

Return type from find_closest and find_tags functions.

Required Fields

  • index_id::Symbol: the id of the index from which the candidates are drawn

  • positions::Vector{Int}: the positions of the candidates in the index

  • scores::Vector{Float32}: the similarity scores of the candidates from the query (higher is better)

source


# PromptingTools.Experimental.RAGTools.AbstractChunkIndex — Type.
julia
AbstractChunkIndex <: AbstractDocumentIndex

Main abstract type for storing document chunks and their embeddings. It also stores tags and sources for each chunk.

Required Fields

  • id::Symbol: unique identifier of each index (to ensure we're using the right index with CandidateChunks)

  • chunks::Vector{<:AbstractString}: underlying document chunks / snippets

  • embeddings::Union{Nothing, Matrix{<:Real}}: for semantic search

  • tags::Union{Nothing, AbstractMatrix{<:Bool}}: for exact search, filtering, etc. This is often a sparse matrix indicating which chunks have the given tag (see tag_vocab for the position lookup)

  • tags_vocab::Union{Nothing, Vector{<:AbstractString}}: vocabulary for the tags matrix (each column in tags is one item in tags_vocab and rows are the chunks)

  • sources::Vector{<:AbstractString}: sources of the chunks

  • extras::Union{Nothing, AbstractVector}: additional data, eg, metadata, source code, etc.

source


# PromptingTools.Experimental.RAGTools.AbstractGenerator — Type.
julia
AbstractGenerator <: AbstractGenerationMethod

Abstract type for generating an answer with generate! (use to change the process / return type of generate).

Required Fields

  • contexter::AbstractContextBuilder: the context building method, dispatching `build_context!

  • answerer::AbstractAnswerer: the answer generation method, dispatching answer!

  • refiner::AbstractRefiner: the answer refining method, dispatching refine!

  • postprocessor::AbstractPostprocessor: the postprocessing method, dispatching postprocess!

source


# PromptingTools.Experimental.RAGTools.AbstractIndexBuilder — Type.
julia
AbstractIndexBuilder

Abstract type for building an index with build_index (use to change the process / return type of build_index).

Required Fields

  • chunker::AbstractChunker: the chunking method, dispatching get_chunks

  • embedder::AbstractEmbedder: the embedding method, dispatching get_embeddings

  • tagger::AbstractTagger: the tagging method, dispatching get_tags

source


# PromptingTools.Experimental.RAGTools.AbstractMultiIndex — Type.
julia
AbstractMultiIndex <: AbstractDocumentIndex

Experimental abstract type for storing multiple document indexes. Not yet implemented.

source


# PromptingTools.Experimental.RAGTools.AbstractRetriever — Type.
julia
AbstractRetriever <: AbstractRetrievalMethod

Abstract type for retrieving chunks from an index with retrieve (use to change the process / return type of retrieve).

Required Fields

  • rephraser::AbstractRephraser: the rephrasing method, dispatching rephrase

  • finder::AbstractSimilarityFinder: the similarity search method, dispatching find_closest

  • filter::AbstractTagFilter: the tag matching method, dispatching find_tags

  • reranker::AbstractReranker: the reranking method, dispatching rerank

source


# PromptingTools.Experimental.RAGTools.AdvancedGenerator — Type.
julia
AdvancedGenerator <: AbstractGenerator

Default implementation for generate!. It simply enumerates context snippets and runs aigenerate (no refinement).

It uses ContextEnumerator, SimpleAnswerer, SimpleRefiner, and NoPostprocessor as default contexter, answerer, refiner, and postprocessor.

source


# PromptingTools.Experimental.RAGTools.AdvancedRetriever — Type.
julia
AdvancedRetriever <: AbstractRetriever

Dispatch for retrieve with advanced retrieval methods to improve result quality. Compared to SimpleRetriever, it adds rephrasing the query and reranking the results.

Fields

  • rephraser::AbstractRephraser: the rephrasing method, dispatching rephrase - uses HyDERephraser

  • embedder::AbstractEmbedder: the embedding method, dispatching get_embeddings (see Preparation Stage for more details) - uses BatchEmbedder

  • processor::AbstractProcessor: the processor method, dispatching get_keywords (see Preparation Stage for more details) - uses NoProcessor

  • finder::AbstractSimilarityFinder: the similarity search method, dispatching find_closest - uses CosineSimilarity

  • tagger::AbstractTagger: the tag generating method, dispatching get_tags (see Preparation Stage for more details) - uses NoTagger

  • filter::AbstractTagFilter: the tag matching method, dispatching find_tags - uses NoTagFilter

  • reranker::AbstractReranker: the reranking method, dispatching rerank - uses CohereReranker

source


# PromptingTools.Experimental.RAGTools.AllTagFilter — Type.
julia
AllTagFilter <: AbstractTagFilter

Finds the chunks that have ALL OF the specified tag(s). A method for find_tags.

source


# PromptingTools.Experimental.RAGTools.AnnotatedNode — Type.
julia
AnnotatedNode{T}  <: AbstractAnnotatedNode

A node to add annotations to the generated answer in airag

Annotations can be: sources, scores, whether its supported or not by the context, etc.

Fields

  • group_id::Int: Unique identifier for the same group of nodes (eg, different lines of the same code block)

  • parent::Union{AnnotatedNode, Nothing}: Parent node that current node was built on

  • children::Vector{AnnotatedNode}: Children nodes

  • `score::

source


# PromptingTools.Experimental.RAGTools.AnyTagFilter — Type.
julia
AnyTagFilter <: AbstractTagFilter

Finds the chunks that have ANY OF the specified tag(s). A method for find_tags.

source


# PromptingTools.Experimental.RAGTools.BM25Similarity — Type.
julia
BM25Similarity <: AbstractSimilarityFinder

Finds the closest chunks to a query embedding by measuring the BM25 similarity between the query and the chunks' embeddings in binary form. A method for find_closest.

Reference: Wikipedia: BM25. Implementation follows: The Next Generation of Lucene Relevance.

source


# PromptingTools.Experimental.RAGTools.BatchEmbedder — Type.
julia
BatchEmbedder <: AbstractEmbedder

Default embedder for get_embeddings functions. It passes individual documents to be embedded in chunks to aiembed.

source


# PromptingTools.Experimental.RAGTools.BinaryBatchEmbedder — Type.
julia
BinaryBatchEmbedder <: AbstractEmbedder

Same as BatchEmbedder but reduces the embeddings matrix to a binary form (eg, BitMatrix). Defines a method for get_embeddings.

Reference: HuggingFace: Embedding Quantization.

source


# PromptingTools.Experimental.RAGTools.BinaryCosineSimilarity — Type.
julia
BinaryCosineSimilarity <: AbstractSimilarityFinder

Finds the closest chunks to a query embedding by measuring the Hamming distance AND cosine similarity between the query and the chunks' embeddings in binary form. A method for find_closest.

It follows the two-pass approach:

  • First pass: Hamming distance in binary form to get the top_k * rescore_multiplier (ie, more than top_k) candidates.

  • Second pass: Rescore the candidates with float embeddings and return the top_k.

Reference: HuggingFace: Embedding Quantization.

source


# PromptingTools.Experimental.RAGTools.BitPackedBatchEmbedder — Type.
julia
BitPackedBatchEmbedder <: AbstractEmbedder

Same as BatchEmbedder but reduces the embeddings matrix to a binary form packed in UInt64 (eg, BitMatrix.chunks). Defines a method for get_embeddings.

See also utilities pack_bits and unpack_bits to move between packed/non-packed binary forms.

Reference: HuggingFace: Embedding Quantization.

source


# PromptingTools.Experimental.RAGTools.BitPackedCosineSimilarity — Type.
julia
BitPackedCosineSimilarity <: AbstractSimilarityFinder

Finds the closest chunks to a query embedding by measuring the Hamming distance AND cosine similarity between the query and the chunks' embeddings in binary form. A method for find_closest.

The difference to BinaryCosineSimilarity is that the binary values are packed into UInt64, which is more efficient.

Reference: HuggingFace: Embedding Quantization. Implementation of hamming_distance is based on TinyRAG.

source


# PromptingTools.Experimental.RAGTools.CandidateChunks — Type.
julia
CandidateChunks

A struct for storing references to chunks in the given index (identified by index_id) called positions and scores holding the strength of similarity (=1 is the highest, most similar). It's the result of the retrieval stage of RAG.

Fields

  • index_id::Symbol: the id of the index from which the candidates are drawn

  • positions::Vector{Int}: the positions of the candidates in the index (ie, 5 refers to the 5th chunk in the index - chunks(index)[5])

  • scores::Vector{Float32}: the similarity scores of the candidates from the query (higher is better)

source


# PromptingTools.Experimental.RAGTools.ChunkEmbeddingsIndex — Type.
julia
ChunkEmbeddingsIndex

Main struct for storing document chunks and their embeddings. It also stores tags and sources for each chunk.

Previously, this struct was called ChunkIndex.

Fields

  • id::Symbol: unique identifier of each index (to ensure we're using the right index with CandidateChunks)

  • chunks::Vector{<:AbstractString}: underlying document chunks / snippets

  • embeddings::Union{Nothing, Matrix{<:Real}}: for semantic search

  • tags::Union{Nothing, AbstractMatrix{<:Bool}}: for exact search, filtering, etc. This is often a sparse matrix indicating which chunks have the given tag (see tag_vocab for the position lookup)

  • tags_vocab::Union{Nothing, Vector{<:AbstractString}}: vocabulary for the tags matrix (each column in tags is one item in tags_vocab and rows are the chunks)

  • sources::Vector{<:AbstractString}: sources of the chunks

  • extras::Union{Nothing, AbstractVector}: additional data, eg, metadata, source code, etc.

source


# PromptingTools.Experimental.RAGTools.ChunkKeywordsIndex — Type.
julia
ChunkKeywordsIndex

Struct for storing chunks of text and associated keywords for BM25 similarity search.

Fields

  • id::Symbol: unique identifier of each index (to ensure we're using the right index with CandidateChunks)

  • chunks::Vector{<:AbstractString}: underlying document chunks / snippets

  • chunkdata::Union{Nothing, AbstractMatrix{<:Real}}: for similarity search, assumed to be DocumentTermMatrix

  • tags::Union{Nothing, AbstractMatrix{<:Bool}}: for exact search, filtering, etc. This is often a sparse matrix indicating which chunks have the given tag (see tag_vocab for the position lookup)

  • tags_vocab::Union{Nothing, Vector{<:AbstractString}}: vocabulary for the tags matrix (each column in tags is one item in tags_vocab and rows are the chunks)

  • sources::Vector{<:AbstractString}: sources of the chunks

  • extras::Union{Nothing, AbstractVector}: additional data, eg, metadata, source code, etc.

Example

We can easily create a keywords-based index from a standard embeddings-based index.

julia

# Let's assume we have a standard embeddings-based index
index = build_index(SimpleIndexer(), texts; chunker_kwargs = (; max_length=10))

# Creating an additional index for keyword-based search (BM25), is as simple as
index_keywords = ChunkKeywordsIndex(index)

# We can immediately create a MultiIndex (a hybrid index holding both indices)
multi_index = MultiIndex([index, index_keywords])

You can also build the index via build_index

julia
# given some sentences and sources
index_keywords = build_index(KeywordsIndexer(), sentences; chunker_kwargs=(; sources))

# Retrive closest chunks with
retriever = SimpleBM25Retriever()
result = retrieve(retriever, index_keywords, "What are the best practices for parallel computing in Julia?")
result.context

If you want to use airag, don't forget to specify the config to make sure keywords are processed (ie, tokenized) and that BM25 is used for searching candidates

julia
cfg = RAGConfig(; retriever = SimpleBM25Retriever());
airag(cfg, index_keywords;
    question = "What are the best practices for parallel computing in Julia?")

source


# PromptingTools.Experimental.RAGTools.ChunkKeywordsIndex — Method.
julia
ChunkKeywordsIndex(
    [processor::AbstractProcessor=KeywordsProcessor(),] index::ChunkEmbeddingsIndex; verbose::Int = 1,
    index_id = gensym("ChunkKeywordsIndex"), processor_kwargs...)

Convenience method to quickly create a ChunkKeywordsIndex from an existing ChunkEmbeddingsIndex.

Example

julia

# Let's assume we have a standard embeddings-based index
index = build_index(SimpleIndexer(), texts; chunker_kwargs = (; max_length=10))

# Creating an additional index for keyword-based search (BM25), is as simple as
index_keywords = ChunkKeywordsIndex(index)

# We can immediately create a MultiIndex (a hybrid index holding both indices)
multi_index = MultiIndex([index, index_keywords])

source


# PromptingTools.Experimental.RAGTools.CohereReranker — Type.
julia
CohereReranker <: AbstractReranker

Rerank strategy using the Cohere Rerank API. Requires an API key. A method for rerank.

source


# PromptingTools.Experimental.RAGTools.ContextEnumerator — Type.
julia
ContextEnumerator <: AbstractContextBuilder

Default method for build_context! method. It simply enumerates the context snippets around each position in candidates. When possibly, it will add surrounding chunks (from the same source).

source


# PromptingTools.Experimental.RAGTools.CosineSimilarity — Type.
julia
CosineSimilarity <: AbstractSimilarityFinder

Finds the closest chunks to a query embedding by measuring the cosine similarity between the query and the chunks' embeddings. A method for find_closest (see the docstring for more details and usage example).

source


# PromptingTools.Experimental.RAGTools.DocumentTermMatrix — Type.
julia
DocumentTermMatrix{T<:AbstractString}

A sparse matrix of term frequencies and document lengths to allow calculation of BM25 similarity scores.

source


# PromptingTools.Experimental.RAGTools.FileChunker — Type.
julia
FileChunker <: AbstractChunker

Chunker when you provide file paths to get_chunks functions.

Ie, the inputs will be validated first (eg, file exists, etc) and then read into memory.

Set as default chunker in get_chunks functions.

source


# PromptingTools.Experimental.RAGTools.FlashRanker — Type.
julia
FlashRanker <: AbstractReranker

Rerank strategy using the package FlashRank.jl and local models. A method for rerank.

You must first import the FlashRank.jl package. To automatically download any required models, set your ENV["DATADEPS_ALWAYS_ACCEPT"] = true (see DataDeps for more details).

Example

julia
using FlashRank

# Wrap the model to be a valid Ranker recognized by RAGTools
# It will be provided to the airag/rerank function to avoid instantiating it on every call
reranker = FlashRank.RankerModel(:mini) |> FlashRanker
# You can choose :tiny or :mini

## Apply to the pipeline configuration, eg, 
cfg = RAGConfig(; retriever = AdvancedRetriever(; reranker))

# Ask a question (assumes you have some `index`)
question = "What are the best practices for parallel computing in Julia?"
result = airag(cfg, index; question, return_all = true)

source


# PromptingTools.Experimental.RAGTools.HTMLStyler — Type.
julia
HTMLStyler

Defines styling via classes (attribute class) and styles (attribute style) for HTML formatting of AbstractAnnotatedNode

source


# PromptingTools.Experimental.RAGTools.HyDERephraser — Type.
julia
HyDERephraser <: AbstractRephraser

Rephraser implemented using the provided AI Template (eg, ...) and standard chat model. A method for rephrase.

It uses a prompt-based rephrasing method called HyDE (Hypothetical Document Embedding), where instead of looking for an embedding of the question, we look for the documents most similar to a synthetic passage that would be a good answer to our question.

Reference: Arxiv paper.

source


# PromptingTools.Experimental.RAGTools.JudgeAllScores — Type.

final_rating is the average of all scoring criteria. Explain the final_rating in rationale

source


# PromptingTools.Experimental.RAGTools.JudgeRating — Type.

Provide the final_rating between 1-5. Provide the rationale for it.

source


# PromptingTools.Experimental.RAGTools.KeywordsIndexer — Type.
julia
KeywordsIndexer <: AbstractIndexBuilder

Keyword-based index (BM25) to be returned by build_index.

It uses TextChunker, KeywordsProcessor, and NoTagger as default chunker, processor, and tagger.

source


# PromptingTools.Experimental.RAGTools.KeywordsProcessor — Type.
julia
KeywordsProcessor <: AbstractProcessor

Default keywords processor for get_keywords functions. It normalizes the documents, tokenizes them and builds a DocumentTermMatrix.

source


# PromptingTools.Experimental.RAGTools.MultiCandidateChunks — Type.
julia
MultiCandidateChunks

A struct for storing references to multiple sets of chunks across different indices. Each set of chunks is identified by an index_id in index_ids, with corresponding positions in the index and scores indicating the strength of similarity.

This struct is useful for scenarios where candidates are drawn from multiple indices, and there is a need to keep track of which candidates came from which index.

Fields

  • index_ids::Vector{Symbol}: the ids of the indices from which the candidates are drawn

  • positions::Vector{TP}: the positions of the candidates in their respective indices

  • scores::Vector{TD}: the similarity scores of the candidates from the query

source


# PromptingTools.Experimental.RAGTools.MultiFinder — Type.
julia
MultiFinder <: AbstractSimilarityFinder

Composite finder for MultiIndex where we want to set multiple finders for each index. A method for find_closest. Positions correspond to indexes(::MultiIndex).

source


# PromptingTools.Experimental.RAGTools.MultiIndex — Type.
julia
MultiIndex

Composite index that stores multiple ChunkIndex objects and their embeddings.

Fields

  • id::Symbol: unique identifier of each index (to ensure we're using the right index with CandidateChunks)

  • indexes::Vector{<:AbstractChunkIndex}: the indexes to be combined

Use accesor indexes to access the individual indexes.

Examples

We can create a MultiIndex from a vector of AbstractChunkIndex objects.

julia
index = build_index(SimpleIndexer(), texts; chunker_kwargs = (; sources))
index_keywords = ChunkKeywordsIndex(index) # same chunks as above but adds BM25 instead of embeddings

multi_index = MultiIndex([index, index_keywords])

To use airag with different types of indices, we need to specify how to find the closest items for each index

julia
# Cosine similarity for embeddings and BM25 for keywords, same order as indexes in MultiIndex
finder = RT.MultiFinder([RT.CosineSimilarity(), RT.BM25Similarity()])

# Notice that we add `processor` to make sure keywords are processed (ie, tokenized) as well
cfg = RAGConfig(; retriever = SimpleRetriever(; processor = RT.KeywordsProcessor(), finder))

# Ask questions
msg = airag(cfg, multi_index; question = "What are the best practices for parallel computing in Julia?")
pprint(msg) # prettify the answer

source


# PromptingTools.Experimental.RAGTools.NoEmbedder — Type.
julia
NoEmbedder <: AbstractEmbedder

No-op embedder for get_embeddings functions. It returns nothing.

source


# PromptingTools.Experimental.RAGTools.NoPostprocessor — Type.
julia
NoPostprocessor <: AbstractPostprocessor

Default method for postprocess! method. A passthrough option that returns the result without any changes.

Overload this method to add custom postprocessing steps, eg, logging, saving conversations to disk, etc.

source


# PromptingTools.Experimental.RAGTools.NoProcessor — Type.
julia
NoProcessor <: AbstractProcessor

No-op processor for get_keywords functions. It returns the inputs as is.

source


# PromptingTools.Experimental.RAGTools.NoRefiner — Type.
julia
NoRefiner <: AbstractRefiner

Default method for refine! method. A passthrough option that returns the result.answer without any changes.

source


# PromptingTools.Experimental.RAGTools.NoRephraser — Type.
julia
NoRephraser <: AbstractRephraser

No-op implementation for rephrase, which simply passes the question through.

source


# PromptingTools.Experimental.RAGTools.NoReranker — Type.
julia
NoReranker <: AbstractReranker

No-op implementation for rerank, which simply passes the candidate chunks through.

source


# PromptingTools.Experimental.RAGTools.NoTagFilter — Type.
julia
NoTagFilter <: AbstractTagFilter

No-op implementation for find_tags, which simply returns all chunks.

source


# PromptingTools.Experimental.RAGTools.NoTagger — Type.
julia
NoTagger <: AbstractTagger

No-op tagger for get_tags functions. It returns (nothing, nothing).

source


# PromptingTools.Experimental.RAGTools.OpenTagger — Type.
julia
OpenTagger <: AbstractTagger

Tagger for get_tags functions, which generates possible tags for each chunk via aiextract. You can customize it via prompt template (default: :RAGExtractMetadataShort), but it's quite open-ended (ie, AI decides the possible tags).

source


# PromptingTools.Experimental.RAGTools.PassthroughTagger — Type.
julia
PassthroughTagger <: AbstractTagger

Tagger for get_tags functions, which passes tags directly as Vector of Vectors of strings (ie, tags[i] is the tags for docs[i]).

source


# PromptingTools.Experimental.RAGTools.RAGConfig — Type.
julia
RAGConfig <: AbstractRAGConfig

Default configuration for RAG. It uses SimpleIndexer, SimpleRetriever, and SimpleGenerator as default components. Provided as the first argument in airag.

To customize the components, replace corresponding fields for each step of the RAG pipeline (eg, use subtypes(AbstractIndexBuilder) to find the available options).

source


# PromptingTools.Experimental.RAGTools.RAGResult — Type.
julia
RAGResult

A struct for debugging RAG answers. It contains the question, answer, context, and the candidate chunks at each step of the RAG pipeline.

Think of the flow as question -> rephrased_questions -> answer -> final_answer with the context and candidate chunks helping along the way.

Fields

  • question::AbstractString: the original question

  • rephrased_questions::Vector{<:AbstractString}: a vector of rephrased questions (eg, HyDe, Multihop, etc.)

  • answer::AbstractString: the generated answer

  • final_answer::AbstractString: the refined final answer (eg, after CorrectiveRAG), also considered the FINAL answer (it must be always available)

  • context::Vector{<:AbstractString}: the context used for retrieval (ie, the vector of chunks and their surrounding window if applicable)

  • sources::Vector{<:AbstractString}: the sources of the context (for the original matched chunks)

  • emb_candidates::CandidateChunks: the candidate chunks from the embedding index (from find_closest)

  • tag_candidates::Union{Nothing, CandidateChunks}: the candidate chunks from the tag index (from find_tags)

  • filtered_candidates::CandidateChunks: the filtered candidate chunks (intersection of emb_candidates and tag_candidates)

  • reranked_candidates::CandidateChunks: the reranked candidate chunks (from rerank)

  • conversations::Dict{Symbol,Vector{<:AbstractMessage}}: the conversation history for AI steps of the RAG pipeline, use keys that correspond to the function names, eg, :answer or :refine

See also: pprint (pretty printing), annotate_support (for annotating the answer)

source


# PromptingTools.Experimental.RAGTools.RankGPTReranker — Type.
julia
RankGPTReranker <: AbstractReranker

Rerank strategy using the RankGPT algorithm (calling LLMs). A method for rerank.

Reference

[1] Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents by W. Sun et al. [2] RankGPT Github

source


# PromptingTools.Experimental.RAGTools.RankGPTResult — Type.
julia
RankGPTResult

Results from the RankGPT algorithm.

Fields

  • question::String: The question that was asked.

  • chunks::AbstractVector{T}: The chunks that were ranked (=context).

  • positions::Vector{Int}: The ranking of the chunks (referring to the chunks).

  • elapsed::Float64: The time it took to rank the chunks.

  • cost::Float64: The cumulative cost of the ranking.

  • tokens::Int: The cumulative number of tokens used in the ranking.

source


# PromptingTools.Experimental.RAGTools.SimpleAnswerer — Type.
julia
SimpleAnswerer <: AbstractAnswerer

Default method for answer! method. Generates an answer using the aigenerate function with the provided context and question.

source


# PromptingTools.Experimental.RAGTools.SimpleBM25Retriever — Type.
julia
SimpleBM25Retriever <: AbstractRetriever

Keyword-based implementation for retrieve. It does a simple similarity search via BM25Similarity and returns the results.

Make sure to use consistent processor and tagger with the Preparation Stage (build_index)!

Fields

  • rephraser::AbstractRephraser: the rephrasing method, dispatching rephrase - uses NoRephraser

  • embedder::AbstractEmbedder: the embedding method, dispatching get_embeddings (see Preparation Stage for more details) - uses NoEmbedder

  • processor::AbstractProcessor: the processor method, dispatching get_keywords (see Preparation Stage for more details) - uses KeywordsProcessor

  • finder::AbstractSimilarityFinder: the similarity search method, dispatching find_closest - uses CosineSimilarity

  • tagger::AbstractTagger: the tag generating method, dispatching get_tags (see Preparation Stage for more details) - uses NoTagger

  • filter::AbstractTagFilter: the tag matching method, dispatching find_tags - uses NoTagFilter

  • reranker::AbstractReranker: the reranking method, dispatching rerank - uses NoReranker

source


# PromptingTools.Experimental.RAGTools.SimpleGenerator — Type.
julia
SimpleGenerator <: AbstractGenerator

Default implementation for generate. It simply enumerates context snippets and runs aigenerate (no refinement).

It uses ContextEnumerator, SimpleAnswerer, NoRefiner, and NoPostprocessor as default contexter, answerer, refiner, and postprocessor.

source


# PromptingTools.Experimental.RAGTools.SimpleIndexer — Type.
julia
SimpleIndexer <: AbstractIndexBuilder

Default implementation for build_index.

It uses TextChunker, BatchEmbedder, and NoTagger as default chunker, embedder, and tagger.

source


# PromptingTools.Experimental.RAGTools.SimpleRefiner — Type.
julia
SimpleRefiner <: AbstractRefiner

Refines the answer using the same context previously provided via the provided prompt template. A method for refine!.

source


# PromptingTools.Experimental.RAGTools.SimpleRephraser — Type.
julia
SimpleRephraser <: AbstractRephraser

Rephraser implemented using the provided AI Template (eg, ...) and standard chat model. A method for rephrase.

source


# PromptingTools.Experimental.RAGTools.SimpleRetriever — Type.
julia
SimpleRetriever <: AbstractRetriever

Default implementation for retrieve function. It does a simple similarity search via CosineSimilarity and returns the results.

Make sure to use consistent embedder and tagger with the Preparation Stage (build_index)!

Fields

  • rephraser::AbstractRephraser: the rephrasing method, dispatching rephrase - uses NoRephraser

  • embedder::AbstractEmbedder: the embedding method, dispatching get_embeddings (see Preparation Stage for more details) - uses BatchEmbedder

  • processor::AbstractProcessor: the processor method, dispatching get_keywords (see Preparation Stage for more details) - uses NoProcessor

  • finder::AbstractSimilarityFinder: the similarity search method, dispatching find_closest - uses CosineSimilarity

  • tagger::AbstractTagger: the tag generating method, dispatching get_tags (see Preparation Stage for more details) - uses NoTagger

  • filter::AbstractTagFilter: the tag matching method, dispatching find_tags - uses NoTagFilter

  • reranker::AbstractReranker: the reranking method, dispatching rerank - uses NoReranker

source


# PromptingTools.Experimental.RAGTools.Styler — Type.
julia
Styler

Defines styling keywords for printstyled for each AbstractAnnotatedNode

source


# PromptingTools.Experimental.RAGTools.SubChunkIndex — Type.
julia
SubChunkIndex

A view of the parent index with respect to the chunks (and chunk-aligned fields). All methods and accessors working for AbstractChunkIndex also work for SubChunkIndex. It does not yet work for MultiIndex.

Fields

  • parent::AbstractChunkIndex: the parent index from which the chunks are drawn (always the original index, never a view)

  • positions::Vector{Int}: the positions of the chunks in the parent index (always refers to original PARENT index, even if we create a view of the view)

Example

julia
cc = CandidateChunks(index.id, 1:10)
sub_index = @view(index[cc])

You can use SubChunkIndex to access chunks or sources (and other fields) from a parent index, eg,

julia
RT.chunks(sub_index)
RT.sources(sub_index)
RT.chunkdata(sub_index) # slice of embeddings
RT.embeddings(sub_index) # slice of embeddings
RT.tags(sub_index) # slice of tags
RT.tags_vocab(sub_index) # unchanged, identical to parent version
RT.extras(sub_index) # slice of extras

Access the parent index that the positions correspond to

julia
parent(sub_index)
RT.positions(sub_index)

source


# PromptingTools.Experimental.RAGTools.SubDocumentTermMatrix — Type.

A partial view of a DocumentTermMatrix, tf is MATERIALIZED for performance and fewer allocations.

source


# PromptingTools.Experimental.RAGTools.TavilySearchRefiner — Type.
julia
TavilySearchRefiner <: AbstractRefiner

Refines the answer by executing a web search using the Tavily API. This method aims to enhance the answer's accuracy and relevance by incorporating information retrieved from the web. A method for refine!.

source


# PromptingTools.Experimental.RAGTools.TextChunker — Type.
julia
TextChunker <: AbstractChunker

Chunker when you provide text to get_chunks functions. Inputs are directly chunked

source


# PromptingTools.Experimental.RAGTools.TrigramAnnotater — Type.
julia
TrigramAnnotater

Annotation method where we score answer versus each context based on word-level trigrams that match.

It's very simple method (and it can loose some semantic meaning in longer sequences like negative), but it works reasonably well for both text and code.

source


# PromptingTools.Experimental.RAGTools._normalize — Function.

Shortcut to LinearAlgebra.normalize. Provided in the package extension RAGToolsExperimentalExt (Requires SparseArrays, Unicode, and LinearAlgebra)

source


# PromptingTools.Experimental.RAGTools.add_node_metadata! — Method.
julia
add_node_metadata!(annotater::TrigramAnnotater,
    root::AnnotatedNode; add_sources::Bool = true, add_scores::Bool = true,
    sources::Union{Nothing, AbstractVector{<:AbstractString}} = nothing)

Adds metadata to the children of root. Metadata includes sources and scores, if requested.

Optionally, it can add a list of sources at the end of the printed text.

The metadata is added by inserting new nodes in the root children list (with no children of its own to be printed out).

source


# PromptingTools.Experimental.RAGTools.airag — Method.
julia
airag(cfg::AbstractRAGConfig, index::AbstractDocumentIndex;
    question::AbstractString,
    verbose::Integer = 1, return_all::Bool = false,
    api_kwargs::NamedTuple = NamedTuple(),
    retriever::AbstractRetriever = cfg.retriever,
    retriever_kwargs::NamedTuple = NamedTuple(),
    generator::AbstractGenerator = cfg.generator,
    generator_kwargs::NamedTuple = NamedTuple(),
    cost_tracker = Threads.Atomic{Float64}(0.0))

High-level wrapper for Retrieval-Augmented Generation (RAG), it combines together the retrieve and generate! steps which you can customize if needed.

The simplest version first finds the relevant chunks in index for the question and then sends these chunks to the AI model to help with generating a response to the question.

To customize the components, replace the types (retriever, generator) of the corresponding step of the RAG pipeline - or go into sub-routines within the steps. Eg, use subtypes(AbstractRetriever) to find the available options.

Arguments

  • cfg::AbstractRAGConfig: The configuration for the RAG pipeline. Defaults to RAGConfig(), where you can swap sub-types to customize the pipeline.

  • index::AbstractDocumentIndex: The chunk index to search for relevant text.

  • question::AbstractString: The question to be answered.

  • return_all::Bool: If true, returns the details used for RAG along with the response.

  • verbose::Integer: If >0, enables verbose logging. The higher the number, the more nested functions will log.

  • api_kwargs: API parameters that will be forwarded to ALL of the API calls (aiembed, aigenerate, and aiextract).

  • retriever::AbstractRetriever: The retriever to use for finding relevant chunks. Defaults to cfg.retriever, eg, SimpleRetriever (with no question rephrasing).

  • retriever_kwargs::NamedTuple: API parameters that will be forwarded to the retriever call. Examples of important ones:

    • top_k::Int: Number of top candidates to retrieve based on embedding similarity.

    • top_n::Int: Number of candidates to return after reranking.

    • tagger::AbstractTagger: Tagger to use for tagging the chunks. Defaults to NoTagger().

    • tagger_kwargs::NamedTuple: API parameters that will be forwarded to the tagger call. You could provide the explicit tags directly with PassthroughTagger and tagger_kwargs = (; tags = ["tag1", "tag2"]).

  • generator::AbstractGenerator: The generator to use for generating the answer. Defaults to cfg.generator, eg, SimpleGenerator.

  • generator_kwargs::NamedTuple: API parameters that will be forwarded to the generator call. Examples of important ones:

    • answerer_kwargs::NamedTuple: API parameters that will be forwarded to the answerer call. Examples:

      • model: The model to use for generating the answer. Defaults to PT.MODEL_CHAT.

      • template: The template to use for the aigenerate function. Defaults to :RAGAnswerFromContext.

    • refiner::AbstractRefiner: The method to use for refining the answer. Defaults to generator.refiner, eg, NoRefiner.

    • refiner_kwargs::NamedTuple: API parameters that will be forwarded to the refiner call.

      • model: The model to use for generating the answer. Defaults to PT.MODEL_CHAT.

      • template: The template to use for the aigenerate function. Defaults to :RAGAnswerRefiner.

  • cost_tracker: An atomic counter to track the total cost of the operations (if you want to track the cost of multiple pipeline runs - it passed around in the pipeline).

Returns

  • If return_all is false, returns the generated message (msg).

  • If return_all is true, returns the detail of the full pipeline in RAGResult (see the docs).

See also build_index, retrieve, generate!, RAGResult, getpropertynested, setpropertynested, merge_kwargs_nested, ChunkKeywordsIndex.

Examples

Using airag to get a response for a question:

julia
index = build_index(...)  # create an index
question = "How to make a barplot in Makie.jl?"
msg = airag(index; question)

To understand the details of the RAG process, use return_all=true

julia
msg, details = airag(index; question, return_all = true)
# details is a RAGDetails object with all the internal steps of the `airag` function

You can also pretty-print details to highlight generated text vs text that is supported by context. It also includes annotations of which context was used for each part of the response (where available).

julia
PT.pprint(details)

Example with advanced retrieval (with question rephrasing and reranking (requires COHERE_API_KEY). We will obtain top 100 chunks from embeddings (top_k) and top 5 chunks from reranking (top_n). In addition, it will be done with a "custom" locally-hosted model.

julia
cfg = RAGConfig(; retriever = AdvancedRetriever())

# kwargs will be big and nested, let's prepare them upfront
# we specify "custom" model for each component that calls LLM
kwargs = (
    retriever_kwargs = (;
        top_k = 100,
        top_n = 5,
        rephraser_kwargs = (;
            model = "custom"),
        embedder_kwargs = (;
            model = "custom"),
        tagger_kwargs = (;
            model = "custom")),
    generator_kwargs = (;
        answerer_kwargs = (;
            model = "custom"),
        refiner_kwargs = (;
            model = "custom")),
    api_kwargs = (;
        url = "http://localhost:8080"))

result = airag(cfg, index, question; kwargs...)

If you want to use hybrid retrieval (embeddings + BM25), you can easily create an additional index based on keywords and pass them both into a MultiIndex.

You need to provide an explicit config, so the pipeline knows how to handle each index in the search similarity phase (finder).

julia
index = # your existing index

# create the multi-index with the keywords index
index_keywords = ChunkKeywordsIndex(index)
multi_index = MultiIndex([index, index_keywords])

# define the similarity measures for the indices that you have (same order)
finder = RT.MultiFinder([RT.CosineSimilarity(), RT.BM25Similarity()])
cfg = RAGConfig(; retriever=AdvancedRetriever(; processor=RT.KeywordsProcessor(), finder))

# Run the pipeline with the new hybrid retrieval (return the `RAGResult` to see the details)
result = airag(cfg, multi_index; question, return_all=true)

# Pretty-print the result
PT.pprint(result)

For easier manipulation of nested kwargs, see utilities getpropertynested, setpropertynested, merge_kwargs_nested.

source


# PromptingTools.Experimental.RAGTools.align_node_styles! — Method.
julia
align_node_styles!(annotater::TrigramAnnotater, nodes::AbstractVector{<:AnnotatedNode}; kwargs...)

Aligns the styles of the nodes based on the surrounding nodes ("fill-in-the-middle").

If the node has no score, but the surrounding nodes have the same style, the node will inherit the style of the surrounding nodes.

source


# PromptingTools.Experimental.RAGTools.annotate_support — Method.
julia
annotate_support(annotater::TrigramAnnotater, answer::AbstractString,
    context::AbstractVector; min_score::Float64 = 0.5,
    skip_trigrams::Bool = true, hashed::Bool = true,
    sources::Union{Nothing, AbstractVector{<:AbstractString}} = nothing,
    min_source_score::Float64 = 0.25,
    add_sources::Bool = true,
    add_scores::Bool = true, kwargs...)

Annotates the answer with the overlap/what's supported in context and returns the annotated tree of nodes representing the answer

Returns a "root" node with children nodes representing the sentences/code blocks in the answer. Only the "leaf" nodes are to be printed (to avoid duplication), "leaf" nodes are those with NO children.

Default logic:

  • Split into sentences/code blocks, then into tokens (~words).

  • Then match each token (~word) exactly.

  • If no exact match found, count trigram-based match (include the surrounding tokens for better contextual awareness).

  • If the match is higher than min_score, it's recorded in the score of the node.

Arguments

  • annotater::TrigramAnnotater: Annotater to use

  • answer::AbstractString: Text to annotate

  • context::AbstractVector: Context to annotate against, ie, look for "support" in the texts in context

  • min_score::Float64: Minimum score to consider a match. Default: 0.5, which means that half of the trigrams of each word should match

  • skip_trigrams::Bool: Whether to potentially skip trigram matching if exact full match is found. Default: true

  • hashed::Bool: Whether to use hashed trigrams. It's harder to debug, but it's much faster for larger texts (hashed text are held in a Set to deduplicate). Default: true

  • sources::Union{Nothing, AbstractVector{<:AbstractString}}: Sources to add at the end of the context. Default: nothing

  • min_source_score::Float64: Minimum score to consider/to display a source. Default: 0.25, which means that at least a quarter of the trigrams of each word should match to some context. The threshold is lower than min_score, because it's average across ALL words in a block, so it's much harder to match fully with generated text.

  • add_sources::Bool: Whether to add sources at the end of each code block/sentence. Sources are addded in the square brackets like "[1]". Default: true

  • add_scores::Bool: Whether to add source-matching scores at the end of each code block/sentence. Scores are added in the square brackets like "[0.75]". Default: true

  • kwargs: Additional keyword arguments to pass to trigram_support! and set_node_style!. See their documentation for more details (eg, customize the colors of the nodes based on the score)

Example

julia
annotater = TrigramAnnotater()
context = [
    "This is a test context.", "Another context sentence.", "Final piece of context."]
answer = "This is a test context. Another context sentence."

annotated_root = annotate_support(annotater, answer, context)
pprint(annotated_root) # pretty print the annotated tree

source


# PromptingTools.Experimental.RAGTools.annotate_support — Method.
julia
annotate_support(
    annotater::TrigramAnnotater, result::AbstractRAGResult; min_score::Float64 = 0.5,
    skip_trigrams::Bool = true, hashed::Bool = true,
    min_source_score::Float64 = 0.25,
    add_sources::Bool = true,
    add_scores::Bool = true, kwargs...)

Dispatch for annotate_support for AbstractRAGResult type. It extracts the final_answer and context from the result and calls annotate_support with them.

See annotate_support for more details.

Example

julia
res = RAGResult(; question = "", final_answer = "This is a test.",
    context = ["Test context.", "Completely different"])
annotated_root = annotate_support(annotater, res)
PT.pprint(annotated_root)

source


# PromptingTools.Experimental.RAGTools.answer! — Method.
julia
answer!(
    answerer::SimpleAnswerer, index::AbstractDocumentIndex, result::AbstractRAGResult;
    model::AbstractString = PT.MODEL_CHAT, verbose::Bool = true,
    template::Symbol = :RAGAnswerFromContext,
    cost_tracker = Threads.Atomic{Float64}(0.0),
    kwargs...)

Generates an answer using the aigenerate function with the provided result.context and result.question.

Returns

  • Mutated result with result.answer and the full conversation saved in result.conversations[:answer]

Arguments

  • answerer::SimpleAnswerer: The method to use for generating the answer. Uses aigenerate.

  • index::AbstractDocumentIndex: The index containing chunks and sources.

  • result::AbstractRAGResult: The result containing the context and question to generate the answer for.

  • model::AbstractString: The model to use for generating the answer. Defaults to PT.MODEL_CHAT.

  • verbose::Bool: If true, enables verbose logging.

  • template::Symbol: The template to use for the aigenerate function. Defaults to :RAGAnswerFromContext.

  • cost_tracker: An atomic counter to track the cost of the operation.

source


# PromptingTools.Experimental.RAGTools.build_context — Method.
julia
build_context(contexter::ContextEnumerator,
    index::AbstractDocumentIndex, candidates::AbstractCandidateChunks;
    verbose::Bool = true,
    chunks_window_margin::Tuple{Int, Int} = (1, 1), kwargs...)

    build_context!(contexter::ContextEnumerator,
    index::AbstractDocumentIndex, result::AbstractRAGResult; kwargs...)

Build context strings for each position in candidates considering a window margin around each position. If mutating version is used (build_context!), it will use result.reranked_candidates to update the result.context field.

Arguments

  • contexter::ContextEnumerator: The method to use for building the context. Enumerates the snippets.

  • index::AbstractDocumentIndex: The index containing chunks and sources.

  • candidates::AbstractCandidateChunks: Candidate chunks which contain positions to extract context from.

  • verbose::Bool: If true, enables verbose logging.

  • chunks_window_margin::Tuple{Int, Int}: A tuple indicating the margin (before, after) around each position to include in the context. Defaults to (1,1), which means 1 preceding and 1 suceeding chunk will be included. With (0,0), only the matching chunks will be included.

Returns

  • Vector{String}: A vector of context strings, each corresponding to a position in reranked_candidates.

Examples

julia
index = ChunkIndex(...)  # Assuming a proper index is defined
candidates = CandidateChunks(index.id, [2, 4], [0.1, 0.2])
context = build_context(ContextEnumerator(), index, candidates; chunks_window_margin=(0, 1)) # include only one following chunk for each matching chunk

source


# PromptingTools.Experimental.RAGTools.build_index — Method.
julia
build_index(
    indexer::KeywordsIndexer, files_or_docs::Vector{<:AbstractString};
    verbose::Integer = 1,
    extras::Union{Nothing, AbstractVector} = nothing,
    index_id = gensym("ChunkKeywordsIndex"),
    chunker::AbstractChunker = indexer.chunker,
    chunker_kwargs::NamedTuple = NamedTuple(),
    processor::AbstractProcessor = indexer.processor,
    processor_kwargs::NamedTuple = NamedTuple(),
    tagger::AbstractTagger = indexer.tagger,
    tagger_kwargs::NamedTuple = NamedTuple(),
    api_kwargs::NamedTuple = NamedTuple(),
    cost_tracker = Threads.Atomic{Float64}(0.0))

Builds a ChunkKeywordsIndex from the provided files or documents to support keyword-based search (BM25).

source


# PromptingTools.Experimental.RAGTools.build_index — Method.
julia
build_index(
    indexer::AbstractIndexBuilder, files_or_docs::Vector{<:AbstractString};
    verbose::Integer = 1,
    extras::Union{Nothing, AbstractVector} = nothing,
    index_id = gensym("ChunkEmbeddingsIndex"),
    chunker::AbstractChunker = indexer.chunker,
    chunker_kwargs::NamedTuple = NamedTuple(),
    embedder::AbstractEmbedder = indexer.embedder,
    embedder_kwargs::NamedTuple = NamedTuple(),
    tagger::AbstractTagger = indexer.tagger,
    tagger_kwargs::NamedTuple = NamedTuple(),
    api_kwargs::NamedTuple = NamedTuple(),
    cost_tracker = Threads.Atomic{Float64}(0.0))

Build an INDEX for RAG (Retriever-Augmented Generation) applications from the provided file paths. INDEX is a object storing the document chunks and their embeddings (and potentially other information).

The function processes each file or document (depending on chunker), splits its content into chunks, embeds these chunks, optionally extracts metadata, and then combines this information into a retrievable index.

Define your own methods via indexer and its subcomponents (chunker, embedder, tagger).

Arguments

  • indexer::AbstractIndexBuilder: The indexing logic to use. Default is SimpleIndexer().

  • files_or_docs: A vector of valid file paths OR string documents to be indexed (chunked and embedded). Specify which mode to use via chunker.

  • verbose: An Integer specifying the verbosity of the logs. Default is 1 (high-level logging). 0 is disabled.

  • extras: An optional vector of extra information to be stored with each chunk. Default is nothing.

  • index_id: A unique identifier for the index. Default is a generated symbol.

  • chunker: The chunker logic to use for splitting the documents. Default is TextChunker().

  • chunker_kwargs: Parameters to be provided to the get_chunks function. Useful to change the separators or max_length.

    • sources: A vector of strings indicating the source of each chunk. Default is equal to files_or_docs.

  • embedder: The embedder logic to use for embedding the chunks. Default is BatchEmbedder().

  • embedder_kwargs: Parameters to be provided to the get_embeddings function. Useful to change the target_batch_size_length or reduce asyncmap tasks ntasks.

    • model: The model to use for embedding. Default is PT.MODEL_EMBEDDING.

  • tagger: The tagger logic to use for extracting tags from the chunks. Default is NoTagger(), ie, skip tag extraction. There are also PassthroughTagger and OpenTagger.

  • tagger_kwargs: Parameters to be provided to the get_tags function.

    • model: The model to use for tags extraction. Default is PT.MODEL_CHAT.

    • template: A template to be used for tags extraction. Default is :RAGExtractMetadataShort.

    • tags: A vector of vectors of strings directly providing the tags for each chunk. Applicable for tagger::PasstroughTagger.

  • api_kwargs: Parameters to be provided to the API endpoint. Shared across all API calls if provided.

  • cost_tracker: A Threads.Atomic{Float64} object to track the total cost of the API calls. Useful to pass the total cost to the parent call.

Returns

  • ChunkEmbeddingsIndex: An object containing the compiled index of chunks, embeddings, tags, vocabulary, and sources.

See also: ChunkEmbeddingsIndex, get_chunks, get_embeddings, get_tags, CandidateChunks, find_closest, find_tags, rerank, retrieve, generate!, airag

Examples

julia
# Default is loading a vector of strings and chunking them (`TextChunker()`)
index = build_index(SimpleIndexer(), texts; chunker_kwargs = (; max_length=10))

# Another example with tags extraction, splitting only sentences and verbose output
# Assuming `test_files` is a vector of file paths
indexer = SimpleIndexer(chunker=FileChunker(), tagger=OpenTagger())
index = build_index(indexer, test_files; 
        chunker_kwargs(; separators=[". "]), verbose=true)

Notes

  • If you get errors about exceeding embedding input sizes, first check the max_length in your chunks. If that does NOT resolve the issue, try changing the embedding_kwargs. In particular, reducing the target_batch_size_length parameter (eg, 10_000) and number of tasks ntasks=1. Some providers cannot handle large batch sizes (eg, Databricks).

source


# PromptingTools.Experimental.RAGTools.build_qa_evals — Method.
julia
build_qa_evals(doc_chunks::Vector{<:AbstractString}, sources::Vector{<:AbstractString};
               model=PT.MODEL_CHAT, instructions="None.", qa_template::Symbol=:RAGCreateQAFromContext, 
               verbose::Bool=true, api_kwargs::NamedTuple = NamedTuple(), kwargs...) -> Vector{QAEvalItem}

Create a collection of question and answer evaluations (QAEvalItem) from document chunks and sources. This function generates Q&A pairs based on the provided document chunks, using a specified AI model and template.

Arguments

  • doc_chunks::Vector{<:AbstractString}: A vector of document chunks, each representing a segment of text.

  • sources::Vector{<:AbstractString}: A vector of source identifiers corresponding to each chunk in doc_chunks (eg, filenames or paths).

  • model: The AI model used for generating Q&A pairs. Default is PT.MODEL_CHAT.

  • instructions::String: Additional instructions or context to provide to the model generating QA sets. Defaults to "None.".

  • qa_template::Symbol: A template symbol that dictates the AITemplate that will be used. It must have placeholder context. Default is :CreateQAFromContext.

  • api_kwargs::NamedTuple: Parameters that will be forwarded to the API endpoint.

  • verbose::Bool: If true, additional information like costs will be logged. Defaults to true.

Returns

Vector{QAEvalItem}: A vector of QAEvalItem structs, each containing a source, context, question, and answer. Invalid or empty items are filtered out.

Notes

  • The function internally uses aiextract to generate Q&A pairs based on the provided qa_template. So you can use any kwargs that you want.

  • Each QAEvalItem includes the context (document chunk), the generated question and answer, and the source.

  • The function tracks and reports the cost of AI calls if verbose is enabled.

  • Items where the question, answer, or context is empty are considered invalid and are filtered out.

Examples

Creating Q&A evaluations from a set of document chunks:

julia
doc_chunks = ["Text from document 1", "Text from document 2"]
sources = ["source1", "source2"]
qa_evals = build_qa_evals(doc_chunks, sources)

source


# PromptingTools.Experimental.RAGTools.build_tags — Function.

Builds a matrix of tags and a vocabulary list. REQUIRES SparseArrays, LinearAlgebra, Unicode packages to be loaded!!

source


# PromptingTools.Experimental.RAGTools.build_tags — Method.
julia
build_tags(tagger::AbstractTagger, chunk_tags::Nothing; kwargs...)

No-op that skips any tag building, returning nothing, nothing

Otherwise, it would build the sparse matrix and the vocabulary (requires SparseArrays and LinearAlgebra packages to be loaded).

source


# PromptingTools.Experimental.RAGTools.chunkdata — Method.

Access chunkdata for a subset of chunks, chunk_idx is a vector of chunk indices in the index

source


# PromptingTools.Experimental.RAGTools.chunkdata — Method.

Access chunkdata for a subset of chunks, chunk_idx is a vector of chunk indices in the index

source


# PromptingTools.Experimental.RAGTools.chunkdata — Method.

Access chunkdata for a subset of chunks, chunk_idx is a vector of chunk indices in the index

source


# PromptingTools.Experimental.RAGTools.cohere_api — Method.
julia
cohere_api(;
api_key::AbstractString,
endpoint::String,
url::AbstractString="https://api.cohere.ai/v1",
http_kwargs::NamedTuple=NamedTuple(),
kwargs...)

Lightweight wrapper around the Cohere API. See https://cohere.com/docs for more details.

Arguments

  • api_key: Your Cohere API key. You can get one from https://dashboard.cohere.com/welcome/register (trial access is for free).

  • endpoint: The Cohere endpoint to call.

  • url: The base URL for the Cohere API. Default is https://api.cohere.ai/v1.

  • http_kwargs: Any additional keyword arguments to pass to HTTP.post.

  • kwargs: Any additional keyword arguments to pass to the Cohere API.

source


# PromptingTools.Experimental.RAGTools.create_permutation_instruction — Method.
julia
create_permutation_instruction(
    context::AbstractVector{<:AbstractString}; rank_start::Integer = 1,
    rank_end::Integer = 100, max_length::Integer = 512, template::Symbol = :RAGRankGPT)

Creates rendered template with injected context passages.

source


# PromptingTools.Experimental.RAGTools.extract_ranking — Method.
julia
extract_ranking(str::AbstractString)

Extracts the ranking from the response into a sorted array of integers.

source


# PromptingTools.Experimental.RAGTools.find_closest — Function.
julia
find_closest(
    finder::BM25Similarity, dtm::AbstractDocumentTermMatrix,
    query_emb::AbstractVector{<:Real}, query_tokens::AbstractVector{<:AbstractString} = String[];
    top_k::Int = 100, minimum_similarity::AbstractFloat = -1.0, kwargs...)

Finds the indices of chunks (represented by DocumentTermMatrix in dtm) that are closest to query tokens (query_tokens) using BM25.

Reference: Wikipedia: BM25. Implementation follows: The Next Generation of Lucene Relevance.

source


# PromptingTools.Experimental.RAGTools.find_closest — Function.
julia
find_closest(
    finder::BinaryCosineSimilarity, emb::AbstractMatrix{<:Bool},
    query_emb::AbstractVector{<:Real}, query_tokens::AbstractVector{<:AbstractString} = String[];
    top_k::Int = 100, rescore_multiplier::Int = 4, minimum_similarity::AbstractFloat = -1.0, kwargs...)

Finds the indices of chunks (represented by embeddings in emb) that are closest to query embedding (query_emb) using binary embeddings (in the index).

This is a two-pass approach:

  • First pass: Hamming distance in binary form to get the top_k * rescore_multiplier (ie, more than top_k) candidates.

  • Second pass: Rescore the candidates with float embeddings and return the top_k.

Returns only top_k closest indices.

Reference: HuggingFace: Embedding Quantization.

Examples

Convert any Float embeddings to binary like this:

julia
binary_emb = map(>(0), emb)

source


# PromptingTools.Experimental.RAGTools.find_closest — Function.
julia
find_closest(
    finder::AbstractSimilarityFinder, index::AbstractChunkIndex,
    query_emb::AbstractVector{<:Real}, query_tokens::AbstractVector{<:AbstractString} = String[];
    top_k::Int = 100, kwargs...)

Finds the indices of chunks (represented by embeddings in index) that are closest to query embedding (query_emb).

Returns only top_k closest indices.

source


# PromptingTools.Experimental.RAGTools.find_closest — Function.
julia
find_closest(
    finder::BitPackedCosineSimilarity, emb::AbstractMatrix{<:Bool},
    query_emb::AbstractVector{<:Real}, query_tokens::AbstractVector{<:AbstractString} = String[];
    top_k::Int = 100, rescore_multiplier::Int = 4, minimum_similarity::AbstractFloat = -1.0, kwargs...)

Finds the indices of chunks (represented by embeddings in emb) that are closest to query embedding (query_emb) using bit-packed binary embeddings (in the index).

This is a two-pass approach:

  • First pass: Hamming distance in bit-packed binary form to get the top_k * rescore_multiplier (i.e., more than top_k) candidates.

  • Second pass: Rescore the candidates with float embeddings and return the top_k.

Returns only top_k closest indices.

Reference: HuggingFace: Embedding Quantization.

Examples

Convert any Float embeddings to bit-packed binary like this:

julia
bitpacked_emb = pack_bits(emb.>0)

source


# PromptingTools.Experimental.RAGTools.find_closest — Function.
julia
find_closest(
    finder::CosineSimilarity, emb::AbstractMatrix{<:Real},
    query_emb::AbstractVector{<:Real}, query_tokens::AbstractVector{<:AbstractString} = String[];
    top_k::Int = 100, minimum_similarity::AbstractFloat = -1.0, kwargs...)

Finds the indices of chunks (represented by embeddings in emb) that are closest (in cosine similarity for CosineSimilarity()) to query embedding (query_emb).

finder is the logic used for the similarity search. Default is CosineSimilarity.

If minimum_similarity is provided, only indices with similarity greater than or equal to it are returned. Similarity can be between -1 and 1 (-1 = completely opposite, 1 = exactly the same).

Returns only top_k closest indices.

source


# PromptingTools.Experimental.RAGTools.find_tags — Method.
julia
find_tags(method::AnyTagFilter, index::AbstractChunkIndex,
    tag::Union{AbstractString, Regex}; kwargs...)

find_tags(method::AnyTagFilter, index::AbstractChunkIndex,
    tags::Vector{T}; kwargs...) where {T <: Union{AbstractString, Regex}}

Finds the indices of chunks (represented by tags in index) that have ANY OF the specified tag or tags.

source


# PromptingTools.Experimental.RAGTools.find_tags — Method.
julia
find_tags(method::AllTagFilter, index::AbstractChunkIndex,
    tag::Union{AbstractString, Regex}; kwargs...)

find_tags(method::AllTagFilter, index::AbstractChunkIndex,
    tags::Vector{T}; kwargs...) where {T <: Union{AbstractString, Regex}}

Finds the indices of chunks (represented by tags in index) that have ALL OF the specified tag or tags.

source


# PromptingTools.Experimental.RAGTools.find_tags — Method.
julia
find_tags(method::NoTagFilter, index::AbstractChunkIndex,
    tags::Union{T, AbstractVector{<:T}}; kwargs...) where {T <:
                                                           Union{
    AbstractString, Regex, Nothing}}
    tags; kwargs...)

Returns all chunks in the index, ie, no filtering, so we simply return nothing (easier for dispatch).

source


# PromptingTools.Experimental.RAGTools.generate! — Method.
julia
generate!(
    generator::AbstractGenerator, index::AbstractDocumentIndex, result::AbstractRAGResult;
    verbose::Integer = 1,
    api_kwargs::NamedTuple = NamedTuple(),
    contexter::AbstractContextBuilder = generator.contexter,
    contexter_kwargs::NamedTuple = NamedTuple(),
    answerer::AbstractAnswerer = generator.answerer,
    answerer_kwargs::NamedTuple = NamedTuple(),
    refiner::AbstractRefiner = generator.refiner,
    refiner_kwargs::NamedTuple = NamedTuple(),
    postprocessor::AbstractPostprocessor = generator.postprocessor,
    postprocessor_kwargs::NamedTuple = NamedTuple(),
    cost_tracker = Threads.Atomic{Float64}(0.0),
    kwargs...)

Generate the response using the provided generator and the index and result. It is the second step in the RAG pipeline (after retrieve)

Returns the mutated result with the result.final_answer and the full conversation saved in result.conversations[:final_answer].

Notes

  • The default flow is build_context! -> answer! -> refine! -> postprocess!.

  • contexter is the method to use for building the context, eg, simply enumerate the context chunks with ContextEnumerator.

  • answerer is the standard answer generation step with LLMs.

  • refiner step allows the LLM to critique itself and refine its own answer.

  • postprocessor step allows for additional processing of the answer, eg, logging, saving conversations, etc.

  • All of its sub-routines operate by mutating the result object (and adding their part).

  • Discover available sub-types for each step with subtypes(AbstractRefiner) and similar for other abstract types.

Arguments

  • generator::AbstractGenerator: The generator to use for generating the answer. Can be SimpleGenerator or AdvancedGenerator.

  • index::AbstractDocumentIndex: The index containing chunks and sources.

  • result::AbstractRAGResult: The result containing the context and question to generate the answer for.

  • verbose::Integer: If >0, enables verbose logging.

  • api_kwargs::NamedTuple: API parameters that will be forwarded to ALL of the API calls (aiembed, aigenerate, and aiextract).

  • contexter::AbstractContextBuilder: The method to use for building the context. Defaults to generator.contexter, eg, ContextEnumerator.

  • contexter_kwargs::NamedTuple: API parameters that will be forwarded to the contexter call.

  • answerer::AbstractAnswerer: The method to use for generating the answer. Defaults to generator.answerer, eg, SimpleAnswerer.

  • answerer_kwargs::NamedTuple: API parameters that will be forwarded to the answerer call. Examples:

    • model: The model to use for generating the answer. Defaults to PT.MODEL_CHAT.

    • template: The template to use for the aigenerate function. Defaults to :RAGAnswerFromContext.

  • refiner::AbstractRefiner: The method to use for refining the answer. Defaults to generator.refiner, eg, NoRefiner.

  • refiner_kwargs::NamedTuple: API parameters that will be forwarded to the refiner call.

    • model: The model to use for generating the answer. Defaults to PT.MODEL_CHAT.

    • template: The template to use for the aigenerate function. Defaults to :RAGAnswerRefiner.

  • postprocessor::AbstractPostprocessor: The method to use for postprocessing the answer. Defaults to generator.postprocessor, eg, NoPostprocessor.

  • postprocessor_kwargs::NamedTuple: API parameters that will be forwarded to the postprocessor call.

  • cost_tracker: An atomic counter to track the total cost of the operations.

See also: retrieve, build_context!, ContextEnumerator, answer!, SimpleAnswerer, refine!, NoRefiner, SimpleRefiner, postprocess!, NoPostprocessor

Examples

julia
Assume we already have `index`

question = "What are the best practices for parallel computing in Julia?"

# Retrieve the relevant chunks - returns RAGResult
result = retrieve(index, question)

# Generate the answer using the default generator, mutates the same result
result = generate!(index, result)

source


# PromptingTools.Experimental.RAGTools.get_chunks — Method.
julia
get_chunks(chunker::AbstractChunker,
    files_or_docs::Vector{<:AbstractString};
    sources::AbstractVector{<:AbstractString} = files_or_docs,
    verbose::Bool = true,
    separators = ["\n\n", ". ", "\n", " "], max_length::Int = 256)

Chunks the provided files_or_docs into chunks of maximum length max_length (if possible with provided separators).

Supports two modes of operation:

  • chunker = FileChunker(): The function opens each file in files_or_docs and reads its contents.

  • chunker = TextChunker(): The function assumes that files_or_docs is a vector of strings to be chunked, you MUST provide corresponding sources.

Arguments

  • files_or_docs: A vector of valid file paths OR string documents to be chunked.

  • separators: A list of strings used as separators for splitting the text in each file into chunks. Default is [\n\n", ". ", "\n", " "]. See recursive_splitter for more details.

  • max_length: The maximum length of each chunk (if possible with provided separators). Default is 256.

  • sources: A vector of strings indicating the source of each chunk. Default is equal to files_or_docs (for reader=:files)

source


# PromptingTools.Experimental.RAGTools.get_embeddings — Method.
julia
get_embeddings(embedder::BatchEmbedder, docs::AbstractVector{<:AbstractString};
    verbose::Bool = true,
    model::AbstractString = PT.MODEL_EMBEDDING,
    truncate_dimension::Union{Int, Nothing} = nothing,
    cost_tracker = Threads.Atomic{Float64}(0.0),
    target_batch_size_length::Int = 80_000,
    ntasks::Int = 4 * Threads.nthreads(),
    kwargs...)

Embeds a vector of docs using the provided model (kwarg model) in a batched manner - BatchEmbedder.

BatchEmbedder tries to batch embedding calls for roughly 80K characters per call (to avoid exceeding the API rate limit) to reduce network latency.

Notes

  • docs are assumed to be already chunked to the reasonable sizes that fit within the embedding context limit.

  • If you get errors about exceeding input sizes, first check the max_length in your chunks. If that does NOT resolve the issue, try reducing the target_batch_size_length parameter (eg, 10_000) and number of tasks ntasks=1. Some providers cannot handle large batch sizes.

Arguments

  • docs: A vector of strings to be embedded.

  • verbose: A boolean flag for verbose output. Default is true.

  • model: The model to use for embedding. Default is PT.MODEL_EMBEDDING.

  • truncate_dimension: The dimensionality of the embeddings to truncate to. Default is nothing, 0 will also do nothing.

  • cost_tracker: A Threads.Atomic{Float64} object to track the total cost of the API calls. Useful to pass the total cost to the parent call.

  • target_batch_size_length: The target length (in characters) of each batch of document chunks sent for embedding. Default is 80_000 characters. Speeds up embedding process.

  • ntasks: The number of tasks to use for asyncmap. Default is 4 * Threads.nthreads().

source


# PromptingTools.Experimental.RAGTools.get_embeddings — Method.
julia
get_embeddings(embedder::BinaryBatchEmbedder, docs::AbstractVector{<:AbstractString};
    verbose::Bool = true,
    model::AbstractString = PT.MODEL_EMBEDDING,
    truncate_dimension::Union{Int, Nothing} = nothing,
    return_type::Type = Matrix{Bool},
    cost_tracker = Threads.Atomic{Float64}(0.0),
    target_batch_size_length::Int = 80_000,
    ntasks::Int = 4 * Threads.nthreads(),
    kwargs...)

Embeds a vector of docs using the provided model (kwarg model) in a batched manner and then returns the binary embeddings matrix - BinaryBatchEmbedder.

BinaryBatchEmbedder tries to batch embedding calls for roughly 80K characters per call (to avoid exceeding the API rate limit) to reduce network latency.

Notes

  • docs are assumed to be already chunked to the reasonable sizes that fit within the embedding context limit.

  • If you get errors about exceeding input sizes, first check the max_length in your chunks. If that does NOT resolve the issue, try reducing the target_batch_size_length parameter (eg, 10_000) and number of tasks ntasks=1. Some providers cannot handle large batch sizes.

Arguments

  • docs: A vector of strings to be embedded.

  • verbose: A boolean flag for verbose output. Default is true.

  • model: The model to use for embedding. Default is PT.MODEL_EMBEDDING.

  • truncate_dimension: The dimensionality of the embeddings to truncate to. Default is nothing.

  • return_type: The type of the returned embeddings matrix. Default is Matrix{Bool}. Choose BitMatrix to minimize storage requirements, Matrix{Bool} to maximize performance in elementwise-ops.

  • cost_tracker: A Threads.Atomic{Float64} object to track the total cost of the API calls. Useful to pass the total cost to the parent call.

  • target_batch_size_length: The target length (in characters) of each batch of document chunks sent for embedding. Default is 80_000 characters. Speeds up embedding process.

  • ntasks: The number of tasks to use for asyncmap. Default is 4 * Threads.nthreads().

source


# PromptingTools.Experimental.RAGTools.get_embeddings — Method.
julia
get_embeddings(embedder::BitPackedBatchEmbedder, docs::AbstractVector{<:AbstractString};
    verbose::Bool = true,
    model::AbstractString = PT.MODEL_EMBEDDING,
    truncate_dimension::Union{Int, Nothing} = nothing,
    cost_tracker = Threads.Atomic{Float64}(0.0),
    target_batch_size_length::Int = 80_000,
    ntasks::Int = 4 * Threads.nthreads(),
    kwargs...)

Embeds a vector of docs using the provided model (kwarg model) in a batched manner and then returns the binary embeddings matrix represented in UInt64 (bit-packed) - BitPackedBatchEmbedder.

BitPackedBatchEmbedder tries to batch embedding calls for roughly 80K characters per call (to avoid exceeding the API rate limit) to reduce network latency.

The best option for FAST and MEMORY-EFFICIENT storage of embeddings, for retrieval use BitPackedCosineSimilarity.

Notes

  • docs are assumed to be already chunked to the reasonable sizes that fit within the embedding context limit.

  • If you get errors about exceeding input sizes, first check the max_length in your chunks. If that does NOT resolve the issue, try reducing the target_batch_size_length parameter (eg, 10_000) and number of tasks ntasks=1. Some providers cannot handle large batch sizes.

Arguments

  • docs: A vector of strings to be embedded.

  • verbose: A boolean flag for verbose output. Default is true.

  • model: The model to use for embedding. Default is PT.MODEL_EMBEDDING.

  • truncate_dimension: The dimensionality of the embeddings to truncate to. Default is nothing.

  • cost_tracker: A Threads.Atomic{Float64} object to track the total cost of the API calls. Useful to pass the total cost to the parent call.

  • target_batch_size_length: The target length (in characters) of each batch of document chunks sent for embedding. Default is 80_000 characters. Speeds up embedding process.

  • ntasks: The number of tasks to use for asyncmap. Default is 4 * Threads.nthreads().

See also: unpack_bits, pack_bits, BitPackedCosineSimilarity.

source


# PromptingTools.Experimental.RAGTools.get_tags — Method.
julia
get_tags(tagger::NoTagger, docs::AbstractVector{<:AbstractString};
    kwargs...)

Simple no-op that skips any tagging of the documents

source


# PromptingTools.Experimental.RAGTools.get_tags — Method.
julia
get_tags(tagger::OpenTagger, docs::AbstractVector{<:AbstractString};
    verbose::Bool = true,
    cost_tracker = Threads.Atomic{Float64}(0.0),
    kwargs...)

Extracts "tags" (metadata/keywords) from a vector of docs using the provided model (kwarg model).

Arguments

  • docs: A vector of strings to be embedded.

  • verbose: A boolean flag for verbose output. Default is true.

  • model: The model to use for tags extraction. Default is PT.MODEL_CHAT.

  • template: A template to be used for tags extraction. Default is :RAGExtractMetadataShort.

  • cost_tracker: A Threads.Atomic{Float64} object to track the total cost of the API calls. Useful to pass the total cost to the parent call.

source


# PromptingTools.Experimental.RAGTools.get_tags — Method.
julia
get_tags(tagger::PassthroughTagger, docs::AbstractVector{<:AbstractString};
    tags::AbstractVector{<:AbstractVector{<:AbstractString}},
    kwargs...)

Pass tags directly as Vector of Vectors of strings (ie, tags[i] is the tags for docs[i]). It then builds the vocabulary from the tags and returns both the tags in matrix form and the vocabulary.

source


# PromptingTools.Experimental.RAGTools.getpropertynested — Function.
julia
getpropertynested(
    nt::NamedTuple, parent_keys::Vector{Symbol}, key::Symbol, default = nothing)

Get a property key from a nested NamedTuple nt, where the property is nested to a key in parent_keys.

Useful for nested kwargs where we want to get some property in parent_keys subset (eg, model in retriever_kwargs).

Examples

julia
kw = (; abc = (; def = "x"))
getpropertynested(kw, [:abc], :def)
# Output: "x"

source


# PromptingTools.Experimental.RAGTools.hamming_distance — Method.
julia
hamming_distance(
    mat::AbstractMatrix{T}, query::AbstractVector{T})::Vector{Int} where {T <: Integer}

Calculates the column-wise Hamming distance between a matrix of binary vectors mat and a single binary vector vect.

This is the first-pass ranking for BinaryCosineSimilarity method.

Implementation from domluna's tinyRAG.

source


# PromptingTools.Experimental.RAGTools.hcat_truncate — Method.
julia
hcat_truncate(matrices::AbstractVector{<:AbstractMatrix{T}},
    truncate_dimension::Union{Nothing, Int} = nothing; verbose::Bool = false) where {T <:
                                                                                     Real}

Horizontal concatenation of matrices, with optional truncation of the rows of each matrix to the specified dimension (reducing embedding dimensionality).

More efficient that a simple splatting, as the resulting matrix is pre-allocated in one go.

Returns: a Matrix{Float32}

Arguments

  • matrices::AbstractVector{<:AbstractMatrix{T}}: Vector of matrices to concatenate

  • truncate_dimension::Union{Nothing,Int}=nothing: Dimension to truncate to, or nothing or 0 to skip truncation. If truncated, the columns will be normalized.

  • verbose::Bool=false: Whether to print verbose output.

Examples

julia
a = rand(Float32, 1000, 10)
b = rand(Float32, 1000, 20)

c = hcat_truncate([a, b])
size(c) # (1000, 30)

d = hcat_truncate([a, b], 500)
size(d) # (500, 30)

source


# PromptingTools.Experimental.RAGTools.load_text — Method.
julia
load_text(chunker::AbstractChunker, input;
    kwargs...)

Load text from input using the provided chunker. Called by get_chunks.

Available chunkers:

  • FileChunker: The function opens each file in input and reads its contents.

  • TextChunker: The function assumes that input is a vector of strings to be chunked, you MUST provide corresponding sources.

source


# PromptingTools.Experimental.RAGTools.merge_kwargs_nested — Method.
julia
merge_kwargs_nested(nt1::NamedTuple, nt2::NamedTuple)

Merges two nested NamedTuples nt1 and nt2 recursively. The nt2 values will overwrite the nt1 values when overlapping.

Example

julia
kw = (; abc = (; def = "x"))
kw2 = (; abc = (; def = "x", def2 = 2), new = 1)
merge_kwargs_nested(kw, kw2)

source


# PromptingTools.Experimental.RAGTools.pack_bits — Method.
julia
pack_bits(arr::AbstractMatrix{<:Bool}) -> Matrix{UInt64}
pack_bits(vect::AbstractVector{<:Bool}) -> Vector{UInt64}

Pack a matrix or vector of boolean values into a more compact representation using UInt64.

Arguments (Input)

  • arr::AbstractMatrix{<:Bool}: A matrix of boolean values where the number of rows must be divisible by 64.

Returns

  • For arr::AbstractMatrix{<:Bool}: Returns a matrix of UInt64 where each element represents 64 boolean values from the original matrix.

Examples

For vectors:

julia
bin = rand(Bool, 128)
binint = pack_bits(bin)
binx = unpack_bits(binint)
@assert bin == binx

For matrices:

julia
bin = rand(Bool, 128, 10)
binint = pack_bits(bin)
binx = unpack_bits(binint)
@assert bin == binx

source


# PromptingTools.Experimental.RAGTools.permutation_step! — Method.
julia
permutation_step!(
    result::RankGPTResult; rank_start::Integer = 1, rank_end::Integer = 100, kwargs...)

One sub-step of the RankGPT algorithm permutation ranking within the window of chunks defined by rank_start and rank_end positions.

source


# PromptingTools.Experimental.RAGTools.preprocess_tokens — Function.
julia
preprocess_tokens(text::AbstractString, stemmer=nothing; stopwords::Union{Nothing,Set{String}}=nothing, min_length::Int=3)

Preprocess provided text by removing numbers, punctuation, and applying stemming for BM25 search index.

Returns a list of preprocessed tokens.

Example

julia
stemmer = Snowball.Stemmer("english")
stopwords = Set(["a", "an", "and", "are", "as", "at", "be", "but", "by", "for", "if", "in", "into", "is", "it", "no", "not", "of", "on", "or", "such", "some", "that", "the", "their", "then", "there", "these", "they", "this", "to", "was", "will", "with"])
text = "This is a sample paragraph to test the functionality of your text preprocessor. It contains a mix of uppercase and lowercase letters, as well as punctuation marks such as commas, periods, and exclamation points! Let's see how your preprocessor handles quotes, like "this one", and also apostrophes, like in don't. Will it preserve the formatting of this paragraph, including the indentation and line breaks?"
preprocess_tokens(text, stemmer; stopwords)

source


# PromptingTools.Experimental.RAGTools.print_html — Method.
julia
print_html([io::IO,] parent_node::AbstractAnnotatedNode)

print_html([io::IO,] rag::AbstractRAGResult; add_sources::Bool = false,
    add_scores::Bool = false, default_styler = HTMLStyler(),
    low_styler = HTMLStyler(styles = "color:magenta", classes = ""),
    medium_styler = HTMLStyler(styles = "color:blue", classes = ""),
    high_styler = HTMLStyler(styles = "", classes = ""), styler_kwargs...)

Pretty-prints the annotation parent_node (or RAGResult) to the io stream (or returns the string) in HTML format (assumes node is styled with styler HTMLStyler).

It wraps each "token" into a span with requested styling (HTMLStyler's properties classes and styles). It also replaces new lines with <br> for better HTML formatting.

For any non-HTML styler, it prints the content as plain text.

Returns

  • nothing if io is provided

  • or the string with HTML-formatted text (if io is not provided, we print the result out)

See also HTMLStyler, annotate_support, and set_node_style! for how the styling is applied and what the arguments mean.

Examples

Note: RT is an alias for PromptingTools.Experimental.RAGTools

Simple start directly with the RAGResult:

julia
# set up the text/RAGResult
context = [
    "This is a test context.", "Another context sentence.", "Final piece of context."]
answer = "This is a test answer. It has multiple sentences."
rag = RT.RAGResult(; context, final_answer=answer, question="")

# print the HTML
print_html(rag)

Low-level control by creating our AnnotatedNode:

julia
# prepare your HTML styling
styler_kwargs = (;
    default_styler=RT.HTMLStyler(),
    low_styler=RT.HTMLStyler(styles="color:magenta", classes=""),
    medium_styler=RT.HTMLStyler(styles="color:blue", classes=""),
    high_styler=RT.HTMLStyler(styles="", classes=""))

# annotate the text
context = [
    "This is a test context.", "Another context sentence.", "Final piece of context."]
answer = "This is a test answer. It has multiple sentences."

parent_node = RT.annotate_support(
    RT.TrigramAnnotater(), answer, context; add_sources=false, add_scores=false, styler_kwargs...)

# print the HTML
print_html(parent_node)

# or to accumulate more nodes
io = IOBuffer()
print_html(io, parent_node)

source


# PromptingTools.Experimental.RAGTools.rank_gpt — Method.
julia
rank_gpt(chunks::AbstractVector{<:AbstractString}, question::AbstractString;
    verbose::Int = 1, rank_start::Integer = 1, rank_end::Integer = 100,
    window_size::Integer = 20, step::Integer = 10,
    num_rounds::Integer = 1, model::String = "gpt4o", kwargs...)

Ranks the chunks based on their relevance for question. Returns the ranking permutation of the chunks in the order they are most relevant to the question (the first is the most relevant).

Example

julia
result = rank_gpt(chunks, question; rank_start=1, rank_end=25, window_size=8, step=4, num_rounds=3, model="gpt4o")

Reference

[1] Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents by W. Sun et al. [2] RankGPT Github

source


# PromptingTools.Experimental.RAGTools.rank_sliding_window! — Method.
julia
rank_sliding_window!(
    result::RankGPTResult; verbose::Int = 1, rank_start = 1, rank_end = 100,
    window_size = 20, step = 10, model::String = "gpt4o", kwargs...)

One single pass of the RankGPT algorithm permutation ranking across all positions between rank_start and rank_end.

source


# PromptingTools.Experimental.RAGTools.receive_permutation! — Method.
julia
receive_permutation!(
    curr_rank::AbstractVector{<:Integer}, response::AbstractString;
    rank_start::Integer = 1, rank_end::Integer = 100)

Extracts and heals the permutation to contain all ranking positions.

source


# PromptingTools.Experimental.RAGTools.reciprocal_rank_fusion — Method.
julia
reciprocal_rank_fusion(args...; k::Int=60)

Merges multiple rankings and calculates the reciprocal rank score for each chunk (discounted by the inverse of the rank).

Example

julia
positions1 = [1, 3, 5, 7, 9]
positions2 = [2, 4, 6, 8, 10]
positions3 = [2, 4, 6, 11, 12]

merged_positions, scores = reciprocal_rank_fusion(positions1, positions2, positions3)

source


# PromptingTools.Experimental.RAGTools.reciprocal_rank_fusion — Method.
julia
reciprocal_rank_fusion(
    positions1::AbstractVector{<:Integer}, scores1::AbstractVector{<:T},
    positions2::AbstractVector{<:Integer},
    scores2::AbstractVector{<:T}; k::Int = 60) where {T <: Real}

Merges two sets of rankings and their joint scores. Calculates the reciprocal rank score for each chunk (discounted by the inverse of the rank).

Example

julia
positions1 = [1, 3, 5, 7, 9]
scores1 = [0.9, 0.8, 0.7, 0.6, 0.5]
positions2 = [2, 4, 6, 8, 10]
scores2 = [0.5, 0.6, 0.7, 0.8, 0.9]

merged, scores = reciprocal_rank_fusion(positions1, scores1, positions2, scores2; k = 60)

source


# PromptingTools.Experimental.RAGTools.refine! — Method.
julia
refine!(
    refiner::NoRefiner, index::AbstractChunkIndex, result::AbstractRAGResult;
    kwargs...)

Simple no-op function for refine!. It simply copies the result.answer and result.conversations[:answer] without any changes.

source


# PromptingTools.Experimental.RAGTools.refine! — Method.
julia
refine!(
    refiner::SimpleRefiner, index::AbstractDocumentIndex, result::AbstractRAGResult;
    verbose::Bool = true,
    model::AbstractString = PT.MODEL_CHAT,
    template::Symbol = :RAGAnswerRefiner,
    cost_tracker = Threads.Atomic{Float64}(0.0),
    kwargs...)

Give model a chance to refine the answer (using the same or different context than previously provided).

This method uses the same context as the original answer, however, it can be modified to do additional retrieval and use a different context.

Returns

  • Mutated result with result.final_answer and the full conversation saved in result.conversations[:final_answer]

Arguments

  • refiner::SimpleRefiner: The method to use for refining the answer. Uses aigenerate.

  • index::AbstractDocumentIndex: The index containing chunks and sources.

  • result::AbstractRAGResult: The result containing the context and question to generate the answer for.

  • model::AbstractString: The model to use for generating the answer. Defaults to PT.MODEL_CHAT.

  • verbose::Bool: If true, enables verbose logging.

  • template::Symbol: The template to use for the aigenerate function. Defaults to :RAGAnswerRefiner.

  • cost_tracker: An atomic counter to track the cost of the operation.

source


# PromptingTools.Experimental.RAGTools.refine! — Method.
julia
refine!(
    refiner::TavilySearchRefiner, index::AbstractDocumentIndex, result::AbstractRAGResult;
    verbose::Bool = true,
    model::AbstractString = PT.MODEL_CHAT,
    include_answer::Bool = true,
    max_results::Integer = 5,
    include_domains::AbstractVector{<:AbstractString} = String[],
    exclude_domains::AbstractVector{<:AbstractString} = String[],
    template::Symbol = :RAGWebSearchRefiner,
    cost_tracker = Threads.Atomic{Float64}(0.0),
    kwargs...)

Refines the answer by executing a web search using the Tavily API. This method aims to enhance the answer's accuracy and relevance by incorporating information retrieved from the web.

Note: The web results and web answer (if requested) will be added to the context and sources!

Returns

  • Mutated result with result.final_answer and the full conversation saved in result.conversations[:final_answer].

  • In addition, the web results and web answer (if requested) are appended to the result.context and result.sources for correct highlighting and verification.

Arguments

  • refiner::TavilySearchRefiner: The method to use for refining the answer. Uses aigenerate with a web search template.

  • index::AbstractDocumentIndex: The index containing chunks and sources.

  • result::AbstractRAGResult: The result containing the context and question to generate the answer for.

  • model::AbstractString: The model to use for generating the answer. Defaults to PT.MODEL_CHAT.

  • include_answer::Bool: If true, includes the answer from Tavily in the web search.

  • max_results::Integer: The maximum number of results to return.

  • include_domains::AbstractVector{<:AbstractString}: A list of domains to include in the search results. Default is an empty list.

  • exclude_domains::AbstractVector{<:AbstractString}: A list of domains to exclude from the search results. Default is an empty list.

  • verbose::Bool: If true, enables verbose logging.

  • template::Symbol: The template to use for the aigenerate function. Defaults to :RAGWebSearchRefiner.

  • cost_tracker: An atomic counter to track the cost of the operation.

Example

julia
refiner!(TavilySearchRefiner(), index, result)
# See result.final_answer or pprint(result)

To enable this refiner in a full RAG pipeline, simply swap the component in the config:

julia
cfg = RT.RAGConfig()
cfg.generator.refiner = RT.TavilySearchRefiner()

result = airag(cfg, index; question, return_all = true)
pprint(result)

source


# PromptingTools.Experimental.RAGTools.rephrase — Method.
julia
rephrase(rephraser::SimpleRephraser, question::AbstractString;
    verbose::Bool = true,
    model::String = PT.MODEL_CHAT, template::Symbol = :RAGQueryHyDE,
    cost_tracker = Threads.Atomic{Float64}(0.0))

Rephrases the question using the provided rephraser template = RAGQueryHyDE.

Special flavor of rephrasing using HyDE (Hypothetical Document Embedding) method, which aims to find the documents most similar to a synthetic passage that would be a good answer to our question.

Returns both the original and the rephrased question.

Arguments

  • rephraser: Type that dictates the logic of rephrasing step.

  • question: The question to be rephrased.

  • model: The model to use for rephrasing. Default is PT.MODEL_CHAT.

  • template: The rephrasing template to use. Default is :RAGQueryHyDE. Find more with aitemplates("rephrase").

  • verbose: A boolean flag indicating whether to print verbose logging. Default is true.

source


# PromptingTools.Experimental.RAGTools.rephrase — Method.
julia
rephrase(rephraser::NoRephraser, question::AbstractString; kwargs...)

No-op, simple passthrough.

source


# PromptingTools.Experimental.RAGTools.rephrase — Method.
julia
rephrase(rephraser::SimpleRephraser, question::AbstractString;
    verbose::Bool = true,
    model::String = PT.MODEL_CHAT, template::Symbol = :RAGQueryOptimizer,
    cost_tracker = Threads.Atomic{Float64}(0.0), kwargs...)

Rephrases the question using the provided rephraser template.

Returns both the original and the rephrased question.

Arguments

  • rephraser: Type that dictates the logic of rephrasing step.

  • question: The question to be rephrased.

  • model: The model to use for rephrasing. Default is PT.MODEL_CHAT.

  • template: The rephrasing template to use. Default is :RAGQueryOptimizer. Find more with aitemplates("rephrase").

  • verbose: A boolean flag indicating whether to print verbose logging. Default is true.

source


# PromptingTools.Experimental.RAGTools.rerank — Method.
julia
rerank(
    reranker::CohereReranker, index::AbstractDocumentIndex, question::AbstractString,
    candidates::AbstractCandidateChunks;
    verbose::Bool = false,
    api_key::AbstractString = PT.COHERE_API_KEY,
    top_n::Integer = length(candidates.scores),
    model::AbstractString = "rerank-english-v3.0",
    return_documents::Bool = false,
    cost_tracker = Threads.Atomic{Float64}(0.0),
    kwargs...)

Re-ranks a list of candidate chunks using the Cohere Rerank API. See https://cohere.com/rerank for more details.

Arguments

  • reranker: Using Cohere API

  • index: The index that holds the underlying chunks to be re-ranked.

  • question: The query to be used for the search.

  • candidates: The candidate chunks to be re-ranked.

  • top_n: The number of most relevant documents to return. Default is length(documents).

  • model: The model to use for reranking. Default is rerank-english-v3.0.

  • return_documents: A boolean flag indicating whether to return the reranked documents in the response. Default is false.

  • verbose: A boolean flag indicating whether to print verbose logging. Default is false.

  • cost_tracker: An atomic counter to track the cost of the retrieval. Not implemented /tracked (cost unclear). Provided for consistency.

source


# PromptingTools.Experimental.RAGTools.rerank — Method.
julia
rerank(
    reranker::RankGPTReranker, index::AbstractDocumentIndex, question::AbstractString,
    candidates::AbstractCandidateChunks;
    api_key::AbstractString = PT.OPENAI_API_KEY,
    model::AbstractString = PT.MODEL_CHAT,
    verbose::Bool = false,
    top_n::Integer = length(candidates.scores),
    unique_chunks::Bool = true,
    cost_tracker = Threads.Atomic{Float64}(0.0),
    kwargs...)

Re-ranks a list of candidate chunks using the RankGPT algorithm. See https://github.com/sunnweiwei/RankGPT for more details.

It uses LLM calls to rank the candidate chunks.

Arguments

  • reranker: Using Cohere API

  • index: The index that holds the underlying chunks to be re-ranked.

  • question: The query to be used for the search.

  • candidates: The candidate chunks to be re-ranked.

  • top_n: The number of most relevant documents to return. Default is length(documents).

  • model: The model to use for reranking. Default is rerank-english-v3.0.

  • verbose: A boolean flag indicating whether to print verbose logging. Default is 1.

  • unique_chunks: A boolean flag indicating whether to remove duplicates from the candidate chunks prior to reranking (saves compute time). Default is true.

Examples

julia
index = <some index>
question = "What are the best practices for parallel computing in Julia?"

cfg = RAGConfig(; retriever = SimpleRetriever(; reranker = RT.RankGPTReranker()))
msg = airag(cfg, index; question, return_all = true)

To get full verbosity of logs, set verbose = 5 (anything higher than 3).

julia
msg = airag(cfg, index; question, return_all = true, verbose = 5)

Reference

[1] Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents by W. Sun et al. [2] RankGPT Github

source


# PromptingTools.Experimental.RAGTools.retrieve — Method.
julia
retrieve(retriever::AbstractRetriever,
    index::AbstractChunkIndex,
    question::AbstractString;
    verbose::Integer = 1,
    top_k::Integer = 100,
    top_n::Integer = 5,
    api_kwargs::NamedTuple = NamedTuple(),
    rephraser::AbstractRephraser = retriever.rephraser,
    rephraser_kwargs::NamedTuple = NamedTuple(),
    embedder::AbstractEmbedder = retriever.embedder,
    embedder_kwargs::NamedTuple = NamedTuple(),
    processor::AbstractProcessor = retriever.processor,
    processor_kwargs::NamedTuple = NamedTuple(),
    finder::AbstractSimilarityFinder = retriever.finder,
    finder_kwargs::NamedTuple = NamedTuple(),
    tagger::AbstractTagger = retriever.tagger,
    tagger_kwargs::NamedTuple = NamedTuple(),
    filter::AbstractTagFilter = retriever.filter,
    filter_kwargs::NamedTuple = NamedTuple(),
    reranker::AbstractReranker = retriever.reranker,
    reranker_kwargs::NamedTuple = NamedTuple(),
    cost_tracker = Threads.Atomic{Float64}(0.0),
    kwargs...)

Retrieves the most relevant chunks from the index for the given question and returns them in the RAGResult object.

This is the main entry point for the retrieval stage of the RAG pipeline. It is often followed by generate! step.

Notes:

  • The default flow is build_context! -> answer! -> refine! -> postprocess!.

The arguments correspond to the steps of the retrieval process (rephrasing, embedding, finding similar docs, tagging, filtering by tags, reranking). You can customize each step by providing a new custom type that dispatches the corresponding function, eg, create your own type struct MyReranker<:AbstractReranker end and define the custom method for it rerank(::MyReranker,...) = ....

Note: Discover available retrieval sub-types for each step with subtypes(AbstractRephraser) and similar for other abstract types.

If you're using locally-hosted models, you can pass the api_kwargs with the url field set to the model's URL and make sure to provide corresponding model kwargs to rephraser, embedder, and tagger to use the custom models (they make AI calls).

Arguments

  • retriever: The retrieval method to use. Default is SimpleRetriever but could be AdvancedRetriever for more advanced retrieval.

  • index: The index that holds the chunks and sources to be retrieved from.

  • question: The question to be used for the retrieval.

  • verbose: If >0, it prints out verbose logging. Default is 1. If you set it to 2, it will print out logs for each sub-function.

  • top_k: The TOTAL number of closest chunks to return from find_closest. Default is 100. If there are multiple rephrased questions, the number of chunks per each item will be top_k ÷ number_of_rephrased_questions.

  • top_n: The TOTAL number of most relevant chunks to return for the context (from rerank step). Default is 5.

  • api_kwargs: Additional keyword arguments to be passed to the API calls (shared by all ai* calls).

  • rephraser: Transform the question into one or more questions. Default is retriever.rephraser.

  • rephraser_kwargs: Additional keyword arguments to be passed to the rephraser.

    • model: The model to use for rephrasing. Default is PT.MODEL_CHAT.

    • template: The rephrasing template to use. Default is :RAGQueryOptimizer or :RAGQueryHyDE (depending on the rephraser selected).

  • embedder: The embedding method to use. Default is retriever.embedder.

  • embedder_kwargs: Additional keyword arguments to be passed to the embedder.

  • processor: The processor method to use when using Keyword-based index. Default is retriever.processor.

  • processor_kwargs: Additional keyword arguments to be passed to the processor.

  • finder: The similarity search method to use. Default is retriever.finder, often CosineSimilarity.

  • finder_kwargs: Additional keyword arguments to be passed to the similarity finder.

  • tagger: The tag generating method to use. Default is retriever.tagger.

  • tagger_kwargs: Additional keyword arguments to be passed to the tagger. Noteworthy arguments:

    • tags: Directly provide the tags to use for filtering (can be String, Regex, or Vector{String}). Useful for tagger = PassthroughTagger.

  • filter: The tag matching method to use. Default is retriever.filter.

  • filter_kwargs: Additional keyword arguments to be passed to the tag filter.

  • reranker: The reranking method to use. Default is retriever.reranker.

  • reranker_kwargs: Additional keyword arguments to be passed to the reranker.

    • model: The model to use for reranking. Default is rerank-english-v2.0 if you use reranker = CohereReranker().

  • cost_tracker: An atomic counter to track the cost of the retrieval. Default is Threads.Atomic{Float64}(0.0).

See also: SimpleRetriever, AdvancedRetriever, build_index, rephrase, get_embeddings, get_keywords, find_closest, get_tags, find_tags, rerank, RAGResult.

Examples

Find the 5 most relevant chunks from the index for the given question.

julia
# assumes you have an existing index `index`
retriever = SimpleRetriever()

result = retrieve(retriever,
    index,
    "What is the capital of France?",
    top_n = 5)

# or use the default retriever (same as above)
result = retrieve(retriever,
    index,
    "What is the capital of France?",
    top_n = 5)

Apply more advanced retrieval with question rephrasing and reranking (requires COHERE_API_KEY). We will obtain top 100 chunks from embeddings (top_k) and top 5 chunks from reranking (top_n).

julia
retriever = AdvancedRetriever()

result = retrieve(retriever, index, question; top_k=100, top_n=5)

You can use the retriever to customize your retrieval strategy or directly change the strategy types in the retrieve kwargs!

Example of using locally-hosted model hosted on localhost:8080:

julia
retriever = SimpleRetriever()
result = retrieve(retriever, index, question;
    rephraser_kwargs = (; model = "custom"),
    embedder_kwargs = (; model = "custom"),
    tagger_kwargs = (; model = "custom"), api_kwargs = (;
        url = "http://localhost:8080"))

source


# PromptingTools.Experimental.RAGTools.run_qa_evals — Method.
julia
run_qa_evals(index::AbstractChunkIndex, qa_items::AbstractVector{<:QAEvalItem};
    api_kwargs::NamedTuple = NamedTuple(),
    airag_kwargs::NamedTuple = NamedTuple(),
    qa_evals_kwargs::NamedTuple = NamedTuple(),
    verbose::Bool = true, parameters_dict::Dict{Symbol, <:Any} = Dict{Symbol, Any}())

Evaluates a vector of QAEvalItems and returns a vector QAEvalResult. This function assesses the relevance and accuracy of the answers generated in a QA evaluation context.

See ?run_qa_evals for more details.

Arguments

  • qa_items::AbstractVector{<:QAEvalItem}: The vector of QA evaluation items containing the questions and their answers.

  • verbose::Bool: If true, enables verbose logging. Defaults to true.

  • api_kwargs::NamedTuple: Parameters that will be forwarded to the API calls. See ?aiextract for details.

  • airag_kwargs::NamedTuple: Parameters that will be forwarded to airag calls. See ?airag for details.

  • qa_evals_kwargs::NamedTuple: Parameters that will be forwarded to run_qa_evals calls. See ?run_qa_evals for details.

  • parameters_dict::Dict{Symbol, Any}: Track any parameters used for later evaluations. Keys must be Symbols.

Returns

Vector{QAEvalResult}: Vector of evaluation results that includes various scores and metadata related to the QA evaluation.

Example

julia
index = "..." # Assuming a proper index is defined
qa_items = [QAEvalItem(question="What is the capital of France?", answer="Paris", context="France is a country in Europe."),
            QAEvalItem(question="What is the capital of Germany?", answer="Berlin", context="Germany is a country in Europe.")]

# Let's run a test with `top_k=5`
results = run_qa_evals(index, qa_items; airag_kwargs=(;top_k=5), parameters_dict=Dict(:top_k => 5))

# Filter out the "failed" calls
results = filter(x->!isnothing(x.answer_score), results);

# See average judge score
mean(x->x.answer_score, results)

source


# PromptingTools.Experimental.RAGTools.run_qa_evals — Method.
julia
run_qa_evals(qa_item::QAEvalItem, ctx::RAGResult; verbose::Bool = true,
             parameters_dict::Dict{Symbol, <:Any}, judge_template::Symbol = :RAGJudgeAnswerFromContext,
             model_judge::AbstractString, api_kwargs::NamedTuple = NamedTuple()) -> QAEvalResult

Evaluates a single QAEvalItem using RAG details (RAGResult) and returns a QAEvalResult structure. This function assesses the relevance and accuracy of the answers generated in a QA evaluation context.

Arguments

  • qa_item::QAEvalItem: The QA evaluation item containing the question and its answer.

  • ctx::RAGResult: The RAG result used for generating the QA pair, including the original context and the answers. Comes from airag(...; return_context=true)

  • verbose::Bool: If true, enables verbose logging. Defaults to true.

  • parameters_dict::Dict{Symbol, Any}: Track any parameters used for later evaluations. Keys must be Symbols.

  • judge_template::Symbol: The template symbol for the AI model used to judge the answer. Defaults to :RAGJudgeAnswerFromContext.

  • model_judge::AbstractString: The AI model used for judging the answer's quality. Defaults to standard chat model, but it is advisable to use more powerful model GPT-4.

  • api_kwargs::NamedTuple: Parameters that will be forwarded to the API endpoint.

Returns

QAEvalResult: An evaluation result that includes various scores and metadata related to the QA evaluation.

Notes

  • The function computes a retrieval score and rank based on how well the context matches the QA context.

  • It then uses the judge_template and model_judge to score the answer's accuracy and relevance.

  • In case of errors during evaluation, the function logs a warning (if verbose is true) and the answer_score will be set to nothing.

Examples

Evaluating a QA pair using a specific context and model:

julia
qa_item = QAEvalItem(question="What is the capital of France?", answer="Paris", context="France is a country in Europe.")
ctx = RAGResult(source="Wikipedia", context="France is a country in Europe.", answer="Paris")
parameters_dict = Dict("param1" => "value1", "param2" => "value2")

eval_result = run_qa_evals(qa_item, ctx, parameters_dict=parameters_dict, model_judge="MyAIJudgeModel")

source


# PromptingTools.Experimental.RAGTools.score_retrieval_hit — Method.

Returns 1.0 if context overlaps or is contained within any of the candidate_context

source


# PromptingTools.Experimental.RAGTools.score_retrieval_rank — Method.

Returns Integer rank of the position where context overlaps or is contained within a candidate_context

source


# PromptingTools.Experimental.RAGTools.score_to_unit_scale — Method.
julia
score_to_unit_scale(x::AbstractVector{T}) where T<:Real

Shift and scale a vector of scores to the unit scale [0, 1].

Example

julia
x = [1.0, 2.0, 3.0, 4.0, 5.0]
scaled_x = score_to_unit_scale(x)

source


# PromptingTools.Experimental.RAGTools.set_node_style! — Method.
julia
set_node_style!(::TrigramAnnotater, node::AnnotatedNode;
    low_threshold::Float64 = 0.0, medium_threshold::Float64 = 0.5, high_threshold::Float64 = 1.0,
    default_styler::AbstractAnnotationStyler = Styler(),
    low_styler::AbstractAnnotationStyler = Styler(color = :magenta, bold = false),
    medium_styler::AbstractAnnotationStyler = Styler(color = :blue, bold = false),
    high_styler::AbstractAnnotationStyler = Styler(color = :nothing, bold = false),
    bold_multihits::Bool = false)

Sets style of node based on the provided rules

source


# PromptingTools.Experimental.RAGTools.setpropertynested — Method.
julia
setpropertynested(nt::NamedTuple, parent_keys::Vector{Symbol},
    key::Symbol,
    value

)

Setter for a property key in a nested NamedTuple nt, where the property is nested to a key in parent_keys.

Useful for nested kwargs where we want to change some property in parent_keys subset (eg, model in retriever_kwargs).

Examples

julia
kw = (; abc = (; def = "x"))
setpropertynested(kw, [:abc], :def, "y")
# Output: (abc = (def = "y",),)

Practical example of changing all model keys in CHAT-based steps in the pipeline:

julia
# changes :model to "gpt4t" whenever the parent key is in the below list (chat-based steps)
setpropertynested(kwargs,
    [:rephraser_kwargs, :tagger_kwargs, :answerer_kwargs, :refiner_kwargs],
    :model, "gpt4t")

Or changing an embedding model (across both indexer and retriever steps, because it's same step name):

julia
kwargs = setpropertynested(
        kwargs, [:embedder_kwargs],
        :model, "text-embedding-3-large"
    )

source


# PromptingTools.Experimental.RAGTools.split_into_code_and_sentences — Method.
julia
split_into_code_and_sentences(input::Union{String, SubString{String}})

Splits text block into code or text and sub-splits into units.

If code block, it splits by newline but keep the group_id the same (to have the same source) If text block, splits into sentences, bullets, etc., provides different group_id (to have different source)

source


# PromptingTools.Experimental.RAGTools.tags_extract — Method.
julia
tags_extract(item::Tag)
tags_extract(tags::Vector{Tag})

Extracts the Tag item into a string of the form category:::value (lowercased and spaces replaced with underscores).

Example

julia
msg = aiextract(:RAGExtractMetadataShort; return_type=MaybeTags, text="I like package DataFrames", instructions="None.")
metadata = tags_extract(msg.content.items)

source


# PromptingTools.Experimental.RAGTools.token_with_boundaries — Method.
julia
token_with_boundaries(
    prev_token::Union{Nothing, AbstractString}, curr_token::AbstractString,
    next_token::Union{Nothing, AbstractString})

Joins the three tokens together. Useful to add boundary tokens (like spaces vs brackets) to the curr_token to improve the matched context (ie, separate partial matches from exact match)

source


# PromptingTools.Experimental.RAGTools.tokenize — Method.
julia
tokenize(input::Union{String, SubString{String}})

Tokenizes provided input by spaces, special characters or Julia symbols (eg, =>).

Unlike other tokenizers, it aims to lossless - ie, keep both the separated text and the separators.

source


# PromptingTools.Experimental.RAGTools.translate_positions_to_parent — Method.
julia
translate_positions_to_parent(index::AbstractChunkIndex, positions::AbstractVector{<:Integer})

Translate positions to the parent index. Useful to convert between positions in a view and the original index.

Used whenever a chunkdata() is used to re-align positions in case index is a view.

source


# PromptingTools.Experimental.RAGTools.translate_positions_to_parent — Method.
julia
translate_positions_to_parent(
    index::SubChunkIndex, pos::AbstractVector{<:Integer})

Translate positions to the parent index. Useful to convert between positions in a view and the original index.

Used whenever a chunkdata() or tags() are used to re-align positions to the "parent" index.

source


# PromptingTools.Experimental.RAGTools.trigram_support! — Method.
julia
trigram_support!(parent_node::AnnotatedNode,
    context_trigrams::AbstractVector, trigram_func::F1 = trigrams, token_transform::F2 = identity;
    skip_trigrams::Bool = false, min_score::Float64 = 0.5,
    min_source_score::Float64 = 0.25,
    stop_words::AbstractVector{<:String} = STOPWORDS,
    styler_kwargs...) where {F1 <: Function, F2 <: Function}

Find if the parent_node.content is supported by the provided context_trigrams.

Logic:

  • Split the parent_node.content into tokens

  • Create an AnnotatedNode for each token

  • If skip_trigrams is enabled, it looks for an exact match in the context_trigrams

  • If no exact match found, it counts trigram-based match (include the surrounding tokens for better contextual awareness) as a score

  • Then it sets the style of the node based on the score

  • Lastly, it aligns the styles of neighboring nodes with score==nothing (eg, single character tokens)

  • Then, it rolls up the scores and sources to the parent node

For diagnostics, you can use AbstractTrees.print_tree(parent_node) to see the tree structure of each token and its score.

Example

julia

node = AnnotatedNode(content = "xyz")  trigram_support!(node, context_trigrams) # updates node.children! ```


[source](https://github.com/svilupp/PromptingTools.jl/blob/045379eac021f614c82e5d8672b17d06ff5f666c/src/Experimental/RAGTools/annotation.jl#L215-L244)

</div>
<br>
<div style='border-width:1px; border-style:solid; border-color:black; padding: 1em; border-radius: 25px;'>
<a id='PromptingTools.Experimental.RAGTools.trigrams-Tuple{AbstractString}' href='#PromptingTools.Experimental.RAGTools.trigrams-Tuple{AbstractString}'>#</a>&nbsp;<b><u>PromptingTools.Experimental.RAGTools.trigrams</u></b> &mdash; <i>Method</i>.




```julia
trigrams(input_string::AbstractString; add_word::AbstractString = "")

Splits provided input_string into a vector of trigrams (combination of three consecutive characters found in the input_string).

If add_word is provided, it is added to the resulting array. Useful to add the full word itself to the resulting array for exact match.

source


# PromptingTools.Experimental.RAGTools.trigrams_hashed — Method.
julia
trigrams_hashed(input_string::AbstractString; add_word::AbstractString = "")

Splits provided input_string into a Set of hashed trigrams (combination of three consecutive characters found in the input_string).

It is more efficient for lookups in large strings (eg, >100K characters).

If add_word is provided, it is added to the resulting array to hash. Useful to add the full word itself to the resulting array for exact match.

source


# PromptingTools.last_message — Method.
julia
PT.last_message(result::RAGResult)

Extract the last message from the RAGResult. It looks for final_answer first, then answer fields in the conversations dictionary. Returns nothing if not found.

source


# PromptingTools.last_output — Method.

Extracts the last output (generated text answer) from the RAGResult.

source


# PromptingTools.pprint — Method.
julia
PromptingTools.pprint(
    io::IO, node::AbstractAnnotatedNode;
    text_width::Int = displaysize(io)[2], add_newline::Bool = true)

Pretty print the node to the io stream, including all its children

Supports only node.style::Styler for now.

source


# PromptingTools.pprint — Method.
julia
PT.pprint(
    io::IO, r::AbstractRAGResult; add_context::Bool = false,
    text_width::Int = displaysize(io)[2], annotater_kwargs...)

Pretty print the RAG result r to the given io stream.

If add_context is true, the context will be printed as well. The text_width parameter can be used to control the width of the output.

You can provide additional keyword arguments to the annotater, eg, add_sources, add_scores, min_score, etc. See annotate_support for more details.

source