Reranking

Enhancing Response Quality with Reranking

PrivateGPT offers a reranking feature aimed at optimizing response generation by filtering out irrelevant documents, potentially leading to faster response times and enhanced relevance of answers generated by the LLM.

Enabling Reranking

Document reranking can significantly improve the efficiency and quality of the responses by pre-selecting the most relevant documents before generating an answer. To leverage this feature, ensure that it is enabled in the RAG settings and consider adjusting the parameters to best fit your use case.

Additional Requirements

Before enabling reranking, you must install additional dependencies:

$
poetry install --extras rerank-sentence-transformers

This command installs dependencies for the cross-encoder reranker from sentence-transformers, which is currently the only supported method by PrivateGPT for document reranking.

Configuration

To enable and configure reranking, adjust the rag section within the settings.yaml file. Here are the key settings to consider:

• similarity_top_k: Determines the number of documents to initially retrieve and consider for reranking. This value should be larger than top_n.
• rerank:
  • enabled: Set to true to activate the reranking feature.
  • top_n: Specifies the number of documents to use in the final answer generation process, chosen from the top-ranked documents provided by similarity_top_k.

Example configuration snippet:

1
rag:
2
  similarity_top_k: 10  # Number of documents to retrieve and consider for reranking
3
  rerank:
4
    enabled: true
5
    top_n: 3  # Number of top-ranked documents to use for generating the answer