Skip to content

[text_similarity_reranker]: Automatically extend top N when it is less than rank window size #142321

@Mikep86

Description

@Mikep86

Description

For rerank services with a top N (or equivalent) parameter, the text similarity reranker currently requires that this value be greater than or equal to the rank window size. However, we can override this value when creating the rerank inference request to automatically increase top N as necessary. This will create a better user experience than throwing an error.

However, for this to work, we also need to gracefully handle when top N is set on a rerank inference request for a rerank service that does not support it. Currently, this is an error. We should investigate how we can be lenient when applying this setting when top N is set internally.

See this this PR thread for more context: #142039 (comment)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions