Function find_duplicates 

pub fn find_duplicates(
    index_dir: &Path,
    corpus_dir: &Path,
    score_threshold: f64,
    per_doc_limit: usize,
) -> Result<Vec<DedupePair>>

Find near-duplicate pairs using Tantivy self-queries plus optional Jaccard floor.

For each document, query the index with a content-derived query and flag other documents that rank above score_threshold (excluding self). Pairs are unique and ordered by score desc.