Rag System, Large Language Models, Document Management.
smoothing, reducing noise but potentially missing subtle transitions. Smaller k values retain more detail but may introduce noise. The smoothed gap scores provide a more unambiguous indication of where topic transitions occur. Example of Gap Scores after Smoothing / Image by the author 4. Boundary