From document to passage
During indexing, a document is validated, text extracted and meaningful passages created. Each passage retains its source, version and permissions. Appropriate chunk size depends on the material: very short passages lose context; very long ones add irrelevant information.
Tables, appendices and references can break during processing. Evaluate extraction quality separately from generation quality. The system should not conceal an unreadable document behind a fluent answer.
Permission precedes similarity
First determine which documents the user can access. Then search that permitted collection. Filtering only the final response is too late: restricted information may already have reached the model’s context.
Apply the rule to conversations, caches and references as well. Revoked access should not leave old results available through another route. Test explicitly with users from different permission groups.
Similarity does not establish truth
Vector search finds related content; it does not determine whether it is current or authoritative. Combine relevance with version, source class and business context. When documents conflict, the answer should reveal the uncertainty.
A citation must support the specific statement. A link at the end of a paragraph is not enough. Verify that a reader can reach the exact passage and confirm the meaning.
Documents may carry hostile instructions
Text such as “ignore previous rules” can appear inside a retrieved file. Treat source material as untrusted data. Permissions, tools and allowed actions cannot depend solely on a model instruction.
Limit the source collection and publishing process. Logs should make source versions traceable without uncontrolled duplication of confidential content.
Example: a withdrawn policy in search results
Consider an expense question for which retrieval finds both an old policy and its replacement. Both contain similar wording and may appear relevant. Without document status, the system can cite the withdrawn threshold and produce an entirely plausible answer.
A control test should establish which version is permitted, whether the old passage has left the active index and what the employee sees. If both versions remain valid for different business entities, the correct organisational context is needed. This tests more than language: it checks the relationship between the business rule, the source data and the user’s permissions.
Evaluate correct refusals too
The test set needs answerable questions, questions without sufficient evidence and attempts to access prohibited material. Score retrieval, citation support and answer grounding separately. Repeat the checks after changing the model or index.