1. A demonstration without success criteria
An impressive meeting can show a possibility, not everyday usefulness. Without a baseline, you cannot tell whether work takes less time or effort has merely moved into checking results.
Define the task, users, existing effort and tolerable failures before the pilot. Include ordinary and difficult cases. Agree the conditions for continuing in advance.
2. An archive without an owner
Duplicate, conflicting and outdated documents produce ambiguous context. A larger model cannot decide which internal document is authoritative. Without content ownership, answer quality gradually deteriorates.
Start with a limited library. Record versions, validity and review responsibility. Establish a withdrawal and index-refresh process before adding the next archive.
3. Permissions postponed until later
Prototypes often begin with one shared index and every document. Moving that arrangement into real use can expose information across departments or clients. “Do not reveal confidential data” is not dependable access enforcement.
Apply permissions before retrieval and test negative cases. Check exports, conversation history and caches, not only the main interface.
4. Automation without an action boundary
Finding information and taking action carry different risks. An assistant with broad send, change or delete permissions can turn a wrong answer into a business consequence.
Start with reading and preparation. Use predefined operations, limited service identities and human approval. Plan how to stop the workflow and safely retry a failed step.
Turn the mistakes into acceptance conditions
Put these five risks into a short acceptance record. Give each an owner, a test and a stopping condition. For example, missing current evidence should trigger a refusal, while disclosure of a prohibited document should stop pilot expansion until the issue is fixed and retested.
Not every control needs complexity. An approved source inventory, a verified backup and clear responsibility are more useful than a long list of promised features. After the pilot, compare the evidence with the agreed conditions and record remaining risks. This makes the deployment decision traceable rather than dependent on enthusiasm for the latest demonstration.
5. No plan after deployment
Models, packages and sources change. Without ownership of monitoring, recovery and reevaluation, the system remains an experiment on which the business has become dependent.
Document maintenance, backups, incident response and service exit. Test recovery into a separate environment. Treat a pilot as evidence for a decision, not automatic permission for a broad rollout.