Source: https://joinfinn.com/blog/two-week-ai-pilot

# How to run a two-week AI pilot.

Plan a focused AI pilot in ten working days with a scope worksheet, data-access checklist, test cases, team responsibilities, and a go-live decision.

By Finn · 5 min read

Published September 11, 2026

## At a glance

A two-week AI pilot should test one bounded workflow against a recorded baseline. Agree the outcome and access first, prepare representative cases, run a limited supervised sample, and finish with evidence for continuing, changing the scope, or stopping.

“Let’s try AI in support” is too broad to evaluate in two weeks. “Handle order-status requests for one store, using current fulfillment records, and hand exceptions to the existing team” is a useful starting point. The second statement identifies work, information, and an owner. This guide uses that illustrative example to build a ten-working-day pilot.

## Confirm the prerequisites before the clock starts.

A two-week schedule assumes the required accounts, permissions, and people are available. Choose a business owner who can resolve policy questions and review examples. Identify the team responsible for system access and a small group of operators who know the current work. Procurement, new integrations, or unresolved data access may require a longer schedule.

OpenAI’s agent guide discusses when a task benefits from an agent and when a simpler solution is sufficient. Apply that question before building: if the entire job is copying a field on a fixed schedule, an ordinary automation may meet the need. Use the pilot to evaluate a business result, not to justify a predetermined technology choice.

- One named workflow owner and someone available to review daily examples.
- Current policy material and permission to use the necessary business records.
- A baseline sample of comparable work, including exceptions.
- A defined channel, customer group, permitted actions, and escalation destination.

Sources: [OpenAI: A practical guide to building agents](https://openai.com/business/guides-and-resources/a-practical-guide-to-building-ai-agents/)

## Use a schedule with a deliverable at each stage.

Ten working days is a planning window, not a promise that every deployment fits it. Keep the first version narrow enough to inspect completely. If access is delayed or the process is disputed, reduce the scope or extend the schedule explicitly. Do not hide missing work inside a demonstration that only shows easy cases.

**Illustrative ten-working-day pilot schedule**

| When | Work | Reviewable output |
| --- | --- | --- |
| Days 1–2 | Agree the job, baseline, owner, sources, and permissions | Signed-off scope and acceptance criteria |
| Days 3–5 | Connect the records and prepare the workflow | A working example with visible results and handoffs |
| Days 6–8 | Test representative requests and failure cases | Results, unresolved issues, and revised instructions |
| Days 9–10 | Run a limited supervised sample and review the evidence | Continue, change, or stop decision with an owner |

## Write a scope that prevents accidental expansion.

For order-status support, define the store, eligible order types, identity checks, information the agent may disclose, and which fulfillment records are authoritative. Keep refunds, address changes, and replacements outside the initial scope unless they have been deliberately included and tested. A customer may ask for them; the workflow needs a useful handoff, not improvised authority.

Record the completed state precisely. In this example, completion might mean the customer received the verified status and any required exception reached an owner. If the carrier has no usable update, that is an unresolved information problem. The agent should say what is known and create the next step rather than invent a delivery date.

**Copyable pilot scope worksheet**

| Field | What to write |
| --- | --- |
| Goal | The business result this workflow should achieve |
| Eligible work | Channel, location, record type, time window, and exclusions |
| Sources | Authoritative systems, policy versions, and source owners |
| Authority | Permitted actions and decisions requiring review |
| Success | Required evidence and the acceptance threshold agreed by the business |
| Operations | Daily reviewer, escalation destination, pause conditions, and next review |

## Test what can go wrong before adding volume.

Use cases that resemble the work the team actually receives. Include a normal request, a missing order number, an identity mismatch, conflicting records, a delayed connection, and a request outside scope. Also test partial completion: the record lookup succeeds but the customer message cannot be delivered. The status should identify what remains unfinished.

Anthropic’s evaluation guidance emphasizes explicit outcomes and representative tasks. Have an operator and the workflow owner independently judge a few examples. If they disagree about what should have happened, clarify the policy before treating it as a model problem. Repeat important cases because one successful attempt does not establish dependable behavior.

Sources: [Anthropic: Demystifying evals for AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents)

## Measure the work the team still does.

Record eligible requests, attempted requests, accepted results, escalations, and corrected outcomes. Measure employee review and recovery time as well as response time. Keep unresolved requests in the report. A high completion rate produced by removing difficult cases from the denominator tells the team very little.

A small pilot can reveal feasibility and operating problems without proving long-term savings. Compare similar work and identify the sample period. If important edge cases did not occur, keep them in the test set and acknowledge the limited live evidence. The owner needs an honest basis for a next step.

## Finish with a decision and a handover.

Continue when the workflow meets the agreed quality bar and the team can operate its exceptions. Change the scope when a specific dependency or class of request is the obstacle. Stop when the task offers too little value, cannot obtain the required information, or creates more work than it removes. All three are useful pilot outcomes.

Keep the source register, examples, results, outstanding issues, and operating instructions together. Finn provides a home for customer and employee agents, connected business work, and recurring Tasks. Begin with the workflow your team can explain and inspect, then use the evidence to decide how much responsibility to add.

## Common questions

### Can every AI pilot launch in two weeks?

No. This sample plan assumes one narrow workflow, available access, and responsive owners. New integrations, procurement, or unresolved policies can require more time.

### How many workflows should an initial pilot cover?

One clearly bounded workflow is usually the most useful starting scope. Add another only when it answers a specific question the first cannot and remains practical to evaluate.

### What is the most important pilot deliverable?

A reviewable decision based on observed results, remaining risks, and operating effort. A successful demo alone does not establish that the team can run the workflow.

## Further reading

- [OpenAI: A practical guide to building agents](https://openai.com/business/guides-and-resources/a-practical-guide-to-building-ai-agents/)
- [Anthropic: Demystifying evals for AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents)

## Explore more

- [Choose the first task](https://joinfinn.com/blog/choose-your-first-ai-task)
- [Test before publishing](https://joinfinn.com/blog/test-an-ai-workflow-before-publishing)
- [Make the next pilot easier](https://joinfinn.com/blog/make-every-ai-pilot-easier)
- [Writing briefs, sources, and editorial notes for this collection](https://joinfinn.com/blog/ai-operations-editorial-notes)

[Discuss your pilot](https://joinfinn.com/contact)

[Discuss your workflow](https://joinfinn.com/contact) · [How it works](https://joinfinn.com/demo/app/#/w/borough/overview) · [Get started](https://joinfinn.com/app/#/signup)