An AI-built prototype is useful when it proves a workflow, narrows requirements, or helps a team see demand. It becomes risky when the demo is treated as evidence that the system is ready for real data, real users, and real operational consequences.

The next question is not "can we ship the demo?" The useful question is "what must be hardened before this prototype is trusted in production?" That usually means behavior, data, permissions, monitoring, fallback, cost control, and ownership.

Symptoms this is your situation

  • The demo works with polished examples, but nobody has tested messy real inputs.
  • One person still runs part of the workflow manually behind the scenes.
  • The prototype reads customer data, documents, or internal notes without a clear data policy.
  • The model can call tools, write records, send messages, or trigger actions with broad permissions.
  • There is no regression set, evaluation log, incident path, or rollback plan.
  • The team is debating "launch it" versus "start over" without a hardening plan.

1. Separate the demo promise from the production job

Start by writing the production job in plain language. Who is the user? What input is accepted? What output is expected? What errors are tolerable? Which actions are forbidden? Which decisions require human approval?

If those answers are unclear, the prototype is still a discovery asset, not a production system. Keep it, but do not pretend the demo already contains the operating rules.

2. Put boundaries around data

AI prototypes often start with convenient data access. Production needs a stricter model. Decide which data classes can enter the workflow, which must stay out, how logs are retained, and whether customer records, internal documents, or credentials could leak through prompts, retrieval, attachments, or tool output.

Use the least data necessary for the job. Keep secrets, production database dumps, private keys, and raw customer records out of normal prompts and support channels. If sensitive data is unavoidable, design the intake, storage, audit, and deletion path intentionally.

3. Harden prompts, tools, and permissions

The highest-risk prototypes are not simple chat demos. They are workflows where the model can read private context, call APIs, write to business systems, or send messages. Treat model output as untrusted until validated.

Prefer allowlisted tools, scoped credentials, separate read and write permissions, output validation, confirmation steps for irreversible actions, and human approval for high-impact changes. Test prompt injection and malicious retrieved content, especially when the prototype reads emails, documents, tickets, websites, or uploaded files.

4. Test behavior, not only code

A working demo can still fail in production because the behavior was never measured. Build a small evaluation set: normal inputs, edge cases, bad instructions, missing context, contradictory documents, and examples where the correct answer is to refuse or ask for human review.

The goal is not academic perfection. The goal is to catch regressions and decide which failures are acceptable before users discover them. Keep the evaluation set close to the real workflow and rerun it when prompts, models, tools, or retrieval sources change.

5. Add observability and fallback

Production needs a way to see what happened without exposing sensitive data unnecessarily. Track model version, prompt version, tool calls, validation results, errors, review decisions, latency, and cost signals. Avoid logging raw secrets or unnecessary customer content.

Also define the boring path for failure. What happens when the model is uncertain, the API is down, output validation fails, a user disputes a result, or the monthly cost spikes? A queue, manual review path, disabled write mode, or rollback plan is part of production hardening, not an afterthought.

6. Decide who owns operation after launch

AI prototypes often stall because nobody owns the operational surface. Decide who changes prompts, who approves new data sources, who reviews incidents, who updates documentation, who handles user complaints, and who can disable risky behavior.

Without that owner, every future bug becomes an argument about whether the system is a product, a prototype, or a temporary script.

What not to do

  • Do not connect a demo directly to production data and business actions.
  • Do not let broad API permissions hide behind a friendly chat interface.
  • Do not trust generated text, SQL, code, emails, or tool arguments without validation.
  • Do not launch without a way to pause, review, or roll back risky behavior.
  • Do not treat one impressive demo as evidence that edge cases are handled.

Quick hardening checklist

  • Define the production user, job, accepted inputs, and forbidden actions.
  • Map data classes, retention rules, and places where sensitive data can leak.
  • Limit tools and credentials to the smallest useful permission set.
  • Add validation before model output reaches databases, messages, or external systems.
  • Create a regression set for normal, messy, adversarial, and refusal cases.
  • Log enough metadata to debug behavior without storing unnecessary secrets.
  • Define manual review, disabled mode, rollback, cost limits, and incident ownership.
  • Document the operating model so the system can survive handoff.

When to request a fit check

Request a fit check when the prototype is promising, but the path to production is unclear. The first result should identify whether the system needs a hardening sprint, a smaller controlled pilot, a data boundary redesign, or a decision to archive the demo and rebuild the useful part cleanly.

Prototype worked in demo but is not safe to ship?

If the AI workflow looks useful but the production risks are unclear, start with an initial fit check before connecting real users, data, or tools.

Request Fit Check

References

  1. NIST, AI Risk Management Framework .
  2. NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile .
  3. OWASP, Top 10 for Large Language Model Applications .
  4. NIST SP 800-218, Secure Software Development Framework (SSDF) Version 1.1 .
  5. ISO/IEC/IEEE 29148:2018, Systems and software engineering - Life cycle processes - Requirements engineering .