Fix it or rebuild it? Running an AI-built app in production — a decision table, and the five places AI-generated code breaks

The short answer: in my judgment, if you are running an AI-coded app or business system in production, it is usually cheaper and faster to fix it and keep using it than to rebuild — reinforcing the fragile parts after the fact, which is called "hardening". The exception is when the way data is stored, or the rules for who may see what, are fundamentally wrong for the business. Then rebuilding part or all of it can come out cheaper. Which case you are in is decided not by how the demo looks, but by the checkable signals in this article's decision table.

This article is written for you if:

• an app or business system you built with Claude Code, Cursor, Lovable, v0, Bolt or similar is already running, or is about to go into production;
• it handles customers' personal data, payments, or work the business depends on (orders, invoicing, inventory and so on); and
• you are a sole proprietor or an owner, or the person responsible for development at a company with no CTO (chief technology officer).

You can make the call without reading code. Engineer-level detail is kept in its own section, labelled "For engineers", near the end.

A disclosure up front: I take on paid reviews and development for apps built with AI. That is exactly why I also spell out the cases where you don't need to spend anything.

1. The decision table — fix it, partly rebuild it, or rebuild it

Two terms first. The data model is the design of what information is stored, in what units, and how the pieces connect. The authorization model is the set of rules for which data a logged-in person may see and change. It is a different mechanism from logging in (proving who you are), and as I see it, it is the one AI-built apps tend to miss.

One more thing comes before the table: someone must be able to say what correct behaviour is. This is not a signal that separates fixing from rebuilding; it is a prerequisite for both. If nobody can explain it, a fix risks quietly changing the specification, and a rebuild has no definition of what to rebuild. The person who explains it can be you. Whichever way you go, writing it down is the first job (section 3 shows how).

Reading it is simple. If most rows land in the left column, fix it and keep using it. If some rows land on the right but they are confined to one area, rebuild only that area. Only when right-hand answers span several rows and there is no real customer data yet does a full rebuild become a cheaper candidate.

So what if right-hand answers span several rows but real customer data is already in production? Say data that must be separated per company has no notion of company, and real customers are using it. What I recommend then is not bolting a check onto each screen, but rebuilding only the layer with the fundamental problem — how data is stored, or how permissions are decided — data migration included (a partial rebuild).

Misreading the table (counter-examples)

• "AI wrote it" or "the code is messy" is not a reason to rebuild. Even if it looks untidy, a straightforward structure is cheaper to fix.
• "There are lots of holes" is not a reason on its own either. Many holes can share one cause — a single check missing from a shared function — and then there is one place to fix. Look at how far the problems spread and how deep they go, not at the count.
• "There are no tests" is not a reason. Tests can be added later. Adding them is, if anything, the core of hardening.
• Conversely, "it works, so fixing it will do" does not always hold. If the design has no boundary separating each company's data and you bolt a check onto each screen, the patchwork grows and leaks get more likely.
• "A rebuild will come out clean" does not always hold either. Rebuild it the same way — taking AI's output without checking it — and the same kinds of holes come back.

Don't decide from a demo

A demo (clicking through the screens) only shows whether it works when things go well. Whether one user can see another's data, what happens when the same button is pressed twice at once, what happens when an outside service goes down — the questions production asks cannot be answered by clicking through screens.

So don't let a demo convince you that "fixing it will do" or that "rebuilding is faster". The material for the decision is findings based on reading the code: where the problems are, what they are, and how far they spread.

The order to decide in

1. Use the free self-check to see whether there are obvious holes (section 3)
2. Write down what must always be true — what correct behaviour is (section 3)
3. Get findings from someone who has read the code, with priorities
4. Fit the findings to the table above
5. If you fix, fix what touches money and personal data first

2. What happens if you leave it

The following are fictional examples made up for illustration. They are not real companies or cases. Each one takes a part AI-built apps tend to miss (section 6) and restates what can go wrong in business terms.

Fictional example 1: another customer's booking is visible

In a shop's booking system, changing the number at the end of the booking-confirmation URL by one shows another customer's name and phone number. The menus have no link to anyone else's booking, so the person who built it never noticed.

The app checks whether someone is logged in, but not whether that booking is theirs. Japan's Personal Information Protection Commission lists, among the situations in which reporting a leak to the authorities is mandatory, a leak involving more than 1,000 individuals and a leak that may have resulted from an act carried out with an unlawful purpose — including cases where a leak is only suspected. The preliminary report to the Commission must be made "promptly", and the Commission's general guidelines put that at roughly 3–5 days from when the business becomes aware of the situation (it depends on the case). Which situation applies depends on the facts, so check the Commission's page in the sources and the legal check before selling an AI-built app. How to test this yourself is in Is the login AI built for you real?.

Fictional example 2: orders are created twice

An online shop confirms orders when its payment provider sends a "payment completed" notification. On a day the notification arrives twice, the same order is created twice, stock is deducted twice, and two shipments are booked.

An automatic notification like this, sent from the payment provider, is called a webhook, and the address that receives it is the webhook endpoint. Stripe's documentation states that a webhook endpoint can receive the same event more than once, and recommends logging the IDs of the events you have processed and skipping any already logged. Stripe also provides a mechanism — idempotency keys — so that a request you resend after a connection error is not carried out twice. If the receiving side doesn't build these in, the result is duplicate orders or duplicate charges. The details are in Why double-charging happens when you let AI build payments, and how to prevent it.

Fictional example 3: a leaked key, and someone else's usage on your bill

The key (API key) used to call an AI text-generation service was sitting in code sent to the browser, or in a public GitHub repository (where the code is stored). A third party used that key heavily, and the next month an invoice arrived for usage nobody recognised.

If a key was in the code, deleting it from the file is not enough. GitHub's documentation likewise says the first step for a leaked secret is not scrubbing it from history but revoking or rotating that key. Why keys end up visible is explained in why the "API key" AI wrote is visible to everyone else, and how to set spending caps in The bill spiked after launch — usage-billing incidents, and how to cap them.

Fictional example 4: no errors, but the invoice amounts are wrong

In an internal invoicing system, the discount calculation missed a condition, and some customers were under-billed for several months. No error ever appeared, so nobody noticed.

This "it runs, but it's wrong" state never shows up as an error alert. It is covered in depth in "It runs, but it's wrong" — the quietest AI-generated bug.

None of these can be found by clicking through a demo.

3. What you can do yourself, when to bring in a specialist, and when you don't need to

When you don't need to bring anyone in

If it is a prototype used only by you or your circle — no login, no stored personal data, no payments, no entry point that can be called directly from outside (a public API), no file upload field, and no calls to a usage-billed service (an AI API, for instance) — in other words, nothing that can be stolen, broken or billed to you — there is little reason to pay someone to look at it. The free self-check and fixes made with AI are usually enough.

But "there's no login, so we're fine" does not follow. Any one of these gives an attacker something even without a login:

• There's an entry point (API) that can be called directly: calling it directly, without the screens, can do things the screens never offer
• There's a file upload field: anyone may be able to put anything there
• It stores data (contact-form submissions, say): the question becomes whether that storage is readable by anyone. Supabase's documentation, for example, says the publishable key placed in the browser can be read by anyone, so it only reaches what Row Level Security (RLS) allows — and asks you to enable RLS on every table. What protects the data is not a login screen but the database's settings
• It calls a usage-billed service (AI text generation, for instance): no login means anyone can call it, and the bill comes to you

The test is not "is there a login?" but "is there anything that can be stolen, broken or billed to me?" Whether you need a human security review at all is worked through, by the kind of data you hold, in the decision table of Does the app you built with AI need a human security review?.

What you can do yourself

• The free 20-question self-check: the pre-launch self-check needs no signup. Answer yes / no / don't know, and you get results ordered by danger plus fix instructions you can paste straight into your AI tool. Your answers are never sent out of your browser and never stored.
• Change the number in the URL: with two accounts, check that neither can see the other's data (the test for example 1).
• Check that no key is exposed in the browser or on GitHub: if one is, don't just delete it — replace it (example 3).
• Set spending caps on usage-billed services: that puts a ceiling on the damage if a key leaks.
• Write down what must always be true: about ten sentences like "a cancelled booking never counts as revenue". Only you can write this, and it is the most useful input for fixing and rebuilding alike.

Asking AI to "fix everything" is not wrong in itself. But AI prioritises getting the instruction in front of it through, and it doesn't guarantee the whole stays consistent. If every fix it makes breaks something else, that is a sign you have reached the limit of doing it yourself.

When to bring in a specialist

If any of these apply, it is worth having a person look before real customer data goes in (or soon, if it already has):

• You hold other people's personal data as part of the business
• There are payments, or money or stock moves through it
• Several companies' or teams' data live in one system (selling to businesses)
• It runs work that stops the business when it stops — invoicing, ordering, payroll
• The self-check left many "don't know" answers
• You want to decide whether to fix or rebuild (that can't be decided without reading the code)

You can see what a human review delivers in advance in the sample report for the pre-launch light review, built on a fictional app. If you want to go as far as deciding whether to fix or rebuild, ask for a consultation or review in which someone reads the code and lists what to fix first, in priority order, then fit those findings to the table in section 1. And if you will keep developing with nobody in-house to make the technical calls, there is also the option of an external development lead (fractional CTO / technical advisor).

4. What pushes the cost up or down

For fixing and rebuilding alike, there is no figure until the scope is set. What moves the scope most is not the amount of code but these factors:

How prepared you are moves the figure too: whether what correct behaviour is has been written down, whether the production code matches your copy, whether there is a test environment. Those are things you can sort out before asking, so they are tabulated in "What drives the quote" in Preparing to consult or hand over to an engineer.

This article does not put a figure on fixing or rebuilding. The ways to engage, and the published starting prices (all "from" figures), are these:

• Pre-launch light review (about 3 business days): a verdict on whether the critical holes that must not go live are present — missing RLS (the database's per-row access limits), exposed secrets, missing ownership checks, and payments processed twice — plus a report written for non-engineers, prioritised fix instructions you can paste into your AI tool, and a 30-minute call to walk through it. The price is on the sample report page. The review is deliberately limited to critical holes, so it does not include a fix-or-rebuild decision.
• Having someone read your existing code: the spot technical consultation on the pricing page (from ¥7,000/hour; it can also cover reviewing existing code; the first 30 minutes are free).
• Implementing fixes, or rebuilding: quoted once the scope is agreed, in the form of the plans on the pricing page. Fixing security holes found in an audit is also offered as the Embedded fix tier, which bundles the audit, implementing the fixes and a re-audit (see the audit page).

5. Before you ask for help

The full preparation, with a checklist, is in Preparing to consult or hand over to an engineer. The minimum is these four:

• Where the code lives (GitHub or similar) and the URL that is live in production
• A list of the outside services it uses (payments, email, AI, database and so on)
• A list of the names of the environment variables (settings that keep keys and connection details outside the code). Do not hand over the key values themselves
• Your note of what must always be true (section 3)

If you are uneasy about showing the code, signing a non-disclosure agreement (NDA) first is normal. In my case, an NDA can be signed before any commercial discussion. How to choose who to hire in the first place is covered in the complete guide to outsourcing system development.

6. For engineers: the five places AI-generated code breaks in production

From here on, this section is for engineers (or for when you ask an engineer to explain). You can skip it; everything above is enough to make the decision.

As I see it, there are patterns in where AI-generated code falls short. These are the five places where I concentrate verification in my own work.

① Boundary input validation

AI tends to treat external input — forms, APIs, webhooks, files — as if it were always correct. Parse it at the boundary with a typed schema (Zod, if you are on TypeScript) and keep anything that fails out of the logic.

② Error handling

It writes the happy path (when things go well) but leaves out what to do on failure. Add timeouts on outside APIs, retries with exponential backoff limited to idempotent operations, and failure logging with alerts. The 2025 edition of the OWASP Top 10 includes "Mishandling of Exceptional Conditions" as A10.

③ Authorization

AI tends to implement authorization as an if-statement in the UI. Unless it is enforced on the server or in the database, calling the API directly gets straight through. A01 in the OWASP Top 10:2025 is "Broken Access Control", and IPA's guide 安全なウェブサイトの作り方 (roughly, "how to build a secure website") also covers missing access control and authorization control.

On Supabase, the publishable key (formerly the anon key) is assumed to be in the browser, and RLS is what protects the data. A secret key (formerly the service_role key) bypasses RLS on any request that carries no user access token (the credential that identifies a signed-in user), so it must never reach the browser.

④ Concurrency and idempotency

A read → compute → write-back sequence races when it runs concurrently. For payments, stock and bookings, make it atomic with unique constraints or conditional updates. Guard against double-processing incoming notifications by recording the IDs of events already handled (the approach Stripe's documentation describes), and attach an idempotency key to create requests sent to outside APIs. The buyer-side version of this is How to build a payment system that prevents double charges, and a procurement checklist.

⑤ Tests

Ask AI for tests and it will write them, but whether they probe boundaries and failure cases is another matter. Tests that only retrace the happy path don't stop regressions when the code changes. Turning the "what must always be true" list from section 3 directly into tests is the order that pays off most.

The order of hardening

The AI-built app (keep the skeleton)
   │
   ├─ 1. Revoke/rotate leaked keys; move secrets to server-side env vars / a secrets manager
   ├─ 2. Enforce authorization on the server / in the DB (ownership checks, RLS, tenant boundaries)
   ├─ 3. Make money and stock handling idempotent and atomic (record event IDs, idempotency keys)
   ├─ 4. Put typed input validation at the boundary
   ├─ 5. Add error handling, timeouts, failure logging and alerts
   └─ 6. Turn "what must always be true" into tests and run them in CI every time
   ↓
A state that holds up in production (not "completely secure")

Checking the table's signals in the code

Verification first

NIST's SSDF (SP 800-218) lists reviewing and analysing human-readable code (PW.7) and testing executable code (PW.8) as practices, and notes that automated methods lower the effort and resources needed to detect vulnerabilities. It also positions the SSDF as a common vocabulary that software purchasers can use with suppliers during acquisition.

Two examples from my own projects. Both are things I built from the start — neither is a case of taking over an app someone built with AI. What they show is the verification machinery.

• AI video-localization platform: backend test coverage of 100% made mandatory in CI (the build fails if it isn't met), mypy strict kept at zero errors, and a design that resumes from the last completed segment even when a spot GPU is forcibly stopped.
• B2B SaaS for lumber distribution (winner of the METI Minister's Award): four rounds of self-conducted security audits, and a self-conducted penetration test across 15 real roles that confirmed 0 missing-authorization findings across all 221 endpoints.

7. Limits and counter-examples

• The table is a guide based on my judgment. The final decision is made from findings based on reading the code. Two systems in the same spot on the table can get different answers depending on the business — how long it can't be stopped, the budget, the plans ahead.
• Hardening does not make it "completely secure". What it does is close the known serious risks and leave behind the machinery (tests, and CI — running them automatically on every change) that keeps them from coming back.
• Sometimes a rebuild is the right answer. When requirements have changed so that you are building something different, and there are no users and no production data yet, rebuilding can cost less than fixing.
• Tools change. As with the Supabase key names mentioned in section 6, AI tools and cloud services change, so confirm configuration details in the official documentation.
• Legal judgements are outside this article. Obligations such as reporting a personal-data leak should be confirmed with the Personal Information Protection Commission's guidance and a professional.

8. Checklist

• I will decide fix vs rebuild from findings based on reading the code, not from a demo
• I know whether the problems are concentrated in specific places or scattered everywhere
• I have checked that the way data is stored and who may see what match how the business actually works
• I have written down about ten things that must always be true
• I know whether production holds real customer data, and whether there is a test environment
• I have checked that no key is exposed in the browser or on GitHub, and replaced any that was
• I have set spending caps on usage-billed services
• I have noted the items the free self-check left as "don't know"
• I have confirmed the code and cloud accounts are in my (my company's) name, and that the stack is one other people can maintain

Summary

Building it with AI is not the problem. Running it in production without checking it is.

In my judgment, fixing it and keeping it is in most cases the cheaper, faster choice. A full rebuild becomes a cheaper candidate when problems pile up — the way data is stored or the rules for who may see what are fundamentally wrong, or what you are building has itself changed — and there is no real customer data in production yet. If customer data is already there, rebuild only the faulty layer, data migration included. Make the call from findings based on reading the code, not from the impression a demo leaves. The first step is the free self-check and writing down what must always be true.