# Does the app you built with AI need a human security review? — who needs one, who doesn't, and what it costs, when to do it and how to ask > Should you pay a person to review the app you built with Claude Code, Cursor or Lovable? A decision table based on the data you hold, what an AI self-check covers and what it can't, when in the launch cycle to ask, typical prices and what drives them, and how to get a review without handing over keys or customer data. - Published: 2026-10-12 - Author: 友田 陽大 - Tags: バイブコーディング, セキュリティ, 脆弱性診断, 個人開発, Supabase, 発注 - URL: https://tomodahinata.com/en/blog/ai-app-security-review-need-cost-timing - Category: Running the app you built with AI - Pillar guide: https://tomodahinata.com/en/blog/ai-app-pre-launch-guide-for-non-engineers ## Key points - Whether you need a human review depends on what your app holds, not on how well it's built. If only you use it, or it stores nobody else's data, free checks plus fixes made with your AI tool are often enough - Claude Code's /security-review and Supabase's Security Advisor are good at problems whose answer is in the code — exposed keys, RLS (the setting for who may see which rows) left switched off. Whether a given person may see a given row is answered outside the code, in your head, so a person has to check it against the code - In my own study of 1,000 public repositories, 9.2% of the 994 that ship RLS had a policy letting any logged-in user read every row. Whether that is a hole depends on what the app is meant to do, which no tool knows - If you are going to ask, the cheapest and most effective moment is before real customer data goes in. Starting payments, adding file uploads and selling to companies are the same kind of milestone - A review doesn't need your key values or customer data. Code, the database structure without data, two test accounts, and a table of who may do what are enough to start --- **The short answer: whether you should pay a person for a security review of an app you built with AI — Claude Code, Cursor, Lovable and the like — depends not on how well it's built, but on what it holds.** If only you use it, or it stores nobody else's data, free checks plus fixes made with your AI tool are often enough. If you are about to hold other people's personal data for your business, or money is about to move, it's worth having a person look at it once — **before** that happens. This article starts with a decision table so you can see which side your app is on. Then it covers when to ask for a review, what it costs, and what to hand over. A disclosure first: I sell a paid "pre-launch light review". That is exactly why **the cases where you don't need one come first**. This is not an argument that everyone needs a paid review. --- ## 1. The decision table — does your app need a human review? You need a human review when you hold other people's personal data or money. The question to ask: if this leaked, who would you have to apologize to? If the only answer is yourself, there's no rush. | Your app | Recommendation | Why | | --- | --- | --- | | Used only by you, family or friends | No human review needed | No one else's data to leak | | Public, but no login and nothing stored (a calculator, a landing page) | No human review needed; just check for exposed keys | The only thing to steal is an API key (a master key to an outside service) | | Other people log in and you store things like their email address | Free check plus AI fixes; ask a person only if something stays unclear | You must confirm nobody can see anyone else's data, but most of it you can check yourself | | You hold customer lists, bookings, inquiries — other people's personal data, for your business | A human review **before real data goes in** | A leak now hurts people other than you | | You take payments, or money or stock moves | A human review **before payments start** | Double charges and fraudulent orders cost more to undo than to prevent | | You handle medical history, health status or other especially sensitive data | A human review is strongly recommended | One of the most serious kinds of leak (see below) | | Several companies or teams share one app, or you sell to businesses | An audit that covers the design | The wall between companies' data can't be verified by a tool | Note that the duty to report to the regulator is not only about the lower rows. Japan's Personal Information Protection Commission lists four kinds of leak a business must report to it: 1. Leaks that include sensitive personal information (such as medical history) 2. Leaks that could cause financial harm if misused (such as credit card details) 3. Leaks that may result from an act with an improper purpose, such as unauthorized access 4. Leaks involving more than 1,000 people **Only the fourth has a headcount threshold.** The deadlines are a preliminary report within 3–5 days of discovery and a final report within 30 days (60 days where an improper purpose may be involved). So even if someone exploits a database misconfiguration and takes only a few people's records, a report may still be required. **That duty can arise as soon as you hold other people's personal data — row 3 of the table included.** The lower rows rank higher not because the duty differs, but because the damage from a leak is heavier. Check the exact requirements on the Commission's page, listed in the sources at the end of this article. Other legal points are collected in [the legal checklist for apps built with AI](/blog/ai-app-legal-checklist-japan). --- ## 2. Isn't asking Claude Code "is this safe?" enough? **It covers about half.** AI and free tools are good at finding problems whose answer is in the code. What they miss are problems whose answer lives outside the code — in your head. ### What you can leave to AI and tools - **Hard-coded keys**: whether an API key is written into the code or sent to the browser - **RLS left switched off**: RLS (the database feature that decides who may see which rows) disabled on a table. Supabase's Security Advisor flags a public table without RLS as an error, warning that anyone with your project URL can read, edit and delete all of its data - **Common injection flaws**: well-known patterns such as an input field rewriting a database command, or someone else's script being embedded in your pages - **Outdated components**: libraries with known vulnerabilities Claude Code has a `/security-review` command, which its documentation describes as a security pass over the changes on your current branch (a branch is a separate line of work). Because it looks at changes, it may never have looked at the app as a whole. If you want the whole thing checked, ask explicitly: "review this entire repository from a security perspective." If you or someone on your side can read code, the developer-facing [vulnerability assessment of AI-generated code (vibe coding)](/blog/ai-generated-code-vulnerability-assessment-vibe-coding-security-guide) walks through closing these issues step by step. ### What a person needs to check The companies building these tools say this themselves: - Anthropic: automated security reviews "should complement, not replace, your existing security practices and manual code reviews" - Lovable: its tools help identify common security issues, "but they cannot guarantee complete security", and "for apps handling sensitive data or critical functionality, consider an additional professional security review" Why the complement is needed is easiest to see with ownership checks. User A changes the number at the end of a URL and can now see user B's booking. That is a classic incident (called IDOR, or BOLA). Yet the request your app receives is perfectly well-formed: A is logged in and the format is valid. **What's wrong is that it breaks the rule "A may only see A's bookings" — and that rule is not in the code. It's in your requirements.** If an AI wrote the code from its guess at your requirements, the same AI checking it against the same guess won't find the guess was wrong. (That last point is my own view, but it points the same way as the two statements above.) ### Why "RLS is on everywhere" still leaves a decision — from my study of 1,000 repositories RLS mistakes have caused publicly reported incidents. In 2025, a vulnerability in apps built with the AI app builder Lovable — inadequate RLS that let logged-out visitors read and write arbitrary tables — was registered as CVE-2025-48757 (severity CRITICAL; the vendor disputes it). So is switching RLS on enough? I statically analyzed the RLS settings of 1,000 public GitHub repositories that use Supabase. **Of the 994 that ship RLS, 91 (9.2%) had a policy letting any logged-in user read every row.** That 9.2% does not mean 9.2% are vulnerable. For data every logged-in user is meant to see, such as a list of announcements, that policy is correct. **It means someone has to confirm the policy is intended** — and only someone who knows what the app is supposed to do can. A missing RLS policy is something a tool can find. Whether the conditions on an existing policy are right is a separate question that remains. The same study found that a different rule in my own scanner had wrongly flagged about 68% (54) of its 80 findings as dangerous. I only found that by reading each actual policy one by one. Tools can overstate (how much they miss, this study did not measure), and sorting real warnings from false ones is a person's job too. The full method and its limits are in [the Supabase RLS field study](/aegis/field-study). ### If you ask your AI, ask like this Ask "is it secure?" and the AI tends to say "yes, it's handled." Instead, **have it produce a table that you check against the right answer.** ```text For every table in this app, build the table below. Do not fill it in by guessing: cite the code, SQL or RLS policy that supports each cell (file name and line). | Table | What a logged-out visitor can do | What logged-in user A can do to user B's rows (read/create/update/delete) | What only admins can do | Evidence | Write "unknown" in any cell you cannot find evidence for. Also list every place the service_role key (the all-powerful key) is used. ``` **If even one cell in the "what user A can do to user B's rows" column differs from what you intended, that's what needs fixing.** If many cells can't be checked, or the table is mostly "unknown", that is where paying a person starts to be worth it. --- ## 3. When to ask — milestones by launch stage **If you are going to ask, the cheapest and most effective moment is just before real customer data goes in.** Before that, fixing is just a settings change. After it, the fix itself needs care, and you also have to find out whether any data may already have been exposed. | Stage | Human review | Notes | | --- | --- | --- | | Prototype that only you touch | Not needed | Money spent here is often wasted when the requirements change | | Friends-only beta with fictional data | Usually not needed | Closing holes with the free check now makes a later review cheaper | | **Just before real customer data goes in** | **The most effective milestone** | If you're in the lower three rows of the decision table, do it here | | Just before you start taking payments | Recommended | See also [the article on preventing double charges](/blog/ai-app-payment-double-charge-prevention) | | When you add file uploads | Recommended | "Can someone open another person's files?" becomes a new question | | When you sell to a company or receive a security questionnaire | Recommended | You may be asked whether a third party has reviewed the app | | After a large change to permissions | Recommended | Invitations, teams and admin features tend to break conditions when rebuilt | | Already live with real data | Now | Start with the free check. If a key was exposed, rotate it before anything else | How to rotate a key is covered in [why other people can see your API key](/blog/ai-app-api-key-exposure-explained). --- ## 4. What a review actually looks at For an app built with AI, reading the code and settings tends to beat probing from the outside on cost-effectiveness — that is my view. First, the terms: - **Vulnerability assessment**: sending requests to a running app from the outside to find holes. Often without looking inside - **Source code review**: reading the code and the database settings - **Audit (design review)**: both of the above, plus checking whether the design of who may do what is itself sound An app built with Claude Code or Lovable has all of its code and database settings in your hands. That is why **reading the code and settings tends to reach more holes for less money than probing from the outside alone.** Before launch, the minimum to have checked is these four, in order of danger: 1. **Database permissions (RLS)**: can anyone see another person's rows? Explained in [RLS explained for non-engineers](/blog/supabase-rls-explained-for-non-engineers) 2. **Exposed keys**: is the all-powerful key being sent to the browser? 3. **Ownership checks**: can being logged in alone let someone act on another person's data? See [the gap between "looks logged in" and real authorization](/blog/ai-generated-login-vs-real-authorization) 4. **Duplicate payment processing**: can the same payment be processed twice? For an app several companies use, the scope widens to the wall between companies' data, every path that uses the all-powerful service_role key, and abuse of business rules (stacking discounts, for example). The developer-facing breakdown of what an audit covers is [What a security audit actually looks at](/blog/nextjs-supabase-security-audit-scope-when-needed-guide). --- ## 5. Cost — typical prices and what pushes them up **Prices differ by an order of magnitude depending on scope.** Comparing the numbers alone will mislead you, so read them alongside what each one does and doesn't cover. The table below is public information as of October 12, 2026, all from the Japanese market, and is not a quality ranking. | Type | Example published price | Scope | Source | | --- | --- | --- | --- | | Free tools | No extra cost (within the plan you already use) | Supabase Security Advisor, Claude Code `/security-review`, Lovable's built-in scanning, etc. Nothing is checked against your requirements | Each vendor's documentation | | AI analysis plus human verification, priced per line | 1 yen per line (excl. tax), minimum 10k yen | Six areas including security; normally within one week | ForestX's "Code Dock" press release | | "Clean-up" service that includes review and refactoring | Light plan from 300k yen (up to 5,000 lines, excl. tax) | Review, refactoring, a report and a review meeting; fixes and tests are in higher plans | IT Leaders article on Triview Innovation's "Vibe Seisho" | | Enterprise web vulnerability assessment | Tool-based 100k–300k yen / manual 500k–1.5M yen / hybrid 600k–2M yen | General market rates for corporate systems | cloudpack column | | **My own service** | Pre-launch light review from ¥29,800 / spot review from ¥98,000 / standard audit from ¥280,000 / hands-on fix implementation from ¥680,000 (excl. tax; the English pricing page lists prices in USD) | The light review is deliberately narrowed to "are there any critical holes?" | [Pricing and scope](/aegis/audit) | What pushes a price up, roughly: - **The number of tables and user roles**: every extra role (regular user, admin, staff…) multiplies the combinations of who may do what - **Whether several companies share the app**: the wall between them has to be checked - **Payments, file storage, and integrations with outside services (webhooks)** - **Functions inside the database**: especially ones that run with elevated rights (`SECURITY DEFINER`) - **Whether the reviewer sends requests to your live app**: more work than reading the code alone - **Whether fixes, and a re-check after fixing, are included** - **A rushed deadline** Count which of these apply to your app before you ask for a quote, and the conversation goes faster. How to read web-app assessment quotes in general is covered in [the cost of a web application vulnerability assessment](/blog/web-application-vulnerability-assessment-cost-pricing-guide). --- ## 6. What you can do for free first Work through these four in order. Sometimes you'll find you don't need a review at all; if you do, the scope will be narrower and cheaper. 1. **The 20-question self-check**: answer yes / no / not sure in [the free pre-launch self-check (no signup)](/vibe-coding/checkup) and you get your risks in order of danger, with fix instructions you can paste into your AI tool. Your answers never leave your browser 2. **Supabase's Security Advisor**: check the dashboard for warnings. Fix any ERROR first 3. **Claude Code's `/security-review`**: have your AI fix what it reports 4. **The prompt from section 2**: check the table it returns against what you intended **If your app is in the top three rows of the decision table and nothing disagrees after these four, you can stop here.** If it's in the lower rows, or cells remain you couldn't check, go on to the next section. --- ## 7. What to hand over, and what not to **A review doesn't need your key values or customer data.** But inviting someone without thinking can grant far more access than you meant. Here is how to share, based on what each service's official documentation says. | What | Recommended way | Watch out for (from the official docs) | | --- | --- | --- | | Code | Invite the reviewer on GitHub | **Repositories owned by a personal account have no read-only collaborator option.** Collaborators can also push. For read-only access, move the repository to a free organization and invite with Read, or agree up front to remove the invitation when the work ends | | Database structure | The `supabase/migrations` folder, or a file produced by `supabase db dump` | A Supabase Read-Only member can run SELECT queries in the SQL Editor — **so they can see real customer data**. Project-scoped roles exist only on the Team and Enterprise plans. `supabase db dump` outputs the structure only by default, with no data, but it excludes the `auth` and `storage` schemas, so describe your file-storage permissions separately. If the command is unfamiliar, ask your AI: "use supabase db dump to output only the structure, without data" | | Hosting (Vercel) | Usually unnecessary; the list of environment variable names and your production URL are enough | Vercel's team roles are a Pro / Enterprise feature (Pro includes a read-only Pro Viewer seat). There's no need to force access | | Key and password values | Don't hand them over | If truly needed, share through a password manager's sharing feature and rotate afterwards. Never paste them into email or chat | | Test accounts | **Two** (user A and user B) | You need two to check that A can't see B's data. Add an admin account if you have admins | | A table of who may do what | Fill in the template below | The single most valuable document, and only you can write it | | Non-disclosure agreement (NDA) | Sign before sharing code | At the free-consultation stage you can often proceed with screens and explanations, without code | The who-may-do-what table can be as simple as this. It doesn't have to be perfect — write "not sure" where you aren't. Those cells are exactly what the review should check. | Data | Logged-out visitor | The owner | Others in the same company or team | Admin | | --- | --- | --- | --- | --- | | e.g. bookings | Can't see | See, create, cancel | See | Everything | | e.g. profile | Can see the name only | See, change | See the name only | See | **You can get a review without a staging environment** (a test copy separate from production). A review that reads code and structure never touches production. If the reviewer will send requests to your live app, agree in writing on the scope, time window and test accounts used, and limit it to methods that don't modify real customer data. General hand-over preparation, such as listing the outside services you use, is in [preparing to consult or hand over to an engineer](/blog/handing-over-ai-app-to-engineer). --- ## 8. Choosing a reviewer — seven questions to ask first Which provider is "best" depends on your app and budget. Instead, here are questions that work with any provider, whether or not you choose me. 1. **Can you state in writing what you will check and what you won't?** Anything outside that scope isn't covered by the review 2. **Does a person verify each automated finding?** As section 2 showed, tools overstate. An unverified list leaves you to do the sorting 3. **Does each finding come with a severity and steps to reproduce it?** 4. **Will the explanation make sense to someone who can't read code, and will it include instructions for fixing?** In a form you can paste into your AI tool, you can fix things yourself 5. **Is a re-check after fixing included, or charged separately?** 6. **Have you reviewed apps built like mine, with Supabase or Next.js?** 7. **Will you send requests to my live app? If so, within what scope, and how do we agree on it?** Conversely, be wary of anyone who guarantees "this is now 100% safe." A review reports the problems found within an agreed scope; it can't vouch for what's outside that scope or for future changes. For an objective reference point in Japan, IPA's list of services meeting its information security service standards has a vulnerability assessment category (the list states explicitly that it "does not imply any guarantee by IPA"). To see what my pre-launch light review delivers, look at [the sample report built on a fictional app](/vibe-coding/sample-report). Whether or not you ever order one, it shows what review results look like. --- ## Summary — building it with AI is not the problem There is nothing wrong with having built your app with AI. AI genuinely makes building faster. The problem comes when you ask the same AI that built it "is this OK?" and feel you've checked even the part only you can answer: who may see which data. - If the only data at stake is your own, free checks and your AI tool are enough - If you hold other people's personal data or money, have a person check it against your rules once, before real data goes in - When you ask, hand over code, structure, two test accounts and a who-may-do-what table — not keys, not data Check everything that can be automated yourself, and hand a specialist only the boundaries worth a person's time. That is the cheapest and most reliable order.