AI-written code in production — what we find most often in audits

Vibe coding has reached production. We clean up after it in audits

The past year has changed the way software gets built. Teams — including those without experienced seniors — ship working features at a pace that was out of reach not long ago. "Vibe coding", meaning writing code by describing your intent to an AI assistant and accepting whatever it generates, has stopped being a Twitter curiosity. It has landed in the repositories we audit.

The result? Code that works in the demo but has never been read by a human with understanding. And when we come into such projects for an audit, we see a repeatable set of problems. Not because AI "is stupid" — the models generate surprisingly correct code. The problem is that they generate it locally correct, but without the context of the whole system and without responsibility for the consequences.

Below are five patterns we most often find in audits of AI-generated code. With specifics, and with how to catch them before your users or a security auditor do.

1. No input validation

By far the most common finding. Asked for "an endpoint that saves an order", an AI assistant will write an endpoint that saves an order. Exactly that — and nothing more. It will not ask what happens when quantity is negative, email is empty, and price arrives as the string "0.00; DROP TABLE".

A typical generated handler looks like this:

app.post('/api/orders', async (req, res) => {
  const { userId, items, total } = req.body;
  const order = await db.orders.create({ userId, items, total });
  res.json(order);
});

It works. In a demo with a single valid request it is flawless. In production total is controlled by the client (you can buy for a penny), items can be anything, and the missing userId check is a ready-made IDOR. The same applies to the database layer: fields left nullable "just in case", no constraints, no unique indexes.

How we catch it: we look for the system's boundaries — controllers, queue handlers, webhooks — and check whether every input passes through a validation schema (Zod, class-validator, Pydantic, anything). The rule of thumb: data from outside is hostile until a schema proves it safe. AI does not adopt that assumption by default.

2. Secrets committed to the repository

The Stripe API key. The production database connection string. The S3 token. All pasted straight into the code or into a committed .env, because "just to make it work for now".

This is not AI's fault alone — developers have always done it. But vibe coding scales the problem. An assistant will happily generate const stripe = new Stripe('sk_live_...') with the key supplied in the prompt, and a junior developer has no reflex to move it into environment variables and add .env to .gitignore.

A key fact many people miss: removing a secret in a later commit does not remove it. The key stays in the Git history and is available to anyone with access to the repo — and in a public repo it gets indexed by bots within minutes.

How we catch it: scanning the history (gitleaks, trufflehog), reviewing configuration and CI/CD. A live secret found means one thing: rotate immediately, because you have to assume it leaked. A commit that "removes" it from the code is a false sense of security.

3. The N+1 query problem

A pattern that will not fail an eyeball code review and will not fail tests against an empty database. It only blows up under real traffic — and then it is expensive.

AI is excellent at generating code that is readable to a human: fetch the users, and for each of them fetch their orders. The problem is that "for each of them" means a separate database query:

users = User.objects.all()
for user in users:
    # a separate SELECT for EVERY user
    print(user.orders.count())

With 10 users — unnoticeable. With 10,000 — that is 10,001 queries on a single HTTP request, timeouts and a database on its knees. The same happens with ORMs (Prisma, Hibernate, ActiveRecord) when lazy loading of relations ends up inside a loop.

How we catch it: we turn on query logging and look at the number of SQL statements per request, and review loops iterating over collections while touching relations. The fix is usually simple (select_related/prefetch, include, JOIN) — provided somebody is looking at all. The assistant will not optimise this itself, because it cannot see your load profile.

4. Dead tests — green, but checking nothing

The most insidious finding, because it creates a false sense of security. The repository has tests, CI is green, coverage looks decent. And yet the tests protect against nothing.

Asked for tests, an assistant often generates code that:

• tests mocks instead of logic — the assertion checks that a mocked function returned what it was told to return;
• has no meaningful assertions — it calls a function and only checks that it "did not throw";
• is tautological — expect(sum(2,2)).toBe(sum(2,2));
• covers the happy path and only the happy path — zero edge cases, the very ones points 1–3 are about.

it('creates a user', async () => {
  const save = jest.fn().mockResolvedValue({ id: 1 });
  await createUser({ save });
  expect(save).toHaveBeenCalled(); // checks the mock, not the logic
});

This test also passes when createUser saves garbage or ignores validation. Coverage goes up; real safety does not.

How we catch it: we read the assertions, not the test count. A good check is mutation testing (Stryker, PIT) — we deliberately break the logic and see whether the tests notice. If the mutant survives, the test is dead.

Why this happens — and why it is not an argument against AI

Worth saying plainly: we are not opponents of AI-generated code. We use these tools daily and they genuinely speed us up. The point is to understand where the limits of their competence lie.

An AI assistant optimises for "does this look like a correct answer to my question". It does not optimise for security, performance under load, consistency with the rest of the system or maintainability two years from now. Those things require context and accountability, which a model simply does not have. Generated code is like an excellent first draft from a very fast junior — and that is how it should be treated in review.

The problem is not that somebody used AI. The problem is that the generated code reached production without that review.

How to protect yourself

A few things worth putting in place if a lot of code in your team is written with AI:

1. Validation at system boundaries as a standard. Every external input through a schema. No exceptions.
2. Secret scanning in CI. gitleaks as a blocking pipeline step — prevention is cheaper than rotation after a leak.
3. A query budget per request. Monitor the number of SQL statements and alert above a threshold. N+1 shows up immediately.
4. Mutation testing on critical paths. At least where a bug is costly — payments, authorisation, personal data.
5. Review on the assumption that the author does not know the context. Because with AI-written code that is literally true.

When an audit is worth commissioning

If your product grew over the past year mainly by shipping features fast with AI — and nobody has systematically reviewed that code for security and performance — this is exactly the moment when an audit pays back fastest. Not because "AI made a mess", but because the pace of writing code has outrun the pace of verifying it.

At Mobilesoft we run such audits regularly — from security and performance reviews to assessing test quality and technical debt. The outcome is not a list of reproaches but a prioritised remediation plan: what poses the greatest risk, what can be fixed quickly, and what is worth rebuilding before it grows further.

If you want to find out what is sitting in your code before somebody else does — let's talk.