What Is Vibe-But-Verify Coding?

Everyone is talking about "vibe coding" right now. If you haven't heard of it, tell me... under which rock have you been dwelling?

You know what everyone has also been talking about? How great vibe coding is. Oh, and also how awful it is.

But what if we could get the best of both worlds? Enter what I'm calling "vibe-but-verify coding".

What's Wrong with Vibe Coding?

Vibe coding, for the rock-dwellers out there, is when you "forget the code exists" and let AI do all the coding for you. You tell it to do the thing, and it does the thing, with much greater ability and speed than you could have (in theory).

The problem is when it doesn't do the thing, or does it in a dumb way. Coders and non-coders alike get stuck when the AI traps itself in error spirals or repeatedly breaks existing functionality, never finishing that last 30% of the project.

There are roughly three general problems with vibe coding:

1. The AI doesn't do what you want it to.
2. The AI does what you want it to, but also breaks things that were already working.
3. The AI does what you want it to, but in a way that's dangerous or that's broken for some users.

Vibe-but-verify coding aims to rectify these problems, without sacrificing agility or ability.

Introducing Vibe-But-Verify Coding

Think of vibe-but-verify coding as a hybrid of traditional coding and vibe coding.

The idea is for the human (that's you) to write automated tests to verify that the code the AI is writing actually does what it should.

In vibe coding, there's one rule:

• Ignore the code; let the AI write it entirely.

In vibe-but-verify coding, there are two rules:

• Ignore the production code; let the AI write it entirely.
• Write the test code yourself; don't let the AI touch it.

In this way, you're forcing the non-deterministic output of the AI to be somewhat deterministic, where it counts. You can write different kinds of tests to enforce what matters (including security and performance), but leave the rest for the AI to work out.

In other words, you can ensure that the AI does the right thing, does it in the right way, and continues to do so. Because that's what automated tests are meant to do in the first place.

Do the Thing

Let's see an example.

First off, it's a good idea to explicitly tell the AI not to mess with your tests, whether in any prompts you write or in a Cursor/Windsurf rule.

I have a line in a Cursor rule that simply says:

Do not edit any tests.

Now, vibe-but-verify coding can take many forms. In one such form, you write the tests first, and then have the AI write the code to pass those tests (you might call this Test-Driven AI Development, if you're a nerd).

For example, here's a test I wrote in a side project:

import { expect, test } from "@playwright/test"

test("renders correctly", async ({ page }) => {
  await page.goto("./")

  await expect(page).toHaveTitle("Competition Helper")
  await expect(page.getByRole("heading", { name: "Competition Helper" })).toBeVisible()
  await expect(page.getByRole("textbox", { name: "Media URL" })).toBeVisible()
  await expect(page.getByRole("spinbutton", { name: "Run Length (seconds)" })).toBeVisible()
  await expect(page.getByRole("spinbutton", { name: "Lead-up Time (seconds)" })).toBeVisible()
  await expect(page.getByRole("spinbutton", { name: "Reminder Time (seconds before end)" })).toBeVisible()
  await expect(page.getByRole("button", { name: "Start" })).toBeVisible()
  await expect(page.getByRole("button", { name: "Reset" })).toBeVisible()
})

After writing that, I literally told the AI:

Implement the changes necessary to make the tests pass

And it did.

I've gotta be honest, I was kinda surprised and excited to see this actually work the first time.

More complicated tests will likely require some more finagling, but the process and the results remain the same.

If you like, you can also add a rule that says something like:

After making changes, run the tests again. If they don't pass, fix it.

As you can see, with this approach we're essentially creating an enforced feature spec. You can be sure that if your tests pass, the thing has been done.

This is also why you shouldn't have the AI write the tests; that would be like the blind leading the blind. The AI doesn't tell itself what to do, the human does (until the uprising, at least).

To Infinity and Beyond

If vibe coding can take you from nothing to working prototype in minutes, how far could vibe-but-verify coding take you? Perhaps past that last 30%.

Maybe this is how it's meant to be, human and AI working together in a symbiotic relationship to create something that neither could've created alone.

But what about non-coders, the target audience for vibe coding? Are they just left out in the cold? Maybe not. Maybe automated tests don't have to be written in code. Maybe new tools will be created to fulfill this need without code.

Whatever happens, remember that technology has existed for approximately 0% of human history (rounding down). All this stuff is brand new. No one knows what the future holds.