The Most Expensive Bug in Your Software Isn't Written in Code

It's written in the boardroom.

Over the last 12 months, I've found myself having the same conversation over and over again. Different companies, different industries, different products. Exactly the same problem. HealthTech, financial services, education, enterprise software.

At first, I thought it was probably just startup life. Then I wondered whether it was because I spend so much time around people building digital products. Now I don't think it's either. I think we're watching the relationship between leadership teams and engineering change in real time, and I'm not sure most organisations have realised it's happening yet.

For years, there was a fairly simple relationship between commercial teams, product and engineering. Customers asked for something. Commercial spotted the opportunity. Product worked out what was actually needed. Engineering estimated the work. There were disagreements, obviously. There always have been. But if an experienced development team came back and said "that's going to take three months," there was acceptance that they understood what sat underneath it. You might challenge the scope, challenge the priorities, or ask whether there was a quicker version. But you generally trusted that the people building the software understood the work involved in building it.

That trust is starting to shift. Not because developers have suddenly become slower, in fact, they have never been faster. And not because Product Owners have stopped being able to plan (and again, the planning is now supercharged). Because leadership teams now have their own frame of reference. They can build enough to become dangerous.

The prototype is not the product

A founder can sit down with Claude, Lovable, Bolt, v0 or any number of other tools and, within a few hours, create something that looks remarkably like software. There are dashboards, customer journeys, settings pages, reports, buttons that work. You can click through it, show it to customers, get excited about it. And to be clear, I think that is brilliant. I use these tools myself. As a non-technical founder, being able to turn an idea into something visual without waiting weeks is hugely valuable. You can test assumptions earlier, show customers what you mean instead of explaining it in a PowerPoint presentation, and work out very quickly whether an idea has legs or whether it needs putting in the bin. That is progress.

The problem is not the prototype. The problem is what happens next.

We've accidentally started benchmarking engineering against prototypes. A founder walks into a meeting with something they built over the weekend, engineering estimates the production work, and then comes the question: "I've already built most of it. Why is this going to take another three months?"

It is a fair question. It is also based on a false comparison.

You haven't built the product. You've built the presentation of the product: the flow, the interface, the concept, the bit the customer can see. You haven't necessarily built the authentication, the user permissions, the infrastructure, the data model, the integrations, the testing, the monitoring, the resilience, the audit trail, the security, the accessibility, the governance, the deployment process, or the recovery plan for when something goes wrong at 2am. You have built something that looks like software. That is not the same as building software people can safely and reliably use.

When AI becomes the second opinion

This becomes even more difficult when leadership asks AI for a second opinion. "How long should this take to build?" "Could this be done in three weeks?" "Is my development team overestimating this?" AI is not lying to you. It is answering exactly what you asked. The problem is that most non-technical leaders do not yet know everything they should be asking.

If your prompt says "how long would it take to build a dashboard with customer login, reports and an admin panel," AI will estimate a dashboard with customer login, reports and an admin panel. It will not automatically understand the regulatory environment you operate in, the assumptions your engineers have made about scale, or that your customer has demanded single sign-on. It will not know that every action needs to be logged, that the system needs different permissions across four user types, what your cyber requirements are, or what happens when the third-party integration fails halfway through a transaction. It will not know any of that unless you know to tell it.

So leadership walks into the meeting thinking engineering has overestimated. Engineering walks into the same meeting trying to explain all the work leadership never asked AI to consider. They are not answering the same question. That is where the friction starts.

One of the development teams I spoke to described being asked to build an application by a CEO who was adamant AI should make the whole thing incredibly fast. He had created a clickable prototype that looked good, moved, and gave the impression that most of the work had already been done. From his perspective, the team were now polishing and connecting it. From their perspective, they were starting the actual build. That gap in understanding is becoming a serious problem. The prototype is no longer being treated as a way to explore an idea. It is being treated as evidence that the engineering estimate must be wrong. And that starts to erode trust.

Abstraction removes visibility, not complexity

AI has not removed complexity. It has hidden it. Technology has always moved in this direction, from machine code to programming languages, frameworks, cloud platforms, no-code tools and now AI. Each layer makes it easier for more people to create, and that is a good thing. But abstraction removes visibility, not complexity. The complexity is still there. You just no longer see it.

It is a little like looking at a beautiful architectural rendering of a house and deciding the builders are taking the piss because the house already looks finished. Of course it looks finished. That is the point of the rendering. But the rendering is not the foundations, the structural engineering, the plumbing, the wiring, the building control, or the hundreds of decisions and skilled hours required to turn an image into somewhere people can safely live. Software prototypes have become so convincing that leadership is starting to confuse the rendering with the building.

I spoke to one developer who said AI-generated prototypes are brilliant for the first few days or weeks of a project. They help people think, bring an idea to life, expose gaps in the customer journey, and help teams have better conversations. But trying to take that prototype and convert it into production software can sometimes take longer than rebuilding it properly. Not because the prototype is bad, but because it was solving a completely different problem. Its job was to prove the idea. Engineering's job is to make the idea work reliably in the real world. Those are not the same job.

Years ago, the process might have been sketch, wireframe, prototype, engineering. Today it is AI prototype, engineering. The engineering phase has not disappeared. The prototype has simply become much more impressive.

Spare a thought for Product

Spare a thought for Product Owners in all of this. Five years ago, they were translating between customers, commercial teams and engineering. Today, they are translating between customers, commercial, leadership, engineering and the prototype the CEO built over the weekend. That is a completely different job. Every estimate now has a competing estimate, every roadmap conversation has a new reference point, and every sprint planning session risks becoming a debate about why the production work bears no resemblance to the three hours it took to generate the first version. Product is being asked to manage ambition, reality and confidence all at once. That is not a small change.

Learning enough to ask better questions

I am not writing this as a developer. I have spent most of my career sitting between commercial teams and engineering teams. I was Chief Commercial Officer at Northcoders before going on to build digital products myself. One of the best decisions I ever made was making software delivery my business. Not learning to code for the sake of saying I could code, but learning enough to understand how software is planned, built, tested and maintained. I sat in sprint planning, listened to architecture conversations, and learned why one apparently simple feature could take a day and another could take six weeks. I learned enough to ask better questions. More importantly, I learned enough to recognise when not to challenge.

That has completely changed the relationship I have with development teams. It does not mean I accept every estimate without question. It means the challenge is informed. What assumptions are we making? Can we reduce the scope? What is essential for the first release? What is driving the complexity? What happens if we leave this out? Those are useful questions. "Claude said it should take two weeks" is not.

The real issue is trust

We are reaching a point where leaders are starting to lose trust in experts because they now have a tool that gives them answers they do not yet have the experience to challenge. That, for me, is the real issue. Not whether AI can write code, not whether developers will be replaced, not whether prototypes can be built faster. Trust.

Once that starts to go, every estimate becomes a negotiation, every roadmap becomes an argument, and every customer commitment becomes a gamble. Engineering becomes defensive, product becomes exhausted, and commercial starts promising against a plan nobody believes in. Then, when delivery slips, leadership points at the development team. That is expensive: financially, operationally, and culturally.

Use AI. But know what it can't tell you.

None of this is an argument against AI. Quite the opposite. Use it. Prototype faster, test ideas, show customers, challenge assumptions, build ugly things and throw them away. Use it to make your thinking better. But do not mistake a brilliant prototype for a production system, do not mistake an AI estimate for engineering judgement, and do not quietly undermine the people you hired because a tool gave you a faster answer.

Leadership teams do not need to become engineers. They do need to become better customers of engineering. Learn enough to ask better questions. Learn enough to understand where the prototype ends and the product begins. Learn enough to know when to challenge and when to trust the people whose expertise you are paying for.

Because just because you can build something that looks like software does not mean you have built software. And if leadership cannot tell the difference, don't blame the developers when the roadmap slips.