Our team recently built an internal tool to track pipeline activities. The need was simple: we're often engaged in conversations, workshops, and pursuits well before anything becomes a Salesforce opportunity, and that early-stage work was living in scattered notes and memory. So we built something to capture it.
AI played a big role. Devin and Claude Code were heavily involved in both building and maintaining the application. The pace was striking: features that would once have taken weeks came together in days. The tool authenticates users through our corporate SSO platform, follows sensible patterns, and works well.
And then it broke in a way nobody anticipated.
The Stale Tab
Here's what happened. I logged in, navigated to my open activities page, and left that tab open. Life happened. The next day I came back to the same tab, clicked into an activity's details, and even opened the edit page. Everything loaded. Nothing seemed wrong.
I made my edits and hit save. The data didn't save, and I was bounced to the login page.
What I didn't know was that my authentication token had expired on the server overnight. The browser had cached the pages and happily rendered them, so the front end had no idea the session was dead. The server, correctly, refused the write. The result was a technically "secure" system and a thoroughly bad experience: lost edits, confusion, and a login screen where I expected a confirmation.
Why This Slipped Through
Nothing here is a security hole. The server did exactly what it should. The failure was in the seam between systems: a long-lived client view backed by a short-lived server session. Authentication worked. Authorization worked. Expiry worked. What nobody designed was the experience of those things interacting over time.
This is the kind of gap that requirements documents rarely capture and that AI doesn't naturally volunteer. Nobody wrote a prompt saying, "Consider what happens when a user leaves a tab open for 18 hours, then edits a form." The AI built what was asked, and built it well. The missing scenario lives in how real people actually behave, which is messy, distracted, and full of open tabs.
The Lesson
The one thing we can't escape is that new software, even AI-derived software, is still new software. Things will be missed. AI can generate code, tests, and even edge cases, but it won't capture every one, especially those rooted in user experience. Those only surface when real humans use the product in real conditions.
That's not a knock on the tools. It's a reminder about expectations. When development gets faster, it's tempting to assume it also gets more complete. Speed and completeness are different things. A tool that ships in days still needs the weeks of living-with-it that every application has always needed.
What We Did About It
The fix itself was modest: detect expired sessions on the client, prompt for re-authentication before a save is attempted, and preserve in-progress edits so nothing gets lost. Straightforward work, and Claude Code handled it quickly once the problem was clearly described.
That's the pattern worth noting. AI accelerated both the build and the repair. But a human had to find the problem, understand why it mattered, and describe it clearly.
Treat AI-built software like any new release. Put it in front of real users early, expect surprises, and make it easy for them to report what they find. Budget time for the long tail of edge cases, particularly those involving time, state, and human habits. And keep humans close to the feedback loop, because the people using the tool will always find the scenarios nobody imagined.
New software is still new software. The tools have changed. The need to learn from real use hasn't.