When AI Gets It Wrong
In the Age of Instant Insights, the Real Competitive Advantage Is Knowing What to Trust
In my first job out of college, I worked as a trombonist in a rock band. But when I finally made my parents happy and got a proper job later that year (rock and roll trombone is pretty niche, and I’m not that good) I worked in the economic research department at Brown Brothers Harriman. It was the kind of place where precision mattered—a lot. I would spend hours combing through capital flows data from Japan, eventually picking up the phone to call someone at the Japanese Ministry of Finance because I wasn’t sure I was interpreting their reporting conventions correctly. That’s what it took to get the data right.
Fast forward to today, and the idea of getting a polished market research report in seconds—courtesy of generative AI—feels miraculous. With tools like ChatGPT and OpenAI’s Deep Research, Deepseek and Claude’s expected upcoming release of a deep research model anyone can produce a sleek document filled with insights, charts, and stats. But here’s the question: in the age of AI, the research is fast—but is it real?
That’s not a rhetorical concern. As generative tools flood inboxes and decision-making meetings with confident-sounding “findings,” we’re entering a strange new era—one where everyone can create an “insights report,” but few can verify it. And the consequences for business, policy, and public trust are significant.
✦ The Mirage of AI-Generated Research
Benedict Evans recently documented his experience with OpenAI’s Deep Research. The tool generated a slick analysis of smartphone adoption in Japan—complete with citations. The only problem? The data was wrong. Key statistics were pulled from outdated or misinterpreted sources like Statista and Statcounter. How wrong? It doesn’t really matter because the end result was a report that looked authoritative but couldn’t be trusted.
This is more than a footnote in AI’s evolution. It’s a cautionary tale. Most large language models (LLMs), including ChatGPT, aren’t retrieval systems—they’re probabilistic engines. They generate the next likely word based on patterns in training data. That can mean they’re pulling from outdated or irrelevant data sources. Or worse, misinterpreting the data entirely failing to understand the nuance of what a dataset actually represents. Yet the results are presented in polished prose, with an air of confidence that makes errors nearly invisible.
For consumers of information, this creates a strange asymmetry: the outputs feel credible, but the underlying logic is opaque. It’s a bit like getting stock advice from someone who sounds like Warren Buffett—until you realize they’re just guessing.
And here’s the real danger: unless you’re a subject matter expert, you won’t know what’s been misrepresented because you won’t even know what to question. The mistakes aren’t always obvious. They live in the assumptions, the framing, the fine print. If you don’t already understand the topic deeply, it’s easy to take the AI’s answer at face value—and that’s exactly when it’s most likely to mislead you. What you’re left with is research that sounds right, feels right, and might be right—but that you have no way of verifying without deep domain knowledge. That’s not just inefficient. It’s dangerous.
✦ Getting it Right
At Co-Created, we encountered this problem firsthand. We were using generative tools to speed up internal research, but we kept running into the same wall: we couldn’t trace anything. Outputs changed when we re-ran the same prompts. Citations disappeared. We couldn’t answer basic questions like, “Where did this data come from?” or “Why did the AI say this?”
The good news is that all the getting it wrong, led us to eventually get it right. Instead of chasing sleek one-off outputs, we wanted something that could reliably support business decisions.
A better solution is an AI-powered research tool designed for structure, traceability, and auditability. Here’s how it can work differently:
• Deterministic Outputs, Not Just Free-Form Text
Sense builds repeatable workflows with structured prompts and data scaffolding. That means it’s not just hoping the AI gets it right—it’s designing for correctness.
• Smart Data Objects, Not Blobs of Text
Sense extracts key primitives—like a problem definition, a customer need, or a competitive insight—and tracks them individually. This enables chaining insights together over time, rather than getting isolated soundbites.
• Full Context Reconstruction
Instead of dropping raw documents into a prompt, the tool should reconstruct and organize relevant content across multiple sources, ensuring the AI model sees the full picture before responding.
• Audit Trails and Source Provenance
Every insight must link back to its origin—whether it’s a public filing, a competitor website, or a user-uploaded artifact. That makes verification easy, and hallucinations much less likely.
• Multi-Model Optimization
ChatGPT relies on one model. A good tool needs to use many—selecting different models for natural language processing, embeddings, or specialized analysis depending on the task.
• Custom Outputs Built for Business
From investor memos to quarterly reports to spreadsheet data dumps, the tool needs to deliver structured, exportable formats that match how teams actually work.
That’s why we built Sense. It’s not just another AI tool—it’s a research system built for teams that need to move fast and get it right. Because in a world where everyone can generate insights, the real edge is knowing which ones to trust.
✦ The Bigger Picture: What We Lose When We Trust Too Quickly
There’s a reason I remember that call to Japan’s Ministry of Finance. It wasn’t about one data point—it was about accountability. When you’re making decisions that affect people’s jobs, investments, or strategies, you need to know what’s real. And knowing means being able to trace back, challenge, and revise—not just consume and move on.
Generative AI isn’t going away. Nor should it. Tools like ChatGPT are invaluable for brainstorming, summarizing, and sparking ideas. But when it comes to research that informs action, businesses need to ask: What are we trusting, and why?
As the AI wave accelerates, the organizations that win won’t just be the ones who use it fastest. They’ll be the ones who build trust into the process—who can separate the insights worth acting on from the noise that just sounds good.
Reach out to start a conversation.