From Search Engine to Answer Machine
Are you a Courier or a Curator?
Listen to Steve read this post (7 min audio)
The internet used to about searching for information. A place to explore the world’s largest filing cabinet of human knowledge. We’d be presented with options, then we’d analyse it, and decide for ourselves.
Today — we get an answer — a step has been removed, and that step is human judgment.
AI overviews are now the default on Google Search. I guess we should now call it Google Answer. Commercially, they didn’t really have a choice, as search was quickly being replaced with AI engines like ChatGPT and Claude. To be fair, Google has done a tremendous job since their internal code red and pivoted radically towards an AI-driven future — in short, they didn’t do a Kodak. Kudos to them.
Previous AI Quiz: A draw - first time ever! (AI was on the left)
The basic entry point of the internet, Google, now has a totally different menu. In order, we now see the following after a search:
AI Summary
A few videos on the topic, via YouTube of course
“People also ask” with more AI summaries
Then some relevant links, largely below the fold
We have moved from options to answers. The primary result now is an AI Summary — most of which are pretty damn good. How good? Recent research shows it is around 90% good! Which, on the surface, sounds like a very nice number — accurate, reliable. But when you take into account that Google serves over 5 trillion searches a year, that translates into tens of millions of incorrect answers every single hour. For my readers in Australia, the number is around 25 million inaccurate AI summaries a day.
Here’s a thought experiment worth taking: Would you eat food if the ingredients list was accurate 90% of the time, take medicine that did what it said 90% of the time, or drive a car if the brakes worked 90% of the time? I wouldn’t, but for some reason it’s OK for Big Tech companies.
Who do you Trust?
And it is pretty easy to be tricked that the summary is accurate — because the research also shows that the top sources provided with the AI summary, the link you can click to see where the AI summary came from, often do not even support the claim which the AI made. If it says Oxford University was the source, then most people would trust it, because very few people click on the link to check if something is true. I know from writing this blog that hardly anyone clicks on links with research. My blog stats show that around 1% click on research links I provide. (I bet you didn’t click on the one above… all of a sudden I’ll get my highest click-through rate yet!)
The reason is that you trust me — and I’m glad you do. The problem is that we’ve now entered an era where the source is more important than ever. In a world where anything and anyone can be faked, knowing where something came from is vital, and not enough of us are asking that question.
** Get me into do an AI keynote at your next event. I’ll use this as my testimonial!
How to Internet?
AI slop is easy to spot. Plausible, erudite, yet incorrect AI summaries are a far more difficult proposition.
It means we and our kids need a new skill base when it comes to using an AI-driven internet. We need judgment and the patience to dig into what we see to test the veracity of what was served up. So while AI has reduced our workload in one area by finding answers or providing information, instead of us having to find it and summarise it, it has also increased the workload in another area. We now have to dig in and see if what it served up was true, and assess the nuance. Not always, but often.
Courier or Curator?
As time passes, we’ll notice a difference in the diligence of people. There’ll be couriers and curators. The couriers will be those who simply deliver what they’ve been given, and to be quite honest, they’ll put their career at risk by doing this. And there’ll be the curators — those that dig deep, cross-reference, and decide what they think based on what and where the information came from. The choice is ours to make, and like always, the easiest path with the least friction will probably be the one that yields the worst results.
Choose wisely.
Keep Thinking,
Steve
The 90% right depends. A 10% wrong recipe where I know how to fix it is fine. A 10% wrong critical business decision where the info holder does not know the domain is catastrophic.
Thanks for this Steve, it’s got me thinking about where I’m unquestioning of the responses and where I add my own judgement, verification, research etc. I’ll certainly ramp up the latter! I appreciate your insights as always.