AI slop, and why judgement beats information

With AI tools, access to information and information processing is considerably democratised; AI is removing barriers to knowledge-gathering. Yet nearly 68% of professionals experience uncertainty about priorities at least a few times a week, while the single most valued improvement would be knowing which deals and tasks to prioritise each day.

 

The challenge isn’t lacking sufficient knowledge. In fact, too much knowledge can actually backfire. Instead, the challenge is processing – knowing how to sift through responses and information, and finding the signals. This is the real advantage in the post-AI era – it now comes from turning information into better judgement, not in adding more data to existing, even flawed processes that cause people daily working trauma.

 

Don’t flop with slop

Most tech industry and business leaders will have at least toyed with AI. At this stage, it’s more unusual if you haven’t. Around the AI boom – Industry 5.0 – bigger firms are placing some large bets. AI dominates discussion in business media, whether it’s about advancing frontier models, the politics of data centres and trade wars, or even practical life advice.

 

But a major issue is that if everyone uses the same resource, trained on the same data, in the same way… well, that’s how we get not only AI slop, but sloppy, lazy thinking. And that will happen if we don’t use these tools mindfully.

 

Human brains are lazy – designed that way by evolution to conserve energy. Give us a shortcut and most of us take it. It’s incredibly important to resist that urge, to learn and train, and to encourage our colleagues to use these tools in ways that support their long-term growth as well as that desire to just make it through the task in front of us. A quote I have been fond of this year is, “you can outsource your thinking but you cannot outsource your understanding.”

 

Judgement trumps information

Good judgement is better than good data. If you can use mental models and systems thinking and not fall into logical fallacies, then you are ahead of the game and many peers. The trouble isn’t that we humans are generally ‘stupid’; it’s that, as a rule, our brains are not designed to want to work hard, and they often take shortcuts that we don’t notice.

 

There are so many examples of this, but we’ve all fallen prey to processes like confirmation bias, where we only notice things that fit with what we already believe. Or the anchoring bias, believing in the first data you get and not being open to updating your mental map.

 

If we are to use AI tools well, we really must live up to the term ‘co-pilot’ and get some training as well as on-the-job ‘flying time’ under our belts. In this case, that means learning to apply sound judgement by applying methods like systems- and second-order thinking to the problems we face. Those of us in technology tend to have an edge in this area as we get some of this training from computing and engineering education and practice. But fighting against cognitive biases and logical fallacies requires some constant vigilance.

 

Here’s an example. AI or a data analysis team may report that sales fell 12%. You apply judgement by asking questions like: is this seasonal? Did we deliberately reduce discounting? Did a competitor launch? Which explanation actually matters? Have we experienced this before, and what can we infer from past experience?

 

Or, a consultant recommends entering a new market since growth rates are high. Judgement asks whether you have the distribution, support capability and brand credibility to succeed there. It asks what the pitfalls might be and applies a carefully weighted measure of risk and reward before making a final decision.

 

The tech industry celebrates the use of mental models. Lots of Silicon Valley luminaries and engineers credit tools like first principles thinking as part of their success. For example, right now, many SaaS companies have introduced AI chatbots in what’s become something of a race. A first principles thinker would approach the issue by looking to what fundamental problem customers are actually trying to solve. They might not care about getting answers to questions as much as maybe faster onboarding, fewer clicks to buy, or better native reporting. A chatbot would not help solve any of those efficiently.

 

Stop falling into traps

Back to those mental shortcuts and biases, the hidden part of the human cognitive experience. We can ask AI an important question, but if framed the wrong way, such as with confirmation bias, we may get a sycophantic and misleading answer. Currently, AI has a tendency to confirm the query put to it. If you ask, "Why is cold email dead?" or "What evidence suggests cold email still works?” AI will produce a persuasive answer with supporting evidence either way. Knowing the right question and how to ask it is key, whether tasking a consultant or a text window.

 

If you can deliberately employ systems thinking you begin to remove some of the risks of lazy thinking. For example, if customer complaints rise, a simple answer is to hire more support staff. A systems thinker might break down this problem into parts and test each one, asking: Why are complaints rising? Is product quality slipping? Is onboarding confusing? Did marketing attract the wrong customers? Merely tackling the surface problem is often like treating a symptom but not the injury that caused it. Always look to fix the system to get the upside faster.

 

Second-order thinking is also a great way to avoid creating more problems than you solve.  For example, a solution for a first-order issue might be to use AI to answer every customer enquiry because it’s cheap. But the second-order effects from this might include customers losing trust when answers are wrong, staff losing their expertise muscle memory when they stop solving problems themselves, and escalations becoming harder when no one has a grasp on the underlying issue.

 

Information on mental models and biases is freely available and can even be put into custom GPT instructions to help apply them to your problem solving.

 

Leaders, learners

"All leaders are readers" is something US President Truman said, highlighting how continuous learning makes for more effective leadership. Both learning and applying learnings with mindfulness are going to be increasingly important as people don’t do all the hard mental work they used to. We must learn to adapt to the technology and data environment we’re in.

 

It’s hard to find practical ways to ensure teams are applying their mental faculties well, and not becoming over-reliant on AI, turning themselves into a commodity employees:

• Ask teams to produce two strategies before consulting AI
• Treat AI as the first draft, never the final decision-maker
• Require every gen-AI recommendation to offer three reasons it could be wrong
• Task a colleague to be a devil’s advocate in strategic meetings
• Reward people who identify flawed assumptions
• Ask staff to explain the reasoning behind AI outputs and how sensible they are
• Mandate the application of mental models as lenses on issues

Leaders who strategically nurture their teams’ thinking abilities as well as people skills and business technologies will create teams able to best use these elements to the most strategic effect.

 

---

 

Joe Futty is Chief Product and Technology Officer at Pipedrive

 

Main image courtesy of iStockPhoto.com and PhonlamaiPhoto