The Expert View: Trust, ethics and transformation: navigating the human-AI balance

One of the most striking themes of AI adoption is the underlying fear that surrounds it. Despite its growing presence, AI remains a relatively unknown concept for many organisations. The uncertainty lies not only in its potential impact but also in defining what it should – and should not – be used for. 

 

Senior leaders from banking, consultancy, technology, academia and life sciences gathered at a recent Business Reporter dinner briefing in London, hosted by Dynatrace, to discuss this. The conversation traversed the practical realities of AI implementation, the ethical dilemmas surrounding autonomy and decision-making and the cultural shifts organisations must navigate as AI permeates every business function.

 

Implementation realities: early successes and lessons learned

 

While enthusiasm for AI innovation is widespread, delegates shared a pragmatic view of what adoption really looks like within large and complex organisations. For most, AI initiatives remain exploratory rather than transformative at present. One attendee described the difficulty of “choosing the right horse” amid a fast-evolving tool landscape, and many participants noted they had begun their AI journeys cautiously, opting to partner with trusted vendors such as Microsoft, GitHub or AWS. These partnerships provided both familiarity with and reassurance in data security – a particularly crucial factor for public and regulated enterprises.

 

Pilot projects have focused largely on process automation and developer productivity. Delegates reported gains in efficiency, particularly in areas such as document summarisation, structured data extraction and code generation. However, the challenge of measuring productivity uplift remains unresolved. “Did the developer write better code, or just skip their coffee?”, remarked one delegate, reflecting broader uncertainty about how to quantify AI’s true value.

 

What was clear across sectors was that AI’s benefits are highly contextual. Tools that deliver impressive results in one domain often underperform in another, underlining the importance of aligning AI solutions to specific use cases rather than applying them universally.

 

Decision-making: where does human judgement end?

 

The question of how much autonomy AI should be granted dominated much of the discussion. Across industries, most participants agreed that AI should serve as a tool to support decision-making, not make decisions itself. As one delegate noted, humans should “remain firmly in the loop”. Others echoed this sentiment, arguing that while AI could accelerate access to information and reduce manual effort, ultimate accountability still rested with human operators.

 

Yet some sectors have already crossed that threshold in limited, low-risk scenarios. One participant noted that their payment systems have long relied on AI for fraud detection, with “about 95% accuracy”. The same organisation is now piloting generative AI for internal learning and sales enablement, reporting measurable boosts in employee confidence and knowledge retention. This contrast between operational reliance and ethical restraint underscored a key theme: that trust in AI is proportional to the risk it manages. Low-impact automation may proceed unhindered, but decision-making that affects customers, safety or reputation demands rigorous oversight.

 

 

The trust paradox: explainability versus outcome

 

Participants wrestled with the notion of trust – what it means, how it’s earned and how much explainability is truly necessary. One speaker argued that humans regularly trust systems they do not understand – from aeroplanes to online banking – because they perform consistently: “Trust isn’t about knowing every algorithmic step. It’s about predictability and acceptable risk.” Others were less convinced, insisting that AI must provide auditable reasoning, especially in regulated industries. “If a customer asks why a decision was made and you can’t explain it, that’s a problem,” one attendee cautioned.

 

The debate evolved into a discussion of risk appetite. Delegates agreed that AI should not be treated as inherently trustworthy or untrustworthy, but as a tool whose reliability must be empirically tested. Trust, then, becomes a function of validation – of knowing how often the system has been right and under what conditions. Another delegate reminded the group that explainability has its limits: “AI is most valuable precisely where we don’t have deterministic rule sets. You can’t always explain every step, but you can measure performance and confidence.”

 

Defining AI: beyond the buzzwords

 

Another recurring challenge was definitional. Several attendees noted that “AI” has become a catch-all term, used interchangeably for automation, machine learning and generative models, among others. This ambiguity fuels both confusion and unrealistic expectations. “Machine learning is one branch of AI, not a separate discipline,” one participant clarified. “Generative AI is just the latest approach.”

 

Others drew historical parallels, likening today’s debate to the introduction of calculators, spreadsheets and industrial robots – all technologies that initially provoked anxiety before becoming indispensable. Yet unlike those deterministic systems, generative AI’s non-deterministic nature – its ability to produce different outputs from identical prompts – represents a fundamental shift. As one participant put it, “Calculators don’t hallucinate; AI sometimes does.”

 

Ethics, regulation and risk

 

Ethical oversight is, delegates agreed, both a regulatory requirement and a moral imperative. Several participants spoke of the need for transparent model validation, traceable decision trails and responsible data governance. One attendee noted that regulators already require organisations to document how models influence decisions, a standard likely to intensify as AI adoption expands. “The regulator doesn’t yet know exactly how to treat AI,” they said. “But the direction of travel is clear: accountability will be non-negotiable.”

 

Others raised sustainability and data ethics, particularly in research and higher education. AI’s vast computing demands, combined with questions about data ownership and copyright, are forcing universities and laboratories to consider the environmental and ethical footprint of their digital infrastructure, alongside the demands of innovation.

 

Workforce evolution: upskilling, trust and the future of talent

 

The conversation also turned to AI’s impact on the workforce – both in terms of automation and opportunity. While some feared that junior roles could disappear as AI handles entry-level analytical tasks, others argued that the baseline of capability would simply rise. “Education must adapt,” one delegate observed. “We need to train digital research professionals who understand both the technology and its context.”

 

Several participants emphasised that AI adoption must go hand-in-hand with upskilling and broader cultural change. “AI won’t take your job,” one leader remarked. “But the person who knows how to use AI better than you will.” At the same time, there was recognition that trust – or lack thereof – could become a barrier to innovation. Employees wary of being replaced may resist the implementation of AI tools unless leaders communicate clearly that automation is intended to elevate, not eliminate human contribution.

 

Beyond fear: the human-AI partnership

 

Ultimately, those present concluded that the biggest challenge is perhaps not technological but psychological. While public understanding of AI still oscillates between fascination and fear – from utopian promises of limitless productivity to “HAL 9000” anxieties – to move forward, organisations must reframe AI, focusing less on artificial intelligence as a concept and more on its tangible, human-centred outcomes. “Maybe we should stop calling it AI altogether,” one participant said. “It’s just another tool – powerful, yes, but only as good as the people guiding it.” In the words of another attendee, “AI is like an overexcited puppy. It wants to do everything, but we’re still holding the leash.”

 

The future in the balance

 

The conversation underscored that AI’s future will hinge on balance; between automation and accountability, innovation and regulation, efficiency and ethics. There was a unified view that while technology can accelerate progress, it is trust – in systems, in governance, and in human judgment – that will determine whether AI delivers on its promise. As one attendee put it: “AI isn’t about replacing people. It’s about amplifying what makes us human.”

 

What became apparent during the event is that even the most senior technical leaders are seeking guidance. They want clarity on how to use AI ethically and responsibly without wasting time or resources. This signals a need for industry-wide standards, best practices and collaborative learning to help organisations move forward with confidence.

 

AI integration requires more than enthusiasm; it demands governance. However, measuring success isn’t straightforward. It must be weighed against the cost and processing power required to deploy AI solutions. The challenge is finding the balance between leveraging benefits quickly and doing so responsibly.

 

Dynatrace supports this need with AI observability that gives teams a unified, real-time view across their technology landscape. It provides actionable insights into performance and security, helping you keep AI-powered operations reliable and efficient. By making it easier to trace and explain how AI decisions are made, Dynatrace helps organisations build the confidence and oversight needed for responsible, successful AI adoption.