AI in procurement – the problem and the solution

Procurement is undergoing a transformation that echoes broader shifts within supply chain management, where it plays a central role. Once primarily viewed as a cost centre, procurement has now emerged as a strategic function – an evolution driven in part by a series of disruptions that began with the pandemic, and in part by the accelerating deployment of artificial intelligence.

 

This elevated status of procurement within the organisation, however, also comes with more responsibilities and workload.

 

Within the supply chain, the onus is particularly on procurement to mitigate the negative effects of market volatility on supply chains by adopting multi-supplier strategies and building inventory buffers.

 

It also falls upon the function to select and source digital technology that can lend supply chains the kind of multi-tier transparency and traceability necessary for achieving efficiencies and resilience.

 

Complex selection criteria for technology procurement – especially for machine learning (ML), large language models (LLMs) and gen AI in general – such as interoperability, data protection standards and AI ethics are further complicated by environmental and social sustainability considerations.

 

Decisions made by procurement are often trade-offs between cost, technological capabilities and ecological footprints – each of which contains mind-boggling interdependencies of its own.

 

General criteria for technology procurement

 

The procurement of AI tools includes steps that also apply when sourcing any other technology, or even non-technological products.

 

Whether sourcing AI-powered software or standard office supplies, the process starts with identifying business needs – those of the wider organisation as well as those of specific departments and users.

 

The next step involves exploring the market to assess what solutions are available and how they compare in terms of features and value.

 

Once suppliers submit their bids and proposals, they must be evaluated and winners selected accordingly.

 

The same applies to the next stage of the procurement cycle, contract management. Contracts – regardless of the product being sourced – must be negotiated, drafted and continuously monitored – or, occasionally, renegotiated between signing and termination.

 

There are, however, additional stages and tasks when the product to be sourced is a data-driven LLM or other gen AI tool, as these come with additional intricacies in training and data.

 

Gen AI technology is still in its infancy and has been marked by false starts and dead ends. As such, investing in one solution over another can feel like a gamble, even when procurement takes all the available information into account.

 

Integration often becomes a stumbling block, and, to prevent failure, collaborative teams of procurement and IT professionals must invest in solutions that not only address the pain points previously identified but also offer the highest integration potential with both legacy systems and those to be implemented in the future.

 

Procurement is also responsible for dismantling silos across functions by gaining an overall view of all the AI deployments in the organisation and aligning them in terms of interoperability.

 

But price, integration and interoperability are far from the only factors that must be considered with gen AI models.

 

Selection criteria specific to gen AI

 

Procurement must also examine all aspects of model training, including the data the solution was trained on and the precautions that were taken to avoid bias and unethical output.

 

It must also ensure that the necessary controls are in place to prevent breaches and new, gen AI-specific cyber-threats, such as prompt injection or model inversion attacks.

 

In mission-critical applications, it’s also imperative that the AI model is transparent – in other words, that the logic behind the output is explainable. This remains a tall order for gen AI models. As a result, for critical deployments, model transparency may take precedence over both affordability and technical performance.

 

Since LLMs and gen AI models typically require post-purchase training to reach their peak performance, in the selection stage it’s also important to factor in how much extra training the model will need before it can go live.

 

Meanwhile, service level agreements must reflect the supplier’s commitment to improving the model’s service level until the deployment reaches its maximum output.

 

With regard to data, procurement professionals should not only weigh the provenance of the training data but also the ownership rights granted by the supplier over the generated output of the AI model.

 

As Philipp Rosenauer, Partner Legal of PwC Switzerland explains, “Suppliers are often hesitant to grant the IP rights due to the uncertainty associated with protecting an AI’s output under IP law and due to the possibility of identical output produced by multiple users.”

 

Therefore, procurement and legal professionals should aim for getting the suppliers of AI models to disclaim any rights to the output during contract negotiations.

 

Standing in the shadows

 

We have all heard about shadow IT, when employees buy IT tools with the department’s budget without getting the green light from procurement – often in ways that contradict company spending policies.

 

Even before the emergence of shadow AI, enforcing these policies had been challenging enough for procurement. Now the function must also monitor unsanctioned departmental purchases to identify signs of AI sourcing that is unaligned with the company’s overall AI strategy and disregards the criteria of company-level interoperability.

 

The examples above clearly demonstrate how the latest advancements in technology increased the workload of the procurement function in the past three years following the emergence of LLMs and other image- and voice-based gen AI models.

 

Even with the support from the IT, legal and finance departments, procurement arguably already has too much on its plate as the adoption of gen AI continues to rise – a situation further aggravated by the fact that processes to evaluate gen AI solutions are still not in widespread use.

 

Paradoxically, as the resources of the function are stretching thin, it’s AI-enabled end-to-end source-to-pay platforms that may provide the light at the end of the tunnel.

 

The mantra often heard about AI deployments – that they free up time from onerous tasks that teams then can spend on high-value work – could also be the recipe for coping with the mounting workload in procurement.

 

The time saved by the automation of purchase order, invoice, contract and supplier management workflows can be redirected to the completely new tasks that the procurement of gen AI systems demand.

 

This shift may occur sooner than expected, as necessity is, after all, also the mother of AI deployments.