Data privacy in the era of AI

The AI era presents a significant challenge when it comes to data privacy. Excessive data collection processes leave people with little control over their personal information once it has been used to train AI models. Digital privacy notices are notoriously complex and often fail to describe how data will be used in an AI context, despite regulations such as the GDPR requiring information to be concise, transparent and intelligible.

 

AI use in the workplace is also raising specific privacy concerns about surveillance, monitoring and accidental data leaks. A global study in 2025 by KPMG and the University of Melbourne found that almost half of employees globally admit to uploading sensitive company information to AI tools, often in violation of policy, creating privacy and security risks. Just 40 per cent of workplaces have clear guidance on generative AI use, leaving employees uncertain about privacy-safe behaviour.

 

Which regulations cover AI data privacy?

 

While the UK has no standalone “AI law”, several data protection and digital regulatory laws directly govern how AI systems must handle personal data. In the UK, the GDPR remains the core legal framework governing personal data used in AI systems. One element is Article 22, which gives individuals the right not to be subject to a decision based solely on automated processing – including profiling – if the decision produces legal effects or similarly significant impacts on them. This means important decisions, such as loan approvals or hiring decisions, cannot be made by AI alone unless it is necessary for a contract, authorised by law or if a person has given explicit consent.

 

However, the Data Use and Access (DUAA) Act 2025 changed the rules in automated decision making, giving organisations greater scope to use AI-driven decisions as long as oversight and safeguards remain in place. The restrictions continue to apply to special category data.

 

Data centre location directly affects which legal jurisdictions apply to organisational data, including government access rights, privacy protections and cross-border transfer restrictions. Regulations such as the UK and EU GDPR require organisations to know where personal data is stored and processed and to ensure appropriate safeguards when data leaves the UK or EEA, regardless of outsourcing arrangements.

 

UK government guidance from the National Cyber Security Centre (NCSC) further emphasises that understanding data residency and jurisdiction is a key element of cloud risk management and supply chain security, particularly for sensitive or public sector data. International standards such as ISO/IEC 27001:2022 – Information security management systems also treat data location awareness as part of effective information security risk management, reinforcing that accountability for data does not transfer with the infrastructure.

 

What risks does the future hold?

 

The future holds several evolving risks that will redefine what “privacy” actually means in a world where software can think and act at machine speed. With agentic AI systems, a rogue or misconfigured AI agent could rapidly leak sensitive data, and the privacy violations may not be known until it is too late. Within organisations, employees are increasingly using unsanctioned AI tools referred to as Shadow AI 2.0, the next generation of unsanctioned AI, ungoverned, and often invisible AI use inside organisations. This can lead to sensitive information scattered across dozens of third-party AI platforms that bypass standard IT security.

 

How privacy is governed is also hard to navigate due to differences, fragmented regulations and their interpretation. Data sovereignty has become, and will continue to be, a core part of an organisation’s privacy strategy. Data sovereignty defines who has legal control over data. The privacy protections depend entirely on the laws of the country governing the data’s location. Countries increasingly enforce data localisation to ensure that personal data stays within national borders, so that local privacy laws remain fully forceable. However, whether and how this is communicated to citizens, and if this then impacts their trust in AI, is unclear.

 

Space-based data centres and orbital computing infrastructure presents a new and complex challenge when it comes to data privacy. International space law provides limited guidance on data protection, meaning that privacy obligations are typically derived from terrestrial laws, such as the UK and EU GDPR, which apply based on the data subject and data controller rather than the physical location of the servers. This creates a jurisdictional grey area where personal data processed in orbit may be subject to multiple, potentially conflicting legal regimes, raising risks in accountability, lawful access and data subject rights. As AI systems increasingly process personal data beyond Earth’s atmosphere, organisations must ensure strong governance, clear jurisdictional alignment and demonstrable safeguards to maintain privacy, trust and regulatory compliance.

 

How can organisations boost data privacy and governance to manage the risk from AI?

 

Data privacy is the foundation of responsible AI governance. Formal AI governance frameworks need to be implemented with specific processes for privacy, for instance, requiring privacy impact assessments and risk assessments for all AI deployments. Companies should ensure that AI use policies covering approved tools, data handling, retention and risks are in place.

 

When procuring AI solutions, products or services as third-party tools, it is important to conduct thorough vendor assessments, as external AI providers may store, learn from or process sensitive data in ways that create privacy, security and compliance risks if not properly governed. Without proper review, organisations risk breaching data protection laws such as GDPR due to inadequate controls on data location, access or cross-border transfer.

 

It is important when developing AI to adopt a privacy-by-design approach and apply security-by-design controls. Firms should strengthen data sovereignty by ensuring they know where data is stored, processed and transferred. This should be transparent to internal and external stakeholders. 

 

Organisations should use monitoring tools to detect unapproved AI usage within the workplace to increase control over Shadow AI 2.0. Ensuring the use of AI models and tools is documented, recorded and audited is important, with training provided on sanctioned safe AI alternatives to reduce unauthorised tool use. Lastly, employees need regular training on safe AI use, including annual mandatory training on AI privacy risks. 

---