Insights / News
Insights / News
With the launch of ChatGPT Health last week, the integration of Artificial Intelligence into our daily lives has crossed a Rubicon. No longer just a tool for drafting emails or debugging code, ChatGPT is now positioned as a personalised health companion, capable of integrating with medical records and wearables to offer guidance on everything from lab results to fitness plans.
While OpenAI is careful to label this as an aid rather than a physician, the reality of how users will interact with it is predictable. People will rely on it. And inevitably, it will get things wrong.
For practitioners in clinical negligence and product liability, this launch crystallises the theoretical debates we have been having for years. When a health AI hallucinates a diagnosis or misses a critical warning sign, who is responsible?
The primary route for redress in the UK remains the Consumer Protection Act 1987 (CPA). However, before a claimant can even argue a case, they must surmount the initial hurdle: Does ChatGPT Health constitute a “product” at all?
Traditionally, the received wisdom – based on cases like St Albans City and District Council v International Computers Ltd – was that software must be physically incarnated (e.g., on a disk) to count as “goods”. “Pure information” was excluded.
However, strict adherence to the tangible/intangible divide is increasingly anachronistic.
Elsewhere ([2025] J.P.I.L., Issue 4 and NLJ 04.07.25) I have considered this question in detail, arguing that the legal landscape shifted following the ECJ’s ruling in The Software Incubator, a case decided under the Commercial Agents (Council Directive) Regulations 1993. In that case, following an ECJ referral, the Supreme Court firmly rejected the Court of Appeal’s analysis, recognising that software – even when supplied electronically – has commercial value and acts as a “good”. The Supreme Court thus confirmed that the mode of delivery (digital vs. physical) should not determine legal status.
ChatGPT Health is the first large-scale launch of its kind, and is less than a week old. It is therefore difficult, when assessing the standard of care under the CPA, to gauge what “persons generally are entitled to expect” (s.3 (1) CPA).
More fundamentally, there is a quasi-philosophical question at the root of the difficulty. As Lavy et al ask in The Law of Artificial Intelligence:
‘How is quality and capability to be judged? Where the subject matter of a contract is more “traditional” software, even absent of a detailed specification, it is generally possible to assess quality by reference to yardsticks such as defect number and severity, and adherence to generally accepted standards with regards to user interfaces and use of resources. However, with AI it is not so straightforward…’
As Lavy et al note, what constitutes a defect will be highly context-specific. ‘The quality infrastructure needed around a tool whose function is to recommend film choices is rather different from one intended to diagnose disease or manage a safety-critical facility’
A key battleground will also be the iterative nature of these models. ChatGPT is designed to learn and self-correct. But this poses a novel legal question: “Does an initial error in a machine learning algorithm, later corrected, denote a lack of satisfactory quality? Or is it simply part of the learning process?” If a health AI hallucinates a diagnosis today but is patched tomorrow, was it “defective” at the moment of harm?
Ultimately, however, despite the futuristic veneer, the legal test may end up mirroring that of more traditional cases such as the Metal-on-Metal Hip Litigation.
As Kellar and Wishart (JPIL 2026) rightly observe, the public is not entitled to expect a product that is entirely risk-free. In the same way that hip prostheses eventually fail or aspirin carries side effects, diagnostic error is an inherent risk of medical practice. The judicial inquiry, therefore, will not be ‘did the AI err?’, but rather ‘is the incidence of error abnormal?’ Their point is essentially that if software purporting to detect skin cancer has a higher false-positive rate than a human GP but detects more early-stage cancers, it would be wrong to deem it “unsafe”. To exclude these benefits from the legal calculus would risk stifling life-saving innovation.
However perhaps the most significant barrier for claimants under the CPA is the “safety net” provided by human oversight. Since the majority of diagnostic AI is distributed as merely a “decision-support tool”, developers can position the user’s own autonomous judgment as a firewall against liability. Consequently, if an AI is accompanied by clear warnings that it must be used under supervision—functioning merely as an aid—courts may deem the product safe when strictly operating within those parameters.
Given the intense controversy surrounding whether ChatGPT Health even falls within the scope of the CPA, it is well worth turning our attention to common law negligence.
While Bolam remains the starting point for medical negligence, the courts have shown an increasing willingness to bypass it in cases of “pure diagnosis” involving the misinterpretation of data – a distinction critical for AI liability.
The leading authorities here are Penney v. East Kent Health Authority and Muller v King’s College Hospital NHS Foundation Trust [2017] EWHC 128. In the latter, Kerr J distinguished between “pure treatment” and “pure diagnosis” cases. In “pure diagnosis” cases, “there is no weighing of risks against benefits and no decision to treat or not to treat; just a diagnostic … decision which is either right or wrong, and either negligent or not negligent”.
This distinction is potentially fatal for a chatbot. If ChatGPT Health misinterprets a clear data point in a lab result (a “pure diagnosis” error), a defendant may struggle to hide behind a “body of opinion” that says such hallucinations are an acceptable quirk of Large Language Models. If the error is factual and “there to be seen,” Muller and Penney suggest the court will not hesitate to find negligence.
Indeed, ironically such cases create an environment more favourable to claimants than the strict liability regime of the CPA.
The argument canvassed above – that the software’s holistic, societal utility outweighs the risk of occasional error – is unlikely to carry much if any weight in common law negligence. The common law does not typically indulge in broad cost-benefit analyses of a product’s social value; it looks at the particular facts of the case and the duty owed to the specific patient. Once a defendant assumes responsibility for a diagnosis, they cannot escape liability for a specific error simply by pointing to the general good the software serves. Thus the CPA allows defendants to plead that ‘persons generally’ expect glitches in new technology, effectively lowering the standard of safety. But if Muller is applied, the common law tolerates no such sliding scale.
The same is arguably also true of the “defence” of human oversight. This too arguably carries less weight under the common law.
Under the CPA a product is judged by “the manner in which it is marketed” and “any instructions for, or warnings with respect to, doing or refraining from doing anything with the product” (s.3(2)). In common law negligence, however, such warnings ‘feel’ superseded by an assumption of responsibility inherent in the fact-specific diagnosis.
For example, warnings or not, if the app incorrectly identifies a cancerous mole as a harmless skin tag, the app has offered a diagnostic opinion. If the user relies on this diagnosis and suffers harm (eg by not consulting a doctor) it is doubtful that a generic disclaimer would absolve the developer. After all, the warning would be somewhat incongruous. It would be analogous to a surveyor certifying a building is structurally sound while simultaneously warning the buyer to verify the foundation themselves – a caveat that is functionally useless given the buyer’s lack of expertise. The disclaimer is fundamentally incompatible with the intended purpose and with the reliance the product invites. If the manufacturer derives the benefit of that reliance – namely, profit – it must surely carry the burden as well.
Thus, not for the first time, we may find that the common law proves itself a sharper tool for claimants than the CPA, notwithstanding the latter’s supposed advantage of strict liability.
ChatGPT Health is a remarkable innovation, yet it enters a UK legal landscape that is beset by uncertainty. While the Consumer Protection Act remains the primary statutory route for redress, the common law of negligence may well still have a role to play and may even ultimately prove itself a more promising avenue for claimants. As illustrated by Penneyand Muller, courts are willing to distinguish between treatment decisions and binary cases of “pure diagnosis” – a distinction that could be fatal for AI developers, despite their best endeavours to exclude liability with disclaimers.
For now, we watch and wait, but as these tools transition from novelty to necessity, the courts will inevitably be called upon to decide precisely where the role of a mere ‘aid’ ends and the duties of a doctor begin.
Harry Lambert specialises in the areas of product liability, medical tech, clinical negligence, personal injury, and human rights law. He is also renowned for his expertise in group litigation claims relating to these areas.
He is at the forefront of thought leadership on the law and emerging tech and is the Founder of the Centre for Neurotechnology and Law, whilst his 15-part series on Neurotechnology and The Law which has garnered academic acclaim and been turned into a podcast run by the Italian equivalent of the Financial Times. Harry’s expertise on neurotechnology is such that he is asked to speak all over the world and he has given or is giving talks in inter alia Spain, Italy, Turkey, India, Australia, Canada, Ethiopia and Chile.
He has also carved out a unique market reputation at the intersection of law, social media and AI. Harry has written extensively on the subject including on tortious liability for algorithmic wrongs in JPIL, and a 3 part series in the NLJ asking whether social media can be conceptualised as a defective product.
This expertise drives his practice: Harry acts for a number of bereaved parents who have lost children to social media harms including the high-profile Ellen Roome inquest involving the ‘Blackout Challenge’.
He has been described, variously, as a “visionary” and as the “Michelangelo of the Product Liability Renaissance”.
To find out more about Harry, contact Paul Barton on +44 (0)20 7427 4907 or Harry Gamble on +44 (0)20 7353 6381 for a confidential discussion.
Legal Blogs, News 13 Jan, 2026
