A recent study has revealed serious dangers in using generative AI tools for legal, medical, and financial advice. RIFT, a UK-based finance and tax specialist, is urging the public to verify AI-generated guidance following research that shows widespread inaccuracies.
A Columbia Journalism Review investigation found that more than 60% of AI responses to news-related queries were inaccurate or misleading. Among the poorest performers was Grok 3 from X, which had a 94% error rate, while ChatGPT Search followed with 67%. Even Perplexity, the most accurate, still produced incorrect answers 37% of the time. These mistakes stem from limitations such as hallucinations, biased training data, lack of context, incomplete information, and unsupported sources.
Such shortcomings become more critical when users seek high-stakes advice. In a UK court case, a litigant in person submitted AI-generated legal documents that cited fabricated case law, severely undermining their position. AI-generated legal advice is not protected by legal privilege, meaning it can be disclosed in legal proceedings. Unlike solicitors or barristers, AI tools are not regulated or covered by professional indemnity insurance, leaving users solely accountable for any consequences.
In the medical field, generative AI often hallucinates or references unreliable sources. It cannot assess individual factors such as medical history, lifestyle, or genetics. This makes AI a poor substitute for clinical expertise, and relying on it may lead to self-misdiagnosis or incorrect treatment. Privacy is another concern, as some AI platforms may store sensitive health data in ways that conflict with the UK’s GDPR regulations.
Errors in financial or tax advice can be equally damaging. AI tools may misread HMRC rules, miss relevant exemptions, or fail to consider the user’s complete financial picture. Filing taxes based on incorrect guidance can result in penalties, compliance issues, or delays. AI-generated advice in this sector is not regulated by the Financial Conduct Authority, offering no protection if mistakes occur.
Bradley Post, managing director of RIFT, warned of the risks: “While AI can be useful for general information or brainstorming, treating it as a substitute for qualified, regulated advice in legal, medical, or financial matters is risky and potentially costly. So whilst it’s tempting to turn to AI for quick answers, verification by professionals remains essential, as these tools don’t carry the responsibility, oversight, or accuracy required in areas like tax, health, or law. In short, what you save in time, you could easily lose in penalties or worse.”
Although AI tools offer easier access to information, they lack accountability and the nuanced judgement required in complex areas. Legal advice from solicitors is both confidential and insured. Medical guidance from qualified practitioners takes personal circumstances into account. Financial advisors like those at RIFT apply current HMRC rules with professional oversight.
To minimise risk, experts recommend that users cross-check AI outputs with trusted sources or professionals. The Columbia study shows that even the best-performing models frequently get facts wrong, especially in sensitive contexts. As AI tools become more common, public awareness of their limitations is essential.
