Blind Justice UK flagged instances where chatbots recommended problematic firms
Justice charity Blind Justice UK has cautioned members of the public against accepting law firm recommendations from artificial intelligence chatbots, reported the Law Society Gazette.
In the Blind Justice UK report “AI’s Blind Spot,” charity founder Edward Romain asked AI chatbots ChatGPT, Claude, and Gemini to list the top conveyancing solicitors in Doncaster. A firm that had been penalized by the Solicitors Regulation Authority topped the lists generated by all three chatbots.
Romain then queried the leading conveyancing solicitor in Maidenhead; Claude suggested a firm the SRA had forced into closure seven months ago. He inquired as to whether a Norwich firm was a good choice; none of the chatbots flagged that the firm in question had been fined for a failure to replace client money.
The report noted that the chatbots referenced Google listings and review sites but not regulatory decisions. The chatbots did provide disciplinary information when asked specifically; however, the report pointed out that individuals looking for solicitors would not be asking such precise questions.
The report indicated that mentions of the SRA centered on the “SRA-regulated” badge used by review sites as a mark of standing, and the chatbots did not dig deeper into the firms’ histories.
Thus, Blind Justice UK has called on the SRA to publicly release sanctions as open, machine-readable data and ditch its policy of unpublishing regulatory decisions after a three-year period. Profiles on review sites must be updated with SRA decisions, and AI developers must train chatbots to check the SRA’s register and advise whether a firm has been penalized or shut down.
Earlier this month, the legal thinktank Justice and research group Administrative Fairness Lab revealed that members of the public are increasingly taking legal questions to AI. Of over 1,400 respondents who had reportedly been involved in a legal dispute in the last two years, 16 percent sought legal assistance, information, and advice from an AI chatbot.
One respondent sent the full text from a lawyer’s email to a chatbot in a request for it to be explained in “normal person language,” per a snippet of Justice’s report that was published by the Gazette.
“The question is not whether AI becomes part of people’s legal journeys - it already is,” said Ellen Lefley, deputy legal director at Justice, in a statement published by the Gazette. “The challenge now is ensuring people receive reliable support, while addressing the unmet legal needs that drive some towards AI in the first place.”
