AI beats lawyers in claims prediction competition

The technology was 86.6% accurate compared to lawyers’ 62.3%

AI beats lawyers in claims prediction competition
Artificial intelligence prevailed over lawyers in a London competition that involved predicting claim outcomes.

CaseCruncher Alpha, which was developed by the same team that created Law Bot, was able to more accurately predict the outcome of payment protection insurance (PPI) mis-selling claims.

The "Lawyer Challenge" attracted 112 lawyers to pre-register for the competition, in which they submitted 775 yes or no predictions whether the Financial Ombudsman would succeed in PPI claims. The participants include lawyers from top firms like Bird & Bird, Kennedys, Weightmans, Allen & Overy, Berwin Leighton Paisner, DLA Piper, DAC Beachcroft, DLA Piper, and more.

After judging by Ian Dodd, UK director of Premonition, and Felix Steffek, senior law lecturer at Cambridge University, the CaseCrunch system came out on top with an accuracy of 86.6%, compared to the lawyers' accuracy of 62.3%.

"We are not necessarily adversaries in this game, the systems like ours can make the legal world more effective for everyone. I am convinced that we have now reached the point where our technology and expertise allow us to satisfy both our vision and our commercial interests. We are looking forward to finding solutions for our clients now," said Jozef Muzak, CaseCrunch managing diretor.

Rebecca Agliolo, CaseCrunch marketing director, said the event was ultimately not about winning or losing.

"It was about showcasing the potential of artificial intelligence and changing the current paradigm not by talking, but by doing," she said. "We hope that the Challenge will be replicated and improved - and we are proud to get the ball rolling."

Ludwig Bull, CaseCrunch scientific director, said that evaluating the results is tricky.

"These results do not mean that machines are generally better at predicting outcomes than human lawyers. These results show that if the question is defined precisely, machines are able to compete with and sometimes outperform human lawyers. The use case for these systems is clear. Legal decision prediction systems like ours can solve legal bottlenecks within organisations permanently and reliably," he said.

Related stories:
AI will help you meet client needs says law firm report
Australia’s first legaltech major launched

Free newsletter

Subscribe to our FREE newsletter service and we’ll keep you up-to-date with the latest breaking news, cutting edge opinion, and expert analysis affecting both your business and the industry as whole.

Please enter your email address below and click on Sign Up for daily newsletters from Australasian Lawyer.

Recent articles & video

Australia's biggest retail transaction succeeds with Gadens' help

Media companies liable for defamatory comments left by users on their Facebook pages: High Court

Allen & Overy flexible resourcing platform hits the US

Clayton Utz elects Australian Takeovers Panel member to board

Mandatory vaccination a rising trend among employers, Piper Alderman reveals

Women-led legaltech startup makes Japanese debut with in-house legal hub

Most Read Articles

NSW Supreme Court upholds mandatory COVID-19 vaccines

Holding Redlich boosts tax controversy practice with EY hire

Mandatory vaccination a rising trend among employers, Piper Alderman reveals

Top firms provide counsel on complex $1.1bn Boart Longyear restructure