Criticism floods in after it turns out that bar exam questions were created using AI

On April 21, 2025, the California Bar Association, an American bar certification organization, announced that 23 of the 171 questions on the bar exam held in February were created using AI. This prompted strong criticism from test takers and educational institutions.

State Bar of California admits it used AI to develop exam questions - Los Angeles Times

According to reports, in February 2025, test-takers in the California bar exam complained that the exam questions contained typos and were difficult to understand. Some test-takers complained that some of the questions read as if they had been written by an AI.

In response to these criticisms, the California Bar Association announced that 100 of the 171 multiple-choice questions on the bar exam were created by a private company called Kaplan Exam Services, 48 were taken from first-year law school students, and the remaining 23 were created with the help of AI by ACS Ventures, an organization of psychometricians, not lawyers.

'I'm speechless,' said Mary Bassick, associate dean for academic skills at the University of California, Irvine School of Law. 'I just can't believe that non-lawyers would use AI to create bar exam questions.'

The Bar Association has taken the position that there is nothing wrong with using AI to create the exam questions, and Executive Director Leah Wilson said in a press release, 'The ACS Ventures questions were created with the assistance of AI and then underwent review by a content validation committee and experts prior to the exam.'

There are differing views among stakeholders on how AI was used. According to Alex Chan, chair of the Board of Bar Examiners, the California Supreme Court, which oversees the bar exam, has encouraged the bar association to 'research new technologies, such as AI, to make the exam more reliable and cost-effective.'

Meanwhile, the Supreme Court told the media that it was 'unaware that AI had been used in creating the multiple-choice questions until the Bar Association issued a press release.'

The controversy over the use of AI to create the exam questions is part of a broader problem that has arisen with this year's bar exam. During the February remote bar exam, which was conducted through the online testing platform Meazure Learning, some candidates reported experiencing technical issues, such as screen freezes, crashes, and errors, which put pressure on the time to answer questions or prevented them from uploading their answers. There were also reports of examiners being unable to answer basic questions, providing incorrect information, or behaving rudely toward candidates.

These failings have prompted some test-takers to file a class action lawsuit against Meazure Learning and have also prompted state legislators to launch an investigation.

Educators have also criticized not only the questions created by AI, but also the use of questions from tests for first-year law students. 'A test to determine whether or not someone has the minimum competency to practice law requires different standards than those used to evaluate first-year law school learning,' Bassick said, expressing concern.

In a press release , the Bar Association announced that following the conclusion reached at a meeting of the Bar Examination Committee held on April 18, 2025 that 'it is appropriate to lower the passing standards in order to ensure fairness,' it had submitted a petition to the Supreme Court, and stated that it plans to reconsider relief measures for test takers at its meeting in May.