The California Bar Exam Disaster

Exam was royally screwed up from soup to nuts. So let's I wanted to let all you know. This is sort of inside lawyery baseball, but it's just a fascinating example of a couple of different things that just massively annoy me, particularly the proliferation of AI to basically arenas that don't need it and actively shouldn't have it, which is a big pet peeve of mine. So let me describe what's happened, all right, the bar exam. Most people have heard of this. The bar exam is the exam

administered by every one of the individual fifty states. It has some in most states, it has some portions of it that are sort of interstate or federal or not, you know, individual states specific, and then it has some stuff in it that is state specific and it is intended to qualify you as a lawyer. If you pass the Bar Exam, you become a licensed lawyer in that state. And it's a very difficult test. It's very it's very intensively like physically demanding. It. It takes place over the

course of two days. Back when I took it in twenty fifteen, I took the California Bar Exam after having I had already taken it in Massachusetts for my first job, and then we moved back home to California, I took the California Bar Exam, so I'm a member of the bar for both Massachusetts and California. The California Bar Exam when I took it in twenty fifteen was a three

day long test. Now it's only a two day long test, which most states have a two day long test, and it's I think it's two three hour sessions per day and very very intensive because basically they can give you questions from almost any body any part of American law. You might have an essay question about torts, you might have an essay question about contract. You might have an essay question about criminal law or criminal procedure. You might have a essay question about this, you might have an

essay question about that. And you have to basically study this enormous, enormous range of material to be like ready to go to answer whatever comes your way. So the lead up to it is super intensive because you have to study study studies so hard. It's such an enormous range of material. Now you have multiple choice questions on

the bar exam. I'm giving you this background, by the way, because the bar exam is only issued twice a year late July and February for people who wrapped up law school at the end of the spring semester at the end of the fall semester, respectively. And the most recent February administration of the bar exam was a train wreck. So let me keep describing how it goes for you so you can kind of under stand the context of how bad it is. The bar exam sort of has

two different kinds of questions. You have multiple choice questions and then you have essay questions. The essay questions present you with a large sort of factual, a large factual scenario, large complex factual scenario. You know, Mike was driving down the road. A police officer pulled him over. The police officer smelled marijuana in the car. He opened the trunk without asking permission first, the Mike fled the scene. He raised his fist as if to punch the officer, but

then did not actually connect with his fist. How many criminal claims? Spot all the issues with this scenario in an essay form? Okay, so they'll present you with this very elaborate factual scenario and you have to identify like all the different legal claims or issues involved within that legal scenario. That's how the essay questions work. And then the multiple choice questions are no walk in the park either.

The multiple choice questions on the bar exam present you with a very tightly written but very complex, little factual pattern, and you have four different options for how to resolve it. It's not easy. I think you think multiple choice, you think, oh, that's not that hard. Now these are hard. The multiple choice questions on the bar exam are really hard. I'd say for me, they were harder than the essay questions. Now,

what happened? Well, the California bar Exam administered this weird hybrid remote version of the bar Exam where there is a computer element to the bar Exam where basically, especially for the essay questions, you can have your laptop up there and it's connected into the system and you write out your answers. So there was this sort of hybrid remote system that was being used, and these electronic features are being used where people are doing the bar Exam

with their laptops. And in the February bar Exam and mission setting, a bunch of stuff failed electronically, just stuff wasn't connected, people couldn't connect, and people just lost out on the opportunity to answer a whole bunch of questions. Furthermore, it was then revealed after the fact that non lawyers utilizing AI had actually drafted some of the multiple choice questions on the bar Exam. Now, this is so bad

and so just just wildly inappropriate. First of all, because first and foremost, because AI is just not that good yet that we could trust it with something like this to develop a factual a series of factual scenarios that would be relevant enough for testing a prospective lawyer on their knowledge of the law. AI is just not good enough. It's certainly not good enough to do that without a lawyer checking up on it, and at a certain point

then it's not useful. You might as well just have the lawyer come up with it rather than have AI create it and then have the lawyer review it. I mean, you all, any one of you listening to this who have Googled anything, you know how unreliable AI is. Okay, I Google, you know, I'm I'm nobody special. I'm sure ninety percent of the population does this. But pretty Google now has like the AI summary for its search result findings that they stick at the top now of every

single stinking Google search. And it's so obnoxious because I feel like, you know, nineteen times out of twenty it just isn't useful or it isn't reliably useful. Because this is the thing with AI, people, I feel like over praise it and over laud it for how revolutionary it is. It's garbage in, garbage out. What AI does is it's just synthesizing all the information that is put into the

system and then it spits it out. You do a Google search for something, you might have five things that are relevant and ten things that are irrelevant, well the five If the irrelevant stuff is up high in the search results, that's going to influence the AI summary. So the AI summary then becomes partially irrelevant or partially not useful. It's garbage in, garbage out. That's how AI works. So you usually cannot trust AI if you have some sort

of simple task to give it. Okay, sure, write out some boilerplate language for you know, a thank you letter for you know, in my case, like for a nonprofit donor. Okay, sure, that's that looks fine. This is a low stakes thing. I can look at it and that's fine. The problem is the higher the stakes are, the less you can really rely on AI. And specifically this is true in

the legal context. Lawyers are are getting sanctioned more and more today because it gets found out that a lawyer used AI, used CHAP GPT, use some sort of AI system for writing or drafting a legal brief or doing something, and it's a breach of their professional responsibilities. I feel like the use of AI in the practice of law is fraught with peril, and nobody who's like really enthusiastic. All these techi people, they're so enthusiastic about AI Oh,

it's gonna revolutionize this, gonna revolutionize that. Yeah, it's gonna revolutionize You ruining your career because you tried to use AI on something important and you didn't double check it and then your client gets screwed. Why is your client paying you money for you to use chat GPT. The use of AI is fraud with peril for attorneys. I feel maybe there's some way that you can think of AI as like a search tool for beginning your legal research or something like that, but relying on AI to

write stuff is fraught with peril. And yet somehow, bizarrely the state bar thought it was okay to have a non lawyer using AI draft bar results. Now, this is so bad, Like the California Apparently people with the cap at the top level of the California Bar did not know that they had commissioned non lawyers using AI to

write bar exam questions. So the very test that you're qualifying new young lawyers, for these new young lawyers that you're gonna have to give all these continuing education for the rest of their careers are going to have to deal with the ethics, the how tos, the what to

avoids of utilizing AI. You are recklessly using AI on the very test that's designed to qualify them and having a non lawyer utilize the AI to write the test, which I'm sure probably cost a lot of money for whatever AI function was used, and makes me wonder why why would you waste time on it? Get some law Get like twenty law professors from different law schools up and down the state. They'll help out on a pro

bono basis. Have each one of them, you know, write you know, five multiple choice questions for the bar exam. Have them submit them. What are we bothering? Why are

we bothering? With AI? It probably would have been cheaper just to ask a bunch of law professors to help contribute questions, which I'm pretty sure is what they normally do anyway, at least for whatever portions of the bar zoom or state specific usually that there's I think California actually rejected the multi state multiple choice question that anyway, it boggles my mind. This and this is the thing I hate about AI insofar as I have opinions on it.

And this is this is a constant point of joking between myself, Jonathan Keller, and producer Colton who our team for the Right to Life radio show, which we air on Saturday mornings here on Power Talk. They love all new technology. They're Apple fanboys. Anytime a new iPhone comes out, they're like first in line to get it. And they're very enthused about AI and the opportunities and possibilities for AI and blah blah blah blah blah. Meanwhile, I hate

all this stuff. I feel like we're willingly walking into some kind of dystopian you know, terminator slash the Matrix slash I robot, you know, dystopian AI future where we're willingly handing over our autonomy two machines. And I just hate this idea, this mindless idea of wanting to shove AI into every single product, function, functionality humanly conceivable. Like I don't need AI to help me write a post on LinkedIn, of all places. I don't need AI to

help me on Facebook. I can waste time perfectly well on my own, thank you very much. I don't need a there's no possible need for AI in I don't need an AI summary thing in quick books. I want a human entering these ledgers. Like, no, I don't need AI for so much. Now when we return the California State. The California State Supreme Court also didn't know that the state bar folks were using AI to write multiple choice questions. That on top of the giant mess that was left

behind from the February bar administration. Now the California Supreme Court has to rule on are we going to lower the scores for passage to allow in a bunch of lawyers who didn't make the normal what we would have said as the normal qualifying score and what the impact of that is. That is next on the John Girardi Show.

So the California State Bar Exam that was administered this past February, and remember they only administer the bar exam for prospective new lawyers two times a year, once in February for either people who failed to pass the bar exam the first time or graduated in spring, and then once in July for people who graduated law school in May. So you only get two shots at this year, and

it's like six months apart. So it really janks up your life schedule if you fail the bar exam, or if the bar exam got so screwed up that you're in the state of limbo of did you or didn't you pass, and your you know, your professional prospects are massively altered. Thereby, you can't practice as a lawyer if you haven't passed the Bar Exam, and maybe you got hired by a firm out of law school and you're you're, you know, waiting to hear back on your Bar exam results.

It's a stressful thing and can be a hugely obviously, it's a hugely limiting thing if you don't pass the Bar Exam or if you don't know. Plus, by the way, it takes a couple of months to grade the Bar Exam. They administered the Bar Exam in February. They're supposed to be releasing Bar Exam results on May second. Now, as I described in the last segment, the California Bar Exam that was administered in February, and California has sort of

taken total control over its state bar exam. Most states have what's called the multi state portion of their exam, where this this entity provides all the states with sort of the same stuff for the multiple choice questions that are used on the Bar Exam. By the way, this is the hardest multiple choice thing you've ever seen in

your life. Okay, imagine the sat is on steroids. There's the multiple choice questions are usually sort of standardized across the country, and then your essay questions are state by state specific. Well, California rejected that. California said, no, we're we're taking control of the multi state portion ourselves, and the multi state multiple choice question portion ourselves, and the individual essay questions ourselves. Okay, that's fine, don't screw it up.

And they royally screwed it up. The basically the electronics system allowing people to take the test, the online electronics system failed massively for large numbers of test takers, resulting in a bunch of people who just weren't able to even finish the test or go through all the answers. And it was later revealed a bunch of the multi state a bunch of the multiple choice questions were written

by non lawyers utilizing AI. So we were now in this position where the California Bar examiners were having to go to the California State Supreme Court, which, for those who don't know, there's the US Supreme Court in Washington, DC. You know, Chief Justice Roberts Clarence Thomas Samlito. Every individual state has its own state Supreme Court. There's a California State Supreme Court, Utah State Supreme Court, Hawaii State Supreme Court, Montana,

et cetera. California's State Supreme Court overseas certain aspects of the legal profession, and because of the exceptional circumstances here, the California Bar Examiners are petitioning the State Supreme Court to lower the passing grade for this February bar exam, given how royally was screwed up. Results are supposed to be released on May second. Yesterday, it was revealed, though, that the State Bar still hadn't even submitted its brief.

They're screwing up left and right. On April eighteenth, according to this statement from the State Bar, on April eighteenth, the Committee of Bar Examiners made its recommendations for scoring adjustments for the February bar exam. The CB asked US California's Supreme Court to rule on a petition the State Bar would file regarding those recommendations by Monday, April twenty eighth. Well, the problem is the State Bar hadn't even filed the petition,

and they're supposed to be filing it today. The timing of the petition will not give the Supreme Court, the State Supreme Court much time they write to rule in a manner that allows us to apply the scoring recommendations adopted by the Court and then release February Bar Exam results on May second. There may therefore be a slight delay in releasing results. This is a huge problem. This

is a huge problem. So first of all, how it impacts your professional prospects if you're admittance to the bar is being delayed, delayed, delayed, Can I start this job that I want? Well, we don't know. And then secondly, there's this all right, The idea of the bar exam is that it's a barrier to entry. It's deliberately hard. On the one hand, it's testing your knowledge of the law. On the other hand, it's just testing if you're smart

and hard working. It's testing whether you've got the goods to really study, really, really, really hard and absorb a lot of material. Because why, well, being a lawyer is a really important responsibility. Your conduct, your behavior can be the difference between a murderer going free or a murderer going to jail. It can be the difference between you know, millions of dollars going rightly or wrongly one way or another. We need good, just competent people being lawyers so that

justice can be administered throughout society. It's actually a really important thing. And they make it hard to pass the bar exame because they don't want dummies being lawyers. And I'm afraid, like, if you're lowering the standard because the state and I mean, what else can the state bar do? They screwed it up. They screwed it up. It wasn't the test takers. It screwed it up. They did. So if you're lowering the standard for admitting people in, are

we just going to concede? Well, I guess we're just gonna let a couple of people who are not as smart as we would normally like let them become lawyers. You know, it's not good, It really is not good. Are we going to get people who are gonna give people add legal advice like that? It's a really bad thing.

And the state Bar is going to just apologize and that Here's well, the one thing that could really give the state bar some real pain and maybe lead to some real reform here is the fact that these are law students whom you have screwed over. This is a litigious group. So and and you've got a lot of lawyers all up and down the state looking at this with great interest about, Hey, how can I sue the

state bar. You know all these bar dues they're collecting. Hmmm, maybe that's a nice pool of money for me to try to sue. Let's see if I can put a slap a class action lawsuit together. Like yeah, this is a dicey situation for the state bar. All right. When we return, one of the big issues in the upcoming papal election China. That's next on the John Girardi Show. An underrated issue in the upcoming election of the Pope

may well be the issue of China. One of the more significant things that happened under Pope Francis's papacy was an agreement, a kind of concordat that was signed between the Catholic Church and the Chinese government. And it had to do with what influence, what power was the Chinese state going to have in the selection of bishops, and what sort of regularized non regularized status would there be

for the Catholic Church in China. Now for years up until now in mainland China, there was this division between the state approved I think it was called the Chinese Catholic Patriotic Association or something, so the state approved Catholic Church in China, and then the underground illegal Catholic Church like the actual Catholic Church, which would be subject to

sporadic harassment persecution by the Chinese governmental authorities. The Vatican and China had broken off any kind of diplomatic relations after the revolution and Malo Zetung and a ton of Chinese Catholics were martyred under the Communist regime and one of the during the Benedict papacy, one of the real stalwart defenders of the underground church in China was the Bishop of Hong Kong, who is a cardinal, Cardinal Zen. Yes, his name is Cardinal Zen ze En, not Cardinals Zin.

Cardinal Zin zi Yan had as a mullet and wears wrap around rainbow colored shades and cut off you know, tank top and cut off blue jeans and smokes Marlboroughs and goes dirt biking with his buddies. No, just kidding, there's no Cardinal Zen. There is a Cardinal Zen though, Okay, Cardinal Zen was the Bishop of Hong Kong and was a staunch defender of the underground Catholic Church and a

critic of the Chinese government. Well Francis's papacy. And by the way, I'm not I'm gonna just report on this, I'm not really giving much of an opinion because, for one, diplomatic relations between the Catholic Church and various kinds of governments have always been dicey, and you often have this situation of the Catholic Church trying to make chicken salad at a chicken you know what, where they've got a hostile government, they've got a anti clerical or anti religious government.

They want the Catholic Church to exist within that government, and maybe certain kinds of concessions have to be made as far as like over you know how bishops get appointed,

what you know, stuff like that. And the Church wants to have regularization with all of the Catholics in China who were part of the Patriotic Catholic Association, the state approved church, and they're trying to figure out, well, is there a way that we can regularize relations and have these Catholics not be schismatic and have them be part of the actual Catholic Church. Is there a way we

can do that? And they developed this sort of concordat allowing basically the Vatican to also nominate bishops for the official state run Chinese Church. So I get what's motivating Pope Francis and most particularly Cardinal Pietro Parolin. So, Pietro Polin is the papal Secretary of State and is clearly seems to think of himself, or well, I shouldn't say think of himself. He is clearly a candidate for Pope.

Pietro Poolin. He's a secretary of State, which is basically he's in charge of the Vatican's relationships with foreign officials, and the Secretary of State actually has a lot of sort of power within the structures of the Vatican itself. And it's clear that he is a candidate for Pope. So he was critical under Pope Francis and signing this agreement with the Chinese Communist Party regarding the above ground Church.

The criticism of this agreement, though, and one of the big critics of this agreement was Joseph Zen, the cardinal Bishop of Hong Kong, or the former he's now retired, he's like ninety years old, but he was extremely critical of this deal, and here he is in Hong Kong. Basically he said, well, you've you're you're conceding so much to the Chinese, Like are you going to be okay? I mean, are the is is the Chinese government going to vet bishops to make sure that they're you know,

supportive of communism? Like are we really going to be sure that bishops that are jointly appointed, that are okayed by the Chinese state and appointed by the Vatican, that they're actually going to reject communism? Are we going to have pictures of Xi and Mao Zedong? Are we gonna have pictures of Jijinping and Mao Zedong hanging up in Catholic churches? And what about all the underground Catholics? Did

you just leave them all out to dry? Did you basically just say that their years of sacrifice meant nothing? So I can see both sides of the debate. I can see on the Vatican side and Cardinal Parolin's side, we wanted to regularize relations with all of these Catholics who are part of this above ground like schusmatic Chinese government controlled organization. We want to bring them into the

fold of the Catholic Church again. On the other hand, here's Cardinal Zen saying, hey, you know, here's all these underground Catholics who've suffered so much under the threat of persecution. Why aren't you helping them? And it's not an academic exercise. For Cardinal Zen, so as I said, he's the Bishop of Hong Kong. For those who are not aware, the last fifteen years for Hong Kong have not been great. Hong Kong had been a British controlled territory along you know,

right next to the communists controlled Chinese mainland. Hong Kong was democratic. Hong Kong had robust British inherited systems of legal, due process and common law. Hong Kong was a relatively free place. The British did this handover. I remember this in the late nineties. The British formally gave back Hong Kong to the Chinese. I guess they had had like one hundred year lease or something, and they agreed to abide by it. I don't know why they agreed to

abide by it. I think that was silly. The Chinese gave some the Communist Party in China. The Chinese Communist governing authorities gave some sort of assurances that Hong Kong's current you know, they're sort of British inherited legal system and system of freedoms, etc. Would be maintained. And in the last fifteen years those promises have all gone out

the window. Hong Kong is now much more aggressively under the control of mainland China and mainland Chinese communist attitudes about free speech and etcetera, etcetera, etcetera to such an extent that Cardinal Zen, Joseph Zen was arrested by the communist authorities and he's basically he was under house arrest. He got permission from the communist authorities to go to the Vatican for the papal election, but he was under

house arrest. A lot of people have noted the case of Jimmy Lai Lai, who has been arrested and arrested and imprisoned by the communist authorities. He's a media mogul in Hong Kong, a pro democracy guy, and a Cardinal Zen has been very supportive of him in his persecution by the communist authorities. He's a Hong Kong businessman and politician. So today, basically the cardinals are all meeting in the Vatican today, So you have something like two hundred plus

cardinals in total. Only the cardinals under the age of eighty are allowed to vote, but all of the cardinals, even those who are over eighty, are able to come to the Vatican for these discussions before the cardinals get locked down and start voting on the pope. The cardinals will start voting for the pope on May seventh, so in about a week. So cardinals En is apparently going to talk today and the cardinals are going to have to decide, well, what is the future for the Catholic

Church in China. Are we going to keep abiding by this agreement that gives the Chinese government you know, a lot of sway in the choice of church leadership. Does it do so at the expense of, you know, weakening down the faith? I mean, on the one hand, we want this schismatic Chinese patriotic church to come back into the fold of the Catholic faith. We want all these you know, gazillions of Catholics who are part of it

to be returned into Catholic unity. At the same time, or was the Vatican diplomatic team headed by Cardinal Perilan where they played for fools by the Communists. Did the Communists get everything they wanted, a submissive Catholic Church supporting communism and we didn't necessarily get that much back in return? Did we throw the underground Catholics under the bus again? I'm not gonna John Girardi is not making a huge judgment about this. I can see both sides of the issue.

I'm I'm probably a little more inclined towards Cardinals Zen's perspective. But this is a huge issue within the election of the next pope is what will be the actual practical political posture of the Catholic Church, not not just against China, but against there are other countries that are having sort of very host countries in Latin America, et cetera, that have very hostile relations with the Catholic Church who want to meddle with the appointment of our bishops, which is

not a new thing, especially since the French Revolution. This is something that this has been a problem that the Catholic Church has had to deal with with long time. We've had a lot of less than ideal situations that the Catholic Church has lived under the Soviet domination of much of you know, eastern Central Europe for example. So this is going to be a huge issue, and you know, I hope, you know, I hope cardinals En has something thoughtful to say to all those cardinals. You know, he's

ninety years old. I don't think he's got much of a filter anymore, and maybe that's good. On the other hand, they're going to hear a lot from Cardinal Perland, probably trying to defend his own record. So it'll be very interesting to see. It's going to be a major issue I think in the upcoming papal election when we return. Money another big issue in the papal election, basically the bad state of the Vatican's finances. Next on the John

Girardi Show. One of the other big issues with the papal election so the cardinals are going to get locked down and start voting on May seventh, is money. Vatican's finances are in bad shape. Donations worldwide, the Vatican gets some donations from the Catholic Church, all of the world are down. The Vatican keeps running a deficit, and one of the sad things was under Pope Francis, he appointed

this Australian cardinal, George pell. He was a very competent man Oxford doctorate holder, very conservative guy, beloved by Benedict the sixteenth to revamp the Vatican's finances, and the Italians basically stymied him at every turn. He was not really able to affect the kind of economic and financial transparency

reforms that were needed. I think popes have had a hard time wrangling various lousy Italian pres bishop's, cardinals and their investments of money and their dealings with shady Italian characters, especially for various kinds of investing opportunities, and it's resulted in the Vatican losing a lot of money on stupid investments. So obviously it shouldn't be the issue, but it's going to be. Ah issue is the finances of the Vatican,

their unfunded pension liabilities and just their donation deficits. It's a real issue, and I hope they can get someone who is a competent person as well as someone who is a holy and smart person that'll do it. John Girardi Show, See you next time on Powered