Welcome to the Bar Exam Toolbox podcast. Have you ever wondered what it's like to be a bar grader? Well, today we have Jennifer Barry from the Bar Exam Toolbox team to share her experience. Your Bar Exam Toolbox hosts are Alison Monahan and Lee Burgess, that's me. We're here to demystify the bar exam experience, so you can study effectively, stay sane, and hopefully pass and move on with your life.
We're the co-creators of the Law School Toolbox, the Bar Exam Toolbox, and the career-related website CareerDicta. Alison also runs The Girl's Guide to Law chool. If you enjoy the show, please leave a review on your favorite listening app, and check out our sister podcast, the Law School Toolbox podcast. If you have any questions, don't hesitate to reach out to us. You can reach us via the contact form on BarExamToolbox.com, and we'd love to hear from you. And with that, let's get started.
Welcome back! Today we are welcoming Jennifer Barry, a member of the Law School Toolbox and Bar Exam Toolbox team, who is sharing some insights from her time as a bar exam grader. I feel like that's such an elusive title that people know you exist, but they have no idea who you are. So thanks for joining us. So, to get things kicked off, could you share a little bit more about yourself, your background, and how on earth did you become a bar grader?
Sure. So, I've been practicing for almost 20 years now. Back when I first passed the bar, I got my dream job working for the California legislature- that was in 2005. But as a state employee, especially a state employee that is a brand new attorney, I just did not make very much money. And I was also a single mom at that time and definitely was thinking, "I've got to figure out a way to supplement this."
I started looking and came across a listing, I think on Indeed.com or something like that, and thought, "Well, I could do that." And I applied for it, and the process for the application was pretty simple. I don't think it's even changed. I was looking at it a while back, and it doesn't even look like the application is very different now from 2005.
So, I applied for the July 2006 administration, and I was selected as an apprentice grader for the Professional Responsibility panel for that test cycle. And that typically means that you wouldn't grade any exams, you would just sort of be in training. Do all of the things that a grader has to do to prep to be able to grade exams, but you actually wouldn't get any exams in your possession. But as it so happened, I did end up grading that cycle.
We needed more graders than were officially chosen. So, I started and from there on, I pretty much graded for, I forget how many cycles now, it was a lot though. It was 11 years and there were only a couple of cycles that I didn't actually grade during, because of some trades or vacations, things like that. Then
you probably graded my bar exam. And I'm sure you put a big sparkling star on the top for how great of a
job I did. Right. Amazing job.
Amazing job. So, you graded for the California bar, and I know that individual jurisdictions have slightly different processes of how people go through their grading process, but can you share a little bit about kind of the typical bar grading process in California? I will say when I started learning about this, I was surprised that it was a pretty big process, like a much bigger process than I think most people appreciate.
Yeah, absolutely. It's pretty intense. And that's part of why I ended up stopping from grading, because it was so intense and time-intensive for me as my son got older. I just couldn't continue to spend that much time in the Bay Area, which was physically required.
So, when you're first assigned to a panel, I think it's within usually a couple of days of the bar being complete, everybody who has accommodations and everything get finished, and then they send out a packet to you as the grader to you get your question, and you're required to do a full research to answer.
So, I would spend my time... I don't remember exactly how long I had to do an answer like that, but I spent a significant amount of time doing full law school level research to answer the question, case cites and statute cites and everything. And everybody would send that to all the other graders in the same day. So everybody would get everybody's answers and then you were expected to read them and have a good sense of what should be expected of the students taking the exam.
And you would meet a couple of weeks or so later in the Bay Area, as a group, usually it was at a hotel. There were a lot of graders, so we'd have a ton of different rooms with each question assigned and the panels were about 12 to 15 people with a leader. And then sometimes somebody from the committee itself would be there to observe.
And you would spend the entire day going through both the answers that were submitted, the proposed answer that the leader would have come up with at that point, and everybody would come to an agreement about what should be in the exam answers, and what points should be allocated to each thing. And then in the afternoon, you would grade a handful of essays independently. In the morning, you also graded essays together to come to an agreement of, "Okay, this is a 55", "This is 75."
In the afternoon, you'd all grade them independently, and then those scores would be submitted and a whole statistical analysis would be done, and you'd come back the following weekend, same setup. And basically, you would go through in the morning, who was the outlier? You did not want to be an outlier if you could. You wanted to stay in that sort of middle of the bell curve place, but there were always people on the outside on one or two questions, who would grade either too high or too low.
So, you would be sort of told you need to adjust your expectations or you need to be harsher, depending on where you were on that bell curve. And so that was the process for the first couple of days, and the theoretical result of that should be that everybody should walk out of that room knowing exactly what a 55 would look like on an ex am. And then everybody else got the same exam, we'd all grade within like five points of each other on either side. So, basically a 10-point span.
And usually after that one, you'd start the grading right away. They would provide you boxes of exams, and there would be a return period. And then you'd return again a few weeks into it, maybe like a month into it, and just verify that you all are still on the same page. So, you would grade exams again, to make sure that as the cycle progresses, nobody's going too high or too low, and that we're all still agreeing on what we're seeing. So, that's the process for actually getting to the exams.
And then you would actually have an exam. You would get a box of a hundred exams and you would dive into that. And, like I said, it was a staggered turn-in time period. So, you kind of had to plan ahead and set aside... I think I usually set aside four or so hours for a box. And you just dive into the exams. I can talk about too, sort of what I looked at the exams, if that's something want to...
Yeah, yeah, I want to do that too, but I think this is a really important point. There are two interesting points that I want to call out for people. One is, at least in California, I think everybody thinks that there's some perfect model answer when they get those tests in the room. And there is no answer at that point, and that's pretty wild. And I think a wild concept for people to sit with.
And then the second thing is, I think it's very powerful, because although I have not been a bar grader, I have taught and I have graded a stack of 50. I taught bar prep classes, so I got the stack of questions. And it is an interesting thing to sit and read a hundred versions of the same answer over and over and over again.
And when you're writing your answer, I think that's an important thing to keep in mind, that you are writing to a grader that has studied this question, but also is spending four hours on a box of a hundred questions. And you kind of want to write to that person. And I think that can be a helpful mindset for anyone in any jurisdiction to think about, is, there is a lawyer somewhere who is going to be sitting with your box of essays, reading one after another after another.
And you want to make their job as easy as possible. There's something about crafting an answer for that audience that I think is really important for people to recognize.
Absolutely. I think one of the things that is interesting about the process is that the essays start to almost fall into a handful of types of essays that you would see too. You obviously want your essay to be in one of the types of essays where I know this is going to be in that 60 to 70 range for sure, the moment I start reading it. And usually that would be the conclusion. So, the goal is that you're writing an essay that I can read easily, and I don't have to fight to give you your points.
And as a grader, I would fight to give people points through my process, but I don't know that everybody was willing to spend as much time doing that. So, that's something to keep in mind. You just want to make it easy for your grader to hand you points as easy as possible.
Well, that's a great segue. So how do you make it easy? How do you make the grader's job easy? How do you make them happy when they pick up your paper?
I think one of the things that probably drives students a little bit crazy is that IRAC, right? Constantly being told you need to IRAC this. And it is something that you need to do in most legal settings throughout your career, so kind of get used to it.
But it's very helpful to follow that direction when you're writing essays, because otherwise, if you're just sort of writing in a flow without thinking about setting out exactly the title of the section you're doing, your issue, and then the rule that you're going to be applying, making it super easy for your grader to see that you understood the rule, and then giving a good analysis that really does go through and apply that rule step by step. And a conclusion I could take or leave.
It didn't matter too much if you forgot to conclude, because I could usually tell what you were concluding through your analysis. But in particular, the I, R, and A portion make it very easy for your grader to be able to assign points. Making sure that you actually hit on every one of the important things in the question. And then of course, also I'm sure annoying for students to hear over and over again is, you do need to know the law.
That said, if you don't know the law, you can still do pretty well by drawing on your own experience and understanding of the law in general, so long as you do it in a way that sounds confident.
Right, exactly, yeah.
forward Saying things that may or may not be totally correct, but are close enough.
Yeah. Years and years ago, when I first started writing about the bar exam, I wrote some blog posts that your answer needs to be wearing a suit and not sweatpants - this idea that you have to kind of dress it up and pretend like you know what you're talking about, which is exactly what you do when you're a lawyer, because every lawyer gets caught off guard in some meeting and does not know exactly what they're talking about.
But part of being in the outfit and in the persona is that you just present yourself as in that. And I think that you can do that in writing. I have always been surprised at how you can feel the anxiety or the uncertainty in someone's work, even when you have no idea who they are or have any information. But you can feel that they're running out of time, or you can feel that they are lost. And giving it that idea of like, "I'm going to put it in a tailored suit.
It's going to be organized, it's going to follow IRAC, it's going to hit all these points" can kind of dress it up. And then you're not like waving a flag to the grader saying, "I'm out of time, I don't know what I'm talking about, I am flustered", because that can't help the grader give you all of these points, like, "Well, this person's ready to go practice law."
Yeah, absolutely. There were students who were like, "I'm very nervous" or, "I'm running out of time." That can be helpful to know. It wouldn't necessarily do you any favors in the process. So yeah, pretending, essentially, that you know what you're talking about, and saying the words in a way that makes it sound like I should believe you. I should start to question whether I know what I'm talking about, based on what I'm reading from what's coming out of your fingers.
I think in that process, you can say something that's not quite right and still get a good portion of the points for that section, especially if your analysis is consistent and makes sense. If you say something just absolutely wrong, we definitely had moments where we said things in our group meetings where I was like, "Well, this person's just lost. They just very clearly don't know what the rule is here."
But if you're able to gather on what you know, what you can remember, you usually can kind of find the plot somewhere in there. So
Yeah. I remember on my bar, I had to make up some law. I mean, almost everyone has an, "I made up law on my bar" story, because there's often some wacky question that tests law that most people don't know. And I do remember being like, "Well, I'm sure that there is some reasonableness element to this. I'm just going to hang my hat on reasonableness" and go for it, if all else fails. But I do think that you have to practice being confident in that.
And one of the things we talk to our students about is you can't only do essays where you know the law, because you have to practice the art of dressing it up and pretending like you know what you're talking about.
Yeah, absolutely. Absolutely. I think that it's really important to practice essays that look scary to you. I had an essay in my exam as well that I went and looked back this year to see what it was and I'm like, "Huh, that really threw me." Now I probably know what I was trying to say, but it definitely threw me. And it threw everybody else too, because they were all talking about it as we walked out.
Yes. Yes. At the time that I took the bar, most people took a BARBRI lecture, the same lecture, and he had this saying of like, "If you don't know it, nobody knows it." And after that session, someone in the room of the thousands and thousands of people at the Oakland Coliseum yells into the room, "If you don't know it, nobody knows it!" And everybody just started laughing, because we all knew what he was talking about.
It's true. I remember walking out of that room thinking, "Okay, well, at least nobody here knows what the heck just happened." I was mortified in the moment, but now looking back...
Yeah. So, what do you think are some of the common misconceptions that people have about bar graders or the bar grading process?
Yeah. One of the things I think that I've heard a lot of over the years is, is it true that the graders don't want you to pass? That's something that a lot of people seem to think. Or, is it true that the graders just don't really pay too much close attention to the exam? And honestly, neither of those things are at all true. The people that I worked with all took this job very seriously. It was a very time-intensive job and it was not the highest paying.
So, you had to be willing to really be committed to the process and to making sure that we were helping sort of pass the people that deserved to pass and to provide negative feedback, I suppose, in the form of not passing people who didn't have a good grasp yet. And I personally took it very seriously. Obviously, it was something that I did for a very long time and I enjoyed it. I enjoyed the process and I thought that I was doing an important service for the bar.
And as far as whether or not I wanted people to pass or not, I actually went in almost every box like, "Alright, how many 80s can I get?" And so, I would get excited about the higher-scoring exams. I didn't like giving 50s and 45s, and 40s sometimes. And so, it was not an enjoyable thing for me to not pass an essay - quote, unquote, "pass an essay", since no single essay is a passing thing. But not awarding a a passing score was not my favorite. I really liked giving like 70 plus.
Yeah. Yeah. And for those who are not in California, a 70 is a pretty good score. I mean, an 80 is a really good score, and a 70, like you're humming along, you're doing pretty great. Most people are trying to at least get that 65, but if you're in the 50 and the 55s or below, if you do that across the board, mathematically, it becomes very difficult to pass. So you talked about it a little bit, that the bar set up this process to try and create consistency and fairness amongst all the graders.
Over the years, I have read a lot of answers probably not as many as you, but I've read a lot. And I've looked at a bazillion score reports. And something they do in California is that if your score is in a certain pocket, you get something called a "second read", which means another grader comes in and does it. And sometimes you see there's only a five-point discrepancy.
Sometimes you see a larger discrepancy that can be very frustrating, because either the score goes way up, or sometimes it goes way down. How do you feel that the bar graders, individually or as a group, try to create that consistency? Because I think that when people see that happen, it makes them question the process. Yeah.
So, part of the goal of, like you were just describing, the goal of the second and third read is to create that more fair process. And of course, when you see that huge difference, you must think one or the other of them was completely wrong, and probably it was whoever gave you the bad grade.
But one thing to remember, obviously, is that we are humans, and so the graders may have just a totally different read on your essay, which was something that would even happen in our meetings as we were trying to calibrate our scores to each other. You would have somebody who would be like, "No, this is absolutely a 55", and everybody else in the room is like, "This is like a 60-65, come on."
And they wouldn't be willing to budge from the 55 until the person in charge would tell them they needed to. I think that's just something to keep in mind. There are going to be some hopefully minor differences between each of the persons reading your exam, which is why when we come out of that first two meetings, really, the goal is that if I read an exam and two other graders read the same exam, we won't give more than, if it's a 60 on average, a 55 and a 65, right?
So theoretically, your 55 could go as high as a 65, unless it's very obviously not a 65. The answer really is, that shouldn't happen. But it can, I think, just because it's the process, we aren't robots.
If you fail in California, you get a copy of your answers. And one of the things that I've seen - and I think this is applicable to all bar graders - is it really goes back to what you were talking about earlier, which is the tight IRAC making it very easy for the grader. I have read some answers that have had a gap, like a larger gap, from the graders, and it's typically a hard essay to read.
And so I think you could see how someone could get in it and just be like, "Ah, this is not what I'm looking for. This is not in the suit, I can't even follow this", versus someone who maybe slowed down and decided they wanted to kind of parse through it, because sometimes the nuggets were a little more hidden.
And I think that that is a great rule of thumb for someone who sees that on their essay, but it's just that great reminder - you can't hide the stuff in these long paragraphs, or these lengthy statements, or this brambling language, because you risk the grader doesn't see it, and then that can hurt your score.
That's exactly, I think, a really good point about this, because essentially, if a grader reads through your exam and based on that first read, they may not be willing to give it a thorough second read. One way I tried to make sure my scores were always consistent one exam to another, and also with our rubric was I created a spreadsheet that I was very careful to check off and give pluses or minuses.
So, if this area was worth a certain number of points, a check mark meant that you got essentially like half the points, and then you'd get a minus if you were just under half, and plus if I thought you got all of them. I had a very complicated personal system that I used to make sure that I was catching things. So, if I got through and I thought they talked about that but I have no check mark there, I could go back and look and see, did they actually get on this or not?
But not everybody did it that way. Some people just would literally sit and read it with the rubric in mind, and others would just read it with the list. We had like an outline or something, that kind of thing. And so we all kind of did it a little differently based on our own personal preferences, but it made it easier for me to identify where I finished an essay and would go, "I thought it was going to be like a 65, but I'm only finding 55 points. How weird."
And I could go back and find more points. But if your grader doesn't have the time to do that, or frankly, the desire to be that thorough, you may not get that extra point. So, I think that that is a really good point there.
Yeah. I read one kind of heartbreaking one, where there were five elements that needed to be discussed, and the student numbered all but one of them. And I think one of the graders didn't think they talked about all five elements.
They just put one, two, three, four. Okay, I only see four. Yeah.
Right. And so, it is those little things. That's not necessarily going to make or break your whole test, but I do think that sometimes we get so caught up in how complicated everything is, that we miss some of the easy stuff, the low hanging fruit that you can do to make it very easy for somebody to grade your thing, like If there are five elements, please list all five. And if you're going to use numbers, don't forget to number one of them. Somebody might then assume that you didn't catch it.
Formatting of your essay can help too in that realm. Like, if you have a rule statement and it has five elements, putting each element on its own line is helpful. And then that way, if you didn't remember the last one, it's not that big of a deal because it's on its own line.
And I think that formatting, when you're under that time pressure, can help keep you grounded as well. We talk about with our students doing an outline, either on scratch paper or on the screen. But sometimes for myself, I will have a list of like, "Okay, I have these five elements." Maybe they just say 1, 2, 3, 4, 5, because I have the elements in my head, but then I still check them off as I talk about them to make sure that I've got the five elements.
And I think it just keeps you really on task. And for students who have any attention deficit issues or any learning differences, that makes that very linear writing more challenging.
I think that structure - boy, it just really helps keep you on task because these little errors, I think, are easier to make for someone who may have ADHD, who may have a learning difference, because if one of the issues is their brain is creative and thinks outside the box, unfortunately, the bar is very in the box. And so, you have to bring yourself back with scaffolding to stay and produce the linear thing that the bar is asking for, and create that scaffolding for yourself.
That's really all you can do.
Yeah. There's the idea of not fighting against the structure that you're expected to follow. There's a time and a place for extra creativity, but it's not on the bar.
Yeah, that's just not the place. You just want to be one "many in the pile", right? You don't want to be the outlier. Please join us next week for Part 2 of our conversation with Jennifer Barry. And with that, we’re out of time. If you enjoyed this episode of the Bar Exam Toolbox podcast, please take a second to leave a review and rating on your favorite listening app. Wed really appreciate it. And be sure to subscribe so you don’t miss anything.
If you have any questions or comments, please don't hesitate to reach out to myself or Alison at lee@barexamtoolbox.com or alison@barexamtoolbox.com. Or you can always contact us via our website contact form at BarExamToolbox.com. Thanks for listening, and we'll talk soon!