1 1 UNITED STATES DISTRICT COURT EASTERN DISTRICT OF MICHIGAN 2 SOUTHERN DIVISION 3 4 BARBARA GRUTTER, For herself and all others 5 Similarly situated, 6 Plaintiff, 7 v. Civil Action No. 97-CV-75928 8 LEE BOLLINGER, JEFFREY LEHMAN, DENNIS SHIELDS, and REGENTS OF 9 THE UNIVERSITY OF MICHIGAN, 10 Defendants. _________________________________________/ 11 12 BENCH TRIAL - VOLUME 11 13 FRIDAY, FEBRUARY 9th, 2001 14 15 BEFORE THE HONORABLE BERNARD FRIEDMAN United States District Judge 16 Theodore Levin United States Courthouse 231 West Lafayette Boulevard, Room 238 17 Detroit, Michigan 18 - - - 19 Appearances: 20 Kirk O. Kolbo, Esq., 21 R. Lawrence Purdy, Esq., 22 On behalf of the Plaintiff, 23 24 John Payton, Esq., Craig Goldblatt, Esq., 25 On behalf of the Defendants Bollinger, et al, 2 1 - - - 2 APPEARANCES (Continued): 3 4 George B. Washington, Esq. Miranda K. S. Massie, Esq. 5 On behalf of Intervening Defendants. 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Joan L. Morgan, Official Court Reporter 21 Proceedings recorded by mechanical stenography. Transcript produced by computer-aided transcription. 22 23 24 25 3 1 2 I N D E X 3 - - - WITNESS: PAGE: 4 EUGENE GARCIA 5 Direct Examination (cont.) by Mr. Washington 15 6 Cross-Examination by Mr. Payton 70 Cross-Examination by Mr. Kolbo 83 7 Redirect Examination by Mr. Washington 108 8 DAVID WHITE 9 Direct Examination by Ms. Massie 111 Voir Dire Examination by Mr. Kolbo 118 10 [...] 13 D A V I D W H I T E , 14 being first duly sworn by the Court to tell the truth, was examined 15 and testified upon his oath as follows: 16 DIRECT EXAMINATION 17 BY MS. MASSIE: 18 Q Hello, Mr. White. 19 A Hello, Ms Massie. 20 THE COURT: I'm going to let you lead with this 21 witness and the remainder of your witnesses so we don't have 22 to be cumulative. I don't know if the testimony would be 23 cumulative but -- 24 BY MS. MASSIE: 25 Q So I'll lead and ask you whether you went to Boston GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 111 1 College? 2 A Yes, I did. 3 Q And when did you graduate from law school? 4 A From law school I graduated in 1973. 5 Q Tell us about your career since then. 6 A After graduating from Harvard Law School I traveled in 7 the next few months to Berkeley, California and began working 8 at something called the Childhood and Government Project which 9 was an adjunct of part of Boalt Hall, School of Law. It was a 10 multi-disciplinary organization, drawing from faculty from the 11 law school, from the school of education, and school of public 12 policy. 13 Q And you were there until 1979? 14 A Yes. 15 Q And tell us what you did from 1979, to 1985. 16 A At that point, I was doing a -- directing a study called 17 the law school admission investigation under the auspices of 18 the National Conference of Black Lawyers. And we received 19 funding from the Spencer Foundation of Chicago, Illinois, and 20 the National Institute of Education to evaluate the law school 21 admission test in terms of its validity and its bias. 22 Q And you were the principal investigator for that project? 23 A Yes. We hired staff and we commissioned papers, and held 24 a national conference, and published a book that complied those 25 studies. GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 112 1 Q And in 1985? 2 A In 1985, I founded testing for the public as a non profit 3 education research corporation in California. 4 Q Just tell the Court what Testing for the Public does. 5 A We do research on standardized testing and we also offer 6 low cost test preparation courses on the law school admission 7 test, the graduate management admission test, and graduate 8 record examination. 9 Q Are your programs targeted for any particular racial 10 groups? 11 A As our name implies, our courses are opened to the public 12 but from the inception of Testing for the Public we've always 13 advertised that we were trying to help women and minority 14 students with the standardized tests and as a result of that we 15 have been asked by a number of universities and universities 16 departments to provide test preparation courses particularly 17 for the graduate record examination. 18 Q What universities currently sponsor Testing for the 19 Public courses? 20 A The University of California sponsors courses through 21 five different subdivisions at Berkeley. And the University of 22 California at Santa Cruise sponsors courses. San Francisco 23 State University sponsors courses. California State Fullerton 24 sponsors courses. And for the last several years I have been 25 spending a week in Jamaica, New York teaching a course at York GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 113 1 College which is part of the City University of New York. 2 THE COURT: The California courses are for 3 minorities only, under-represented minorities? 4 THE WITNESS: The ones that are sponsored by 5 universities are under-represented groups. They typically are 6 minority students, but in the field of the sciences they also 7 include females which as you know are very under-represented 8 in those fields. 9 THE COURT: And they're sponsored by the California 10 universities? 11 THE WITNESS: The universities contract with us to 12 offer the courses, yes. 13 BY MS. MASSIE: 14 Q Let me take you quickly through your publications, you've 15 had articles published in the "Harvard Educational Review." 16 A That's correct, twice. 17 Q And you've published a book on the LSAT. You were the 18 editor and director of a series of projects that was a 19 published as a book on the LSAT. 20 A That's right. 21 Q And you published various law review articles on testing, 22 bias, testing validity and with a focus on the LSAT? 23 A That's a correct. 24 Q And then a major research report for the National 25 Conference of Black Lawyers called "The Effect of Coaching, GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 114 1 Defective Questions, and Cultural Bias on the Validity of the 2 SLAT." 3 A That's correct. 4 Q In addition to various test preparation manuals that 5 you've prepared for your students over the years? 6 A Certainly. 7 Q You've also given various academic presentations 8 including the American Association of Law Schools. 9 A Yes. 10 Q The National Institute of Health? 11 A Yes. 12 Q The Society of American Law Teachers? 13 A On several occasions. 14 Q And various other academic presentations mostly again 15 focused the LSAT, but also focused on other tests and the 16 bias-- sources of bias in other tests. 17 A That's correct. My other major focus has been on the bar 18 examination. 19 Q In your published work and also your presentations, and 20 I'll get to your previous testimony before -- your legislative 21 testimony, you focused on national aggregate data as well as 22 question content analysis for particular questions but a whole 23 range of data sets for understanding the validity and bias of 24 tests like the LSAT. 25 A That's correct. GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 115 1 Q Tell us about the legislative testimony you've offered. 2 A I testified before a legislative committees of the New 3 York State, Texas State and California State legislatures. The 4 most recent testimony incorporated some of the material we'll 5 be discussing today which was national information and 6 information at Boalt Hall Law School. 7 Q And you've served as a consultant both to the Florida 8 Supreme Court and to the Department of the Army; correct? 9 A Yes, the Florida Supreme had a major undertaking 10 investigating bias of the judicial and legal system, and they 11 asked me to evaluate the Florida portion of the bar examination 12 which is a multiple choice test. And the Department of the 13 Army was interested in evaluating the armed forces 14 qualifications test. 15 Q In addition to your research, your academic 16 presentations, your consultancies and so on, you have taught 17 many hundreds of students over the years, trained other 18 teachers and logging testing for the public as a research and 19 education non profit. 20 A That's correct. 21 MS. MASSIE: Judge, I would ask that the Court 22 certify Mr. White as an expert on testing on bias issues. 23 MR. KOLBO: Your Honor, I acknowledge that the 24 witness has written on the subject, and has testified in front 25 a public body, but I don't think she's given a foundation that GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 116 1 he has a specialize knowledge and training to testify about 2 test design, psychometrics and test bias. 3 THE COURT: Does he intend to testify as to those 4 three areas? 5 MS. MASSIE: He's going to testify about test bias. 6 He's not a cycle nutritionist, and we're not asking him to 7 testify -- 8 THE COURT: Why don't you lay some foundation as to 9 what his expert -- how he has obtained his expertize. 10 BY MS. MASSIE: 11 Q The publications, and the presentations and the 12 consultancies that we've just gone through all involve the 13 question of bias in standardized testing; correct? 14 A That's correct. 15 THE COURT: How did you obtain the knowledge in 16 order to write these papers and to give the information? 17 THE WITNESS: As I indicated, your Honor, the first 18 knowledge that I had was based on national data about the 19 discriminatory impact of the LSAT particularly compared to 20 undergraduate grade point average. 21 THE COURT: Other persons researched and so forth? 22 THE WITNESS: It was national data that I was 23 evaluating. There was also additional research. I've read as 24 much as I possibly can about the law school admission test. I 25 keep up with the literature on the changes on the law school GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 117 1 admission test. I attend the National Council on Measurement 2 and Education every year. 3 The other way in which I understand bias is through 4 the courses that I offer or where I am talking in particular, 5 person-to-person conversations with students of variety of 6 backgrounds with individual LSAT questions and evaluating how 7 those individual students react to those questions both in 8 terms of accuracy and in terms of interpersonal reactions. 9 THE COURT: It was on-the-job training. You haven't 10 received a degree or haven't studied any specialized study -- 11 THE WITNESS: There would be no place in the country 12 that I would be able to go to study the things I've been able 13 to learn in the test preparation courses. 14 MS. MASSIE: Judge Friedman, if I could, if we can 15 -- the studies that Mr. White will be presenting today, I 16 think there won't be any dispute that he's qualified to 17 testify about them. He directed the work that we'll be 18 presenting today. 19 THE COURT: Any questions you would like to ask? 20 MR. KOLBO: If I could ask him a few questions. 21 THE COURT: Sure. 22 Peer reviews, have you ever had any peer reviews, 23 anybody ever reviewed your work? 24 THE WITNESS: I'm not sure what peer review would 25 mean. They've been published in "Harvard Education Review and GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 118 1 the "Harvard Civil Rights" -- 2 THE COURT: But by you. 3 THE WITNESS: It was submitted for publication by 4 me. It was accepted for publication by three journals at 5 Harvard. 6 VOIR DIRE EXAMINATION 7 BY MR. KOLBO: 8 Q My understanding is that your formal training is you have 9 a BA in English? 10 A That's right. 11 Q And you have a JD, you're a lawyer. 12 A That's right. 13 Q You have no formal training in statistics? 14 A A statistician doesn't know what I know, and I wouldn't 15 need to know a statistician's job to know what I know. 16 Q And I take it you have no formal training in statistics? 17 A No. 18 Q You have no formal training in education? 19 A No formal training, no. 20 Q Do you have any training in psychology? 21 A Not formal training, no. 22 Q Do you have any training in psychometric test design? 23 A That's nothing of concern for my opinions today, no. 24 Q So the answer is you don't? 25 A No. GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 119 1 Q Have you ever worked for one of the testing 2 organizations? 3 A No. 4 Q Have you ever been a consultant for them? 5 A If you consider the Florida State Supreme Court which 6 supervises the Florida Bar Examination which administers the 7 Florida Bar, I would have to say that I was. If you consider 8 the Army which administered the armed forces qualifications 9 test, I would say that I was. The fact is when people look 10 around the country for someone to investigate test bias I under 11 up getting a telephone call and receive a plane flight. 12 Q You don't have access to the data that the testing 13 companies have about whether or not there is bias in testing 14 based on the outcome of the tests they administer? 15 A The test questions are available and the statistics on 16 accuracy for the graduate record examination are available. 17 Q Do you have access to the outcome for the tests, for the 18 LSAT courses administered by Law Services? 19 A Oh, yes, the national data about discriminatory impact of 20 the SLAT is very available. 21 Q Have you ever been qualified by a court to testify as an 22 expert on test design? 23 A This is the first time that I've been called to testify 24 in court. I've usually been an expert for departments and 25 agencies. GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 120 1 Q What formal training have you had for the purpose 2 rendering opinions with respect to test design, standardized 3 test design? 4 A I'm not discussing test design today. 5 Q Are you going to discuss the issue of test bias and also 6 the racial test bias? 7 A Yes. 8 Q What formal training have you had in that area? 9 A As I've indicated before the information that I have is 10 not available in formal training. So that you would not be 11 able to find a professor that knows what I know about the 12 interaction between students and the LSAT items and LSAT tests. 13 Q Do you understand what psychometrics is? 14 A Yes, those are applied mathematicians. They have no idea 15 about psychology. They have no idea about education. They 16 study calculus. They have studied and statistics and they have 17 been hired by a testing company. 18 Q The two fields you said that cycle nutritionists have no 19 expertise in, psychology and education, you have no training in 20 those areas; correct? 21 A No formal training, but the experience that I've had over 22 fifteen years of dealing with students is part of the basis on 23 which I make my opinions. But most of the opinions that I 24 going to be stating are national data which are transparent. 25 MR. KOLBO: Your Honor, I just don't believe this GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 121 1 meets the standards for an expert witness. I make my 2 objection on that ground. 3 MS. MASSIE: Judge, Mr. White has been working on 4 these issues since 1979. He is a nationally recognized expert 5 -- 6 THE COURT: But if the figures as he's indicated are 7 transparent why do we need an expert? 8 MS. MASSIE: There's been testimony to suggest and 9 your own questions have suggested that socio-economic status 10 may be the real problem on the LSAT. We're going to convince 11 with Mr. White's material that it's not. 12 THE COURT: He doesn't have to be an expert. If all 13 these -- he's indicated all he's done is put some numbers 14 together -- 15 MS. MASSIE: He is an expert. "Harvard Educational 16 Review" is a peer review journal. He's been published in it. 17 He's been hired by the Florida State Supreme Court. He has a 18 wealth of experience that clearly qualifies him as an expert 19 under the rules. 20 THE COURT: I'll allow him to testify; however, with 21 the understanding that the plaintiffs -- please make an 22 objection. I'm letting him testify; however in a very limited 23 area. He doesn't have any special education or training -- 24 MS. MASSIE: You are making it perfectly clear that 25 you won't listen to our case on testing bias. He hasn't been GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 122 1 qualified as a court expert before because there's never been 2 this situation before, Judge. We have never been in this 3 circumstance before. It's totally unfair. 4 THE COURT: Please don't yell at me, number one. 5 Number two, throughout the whole you've treated me as the 6 enemy. I'm not the enemy. I have a job to do. I make 7 rulings. Don't yell at me. I don't yell at you, and I would 8 expect that you don't yell at me. I have to make rulings. 9 That's my job. You have a job to present your case. I want 10 to hear it. I'm doing everything I possibly can for all sides 11 to make sure because I want as much knowledge before I decide 12 this case as I possibly can. But I also have a duty and that 13 duty is to follow the rules. I'm not the enemy, and I don't 14 appreciate you yelling at me like that. Go on. 15 MS. MASSIE: Judge Friedman, I apologize for raising 16 my voice. You promised at the outset of this case after Erika 17 Dowdell testified that you wanted to listen that we had to 18 present with an open mind. What you have just said about Mr. 19 White's expertise given the obvious and undeniable character 20 of his expertise to me indicates that you will not be 21 listening with an open mind to the things that he has to teach 22 all of us in this room about the SLAT. And they are critical. 23 They are imperative for our case. We can't make our case 24 without making that case. 25 THE COURT: First of all if I conveyed that to you, GRUTTER -v- BOLLINGER, ET. AL. BENCH TRIAL - VOLUME 11 FRIDAY, FEBURARY 9TH, 2001 123 1 I apologize. I only make rulings based upon what I hear. 2 That's my job. That's what I'm supposed to do. And it's not 3 a personal ruling. It's not personal to you or personal to 4 your case, or personal to Mr. White. But we have rules. You 5 would like to present your case, but you're going to present 6 your case just like the plaintiffs are and just like the 7 defendants are within those rules. And if you believe that my 8 rulings are indication of the outcome of this case or anything 9 of that nature you're just absolutely wrong. But you're not 10 going to take those rulings and try because of the tone of 11 your voice to intimidate me because I won't be intimated. I'm 12 going to make the rulings the way I feel. I have already 13 ruled that I'm going to let him testify. I have already ruled 14 that if the plaintiff believes he's gone beyond his expertise 15 that they'll make an objection, and I will rule upon that 16 objection. 17 MS. MASSIE: I don't take it personally on my own 18 behalf, Judge. It's the students I represent who need this 19 information out there. 20 THE COURT: As lawyers we all represent clients. But 21 we all also do two things: Number one is we all have to 22 follow the rules. Number two, we have to abide by those 23 rules. 24 Let me say no more. We'll stand in recess until 25 2:00 p.m. GRUTTER -v- BOLLINGER, ET. AL. 124 1 (Afternoon session) 2 -- --- -- 3 COURT CLERK: Please rise. 4 THE COURT: Thank you. You may be seated. 5 You may proceed. 6 MS. MASSIE: Thank you, Judge. 7 Judge, if you don't mind, I would like to take a 8 step back and lay a little bit more of a foundation than I 9 did before on Mr. White's expertise. 10 THE COURT: Sure, you can, however, I'm going to 11 allow him to testify, as I indicated to you before, but if 12 you want to put more in. 13 MS. MASSIE: If I could, just briefly. 14 THE COURT: Sure. 15 DIRECT EXAMINATION (Continued) 16 BY MS. MASSIE: 17 Q Mr. White, how many experts on testing bias are there 18 in the country today? 19 A I don't have a definition of what a testing bias 20 expert would be, but I said before that when institutions 21 of government look to see how tests can be evaluated from a 22 potential bias, I find out that they end up calling me. 23 I have dealt with other people on these panels and I 24 certainly consider those to be experts, as well, but there 25 is obviously not a company that does test bias, and it's not 125 1 surprising that there are companies that make tests that 2 would not hire test bias experts, although they certainly 3 are interested in it. 4 But it's a question of recognizing that there is 5 a problem from a different perspective of a tester, and 6 actually, frankly, a little from a different perspective 7 from people who call themselves psychometricians, who I 8 have great respect for in terms of mathematical ability. 9 They can do a lot of things that I can't in terms of 10 mathematics. 11 Q So in other words, the Texas, New York and California 12 Legislatures didn't contact the testing companies themselves 13 because the testing companies have an interest in not saying 14 that there is bias, so if the Legislature is interested in 15 bias -- 16 A Well, that's the whole point, that the Legislature was 17 interested, and the Committee on Higher Education held an 18 interim hearing where the first witness was the President 19 of the Law School Admission Council Board of Trustees, 20 Professor Gerald Torres, and the next witness was me. 21 And so they wanted to hear both sides and I was 22 the person that they decided that they should fly out to 23 Austin to hear from. And they heard from other experts, as 24 well. They heard from the co-author of Claude Steele, who 25 is Dr.-- 126 1 Q Aaronson? 2 A Aaronson, Joshua Aaronson, who testified right after 3 I did about the stereotype threat. And so I find that when 4 people ask about this, that they end up asking me. They 5 certainly end up asking other very qualified people, 6 as well. 7 Q And the Florida State Supreme Court, the American 8 Association of Law Schools, all the other entities that we 9 went over before, those are all entities, governments, 10 governmental bodies, et cetera, which invited you? 11 A That's right. 12 Q Which requested that you come explain something to 13 them about bias and testing? 14 A That's right, including the General Counsel's Office 15 of the Department of the Army. 16 Q And so it's fair to say that those people recognized 17 you as one of the foremost national experts on bias in 18 testing and in particular, again, on the LSAT? 19 A I'd like to think so. 20 Q Can you tell us about your testimony for the Florida 21 State Supreme Court, just very briefly? 22 A Well, you mean the Texas Legislature? 23 Q Well, I -- 24 A Oh, I think the Florida State Supreme Court was 25 actually a very interesting thing, where we got to look at 127 1 Bar examination questions. As I mentioned, there were two 2 psychometricians, and there was Dr. Mary Hoover, who had 3 taught at San Francisco State and taught at Stanford, is 4 now teaching at Howard University, and is an expert in 5 linguistics and psychology and education, and a man who was 6 Cuban and who had been trained as a lawyer, but he left just 7 in the nick of time, according to his flight, and moved to 8 New York and became a professor of romance languages, and it 9 was very interesting to see after that long a time not being 10 a lawyer that he still sounded like one when he looked at a 11 bunch of the Bar examination questions. But that's what 12 they did, is they looked at questions that they thought 13 might have some psychometric qualities that were 14 questionable. 15 And they asked the panel of us, including another 16 African American lawyer who had obviously passed the Bar in 17 Florida and was a very sharp character, to see what they 18 thought might be going on in terms of the items themselves. 19 And this is the sort of thing that psychometricians suggest 20 on a regular basis when they evaluate test bias. 21 Q And how did you first get involved in the field of 22 testing bias? 23 A I ended up in an interesting place at the right time. 24 I graduated from law school in 1973 and moved to Berkeley 25 to work on the Childhood and Government Project. I was 128 1 doing research essentially on the Griggs versus Duke Power 2 Company, which I'm sure Your Honor has heard of, which was 3 one of the first Title VII cases, the question of the use of 4 high school diplomas and standardized tests in employment. 5 In 1976, Alan Bakke became a very famous individual 6 in the California State Supreme Court and then subsequently 7 in the United States Supreme Court, and the students at the 8 law school at Boalt Hall, especially the Black Law Students 9 Association, felt that their voice should be heard and they 10 were particularly interested in challenging what was at 11 that point an unacknowledged assumption that they were less 12 qualified to be in law school. 13 And they thought that the basic problem that they 14 felt was not being addressed was the test. And so I ended 15 up helping coauthor a brief to the Supreme Court to talk 16 about the invalidity of the MCAT and had an appendix about 17 the invalidity and bias on the LSAT. 18 As it turned out, the Bakke case came down as it 19 did, and the National Institute of Education and the Spencer 20 Foundation funded a study for the National Conference of 21 Black Lawyers to evaluate the law school admission test and 22 I was asked to be the principal investigator and I conducted 23 that in Berkeley. 24 That's when I learned a lot of information, some of 25 which we're presenting today, and had a number of different 129 1 experts from a variety of different fields talk about the 2 LSAT and talk about law school admissions and talk about the 3 graduates of law school. 4 At that point, truth in testing occurred in 1980 and 5 I was frankly shocked, Your Honor, because we had never seen 6 a real LSAT test question before and we had been told that 7 they weren't biased. 8 And when we wrote our initial report for the 9 National Conference of Black Lawyers, all we could see were 10 the sample questions, and I have actually included some of 11 those sample questions in the report that I gave, and it's a 12 fascinating process, because what I did, and this is what I 13 did at the Association of American Law Schools, is I read 14 one of those questions, and before I was halfway through 15 reading that question, the audience was almost literally 16 rolling in the aisles, because I was reading it to the 17 Section on Minority Education, and as you would expect, the 18 vast majority of the audience were members of minority group 19 law professors. 20 I read that same question in 1988 -- '86, when I 21 was asked again to speak at the Association of American Law 22 Schools, got a completely different reaction, because that 23 audience was the Section on Prelegal Education, and it was a 24 wildly diverse audience, mostly white. 25 And it showed me what I was finding for the next 130 1 fifteen years, is that different people react to different 2 questions in -- the same question in very different ways. 3 And so the research that I can present is statistics, but 4 as I indicated, they are statistics that reflect one-by-one 5 decisions by individual students on individual questions, 6 and what we have discovered, I think quite clearly from the 7 data, is that different people from different racial groups 8 have different test scores largely because of the test. 9 Q Mr. White, let me just interpose there, if I can. 10 So you became aware that there were questions that 11 people answered differently in a way that had to do with 12 race? 13 A Absolutely. 14 Q And that was what started your interest in testing 15 bias, and in particular, bias on the LSAT? 16 A It was fascinating. There were four tests a year and 17 I could have a new one come out and look at it. 18 Q Before that time, I take it, you had only had 19 reference -- had the ability to reference questions which 20 were written as samples and weren't actual questions; is 21 that right? 22 A In the bulletin of information that you get for your 23 application and the codes and everything that would have the 24 number of questions and some examples. 25 Q And since that time, since I guess before Bakke, since 131 1 the mid '70's, you have been working on these questions, you 2 have made presentations, including the material that you'll 3 be presenting today, to institutions like the Texas State 4 Legislature and the Committees of the New York and 5 California State Legislatures before which you have 6 testified; right? 7 A Yes. 8 Q And as far as you know, there really aren't a lot of 9 other people who do this kind of work; is that right? 10 A No one seems to have the interest in it that I do. 11 Q And by the way, you are a member of the National 12 Council on Measurement and Education? 13 A That's correct. I go to their convention every year 14 and I'm a member of the American Educational Research 15 Association, and Division D, which is Measuring and 16 Evaluation. 17 MS. MASSIE: That's all, Judge. I just wanted to 18 give the Court more background on why it is that Mr. White 19 hasn't worked for a testing company and things like that. 20 THE COURT: Go on. 21 BY MS. MASSIE: 22 Q There has been a lot of testimony about the overall 23 gap nationally in the law school applicant pool, the gaps 24 among law school applicants of different races on the LSAT 25 and I'm not going to dwell on that too long. I think it's 132 1 been very conclusively shown to exist in the case. 2 There has also been some information that the gap on 3 the LSAT far outstrips the gap in undergrad GPA's; in other 4 words, there is a gap in undergrad GPA's, but it is nowhere 5 near as large as the gap, aggregate gap by race in LSAT 6 scores. 7 A Yes, there is a much larger gap in both the applicant 8 pool and the accepted student pool at the University of 9 Michigan, in this case, on the LSAT, much larger than any 10 gap on the GPA. What we're showing in this information is 11 that it's true nationwide. 12 Q And we're going to be focusing on the national data? 13 A Yes. 14 Q And this gap and the difference between the two gaps 15 has existed for a long time; is that right? 16 A At least 25 years. 17 MS. MASSIE: Okay. Let's quickly go through a 18 couple of charts that will help map that out. If everybody 19 could turn to Exhibit 218. 20 In case, Judge, I'm getting the numbers wrong, since 21 we didn't straighten that out yet, I'm just going to read 22 what it says at the top, as well. 1976 Number and Percent 23 of Applicants At or Above Selected Levels of LSAT Scores and 24 College Grade Averages. 25 THE COURT: And that's 218, as far as you believe at 133 1 this time? 2 MS. MASSIE: What's that? 3 THE COURT: That's 218? 4 MS. MASSIE: Yes, that's the best information we 5 have. We're going to have to confirm it. 6 THE COURT: That's fine. If it turns out different, 7 you can just make one an A or something. 8 MS. MASSIE: Yes, that's a good idea. 9 THE COURT: The rest of the package you have here, 10 the '96, '97 will be, just so we do it right, 219. 11 MS. MASSIE: Good. 12 THE COURT: 220 will be the next chart. 13 MS. MASSIE: Great. 14 THE COURT: And then just keep going until we're 15 through, right? 16 MS. MASSIE: Sounds good. 17 BY MS. MASSIE: 18 Q This chart was apparently from a brief filed by the 19 Law School Admission Council, an organization that's been 20 mentioned often in this trial, in the Bakke case? 21 A That's correct. The Law School Admission Council 22 sponsors the law school admission tests and filed one of, 23 I think, 62 amicus curiae briefs in the Bakke case. 24 Q Tell us, quickly, what do we have to learn from this 25 chart? 134 1 A This was the clearest indication that I had seen to 2 date of the independent discriminatory impact on the LSAT 3 compared to grades for black students as opposed to white 4 students. 5 Q How is that true? 6 A What I did was look originally at the largest group, 7 which is obviously the whites, and look at how 40 percent 8 of the whites who applied to law school in 1976 applied 9 with a college grade point average of 3.25 or above, and 10 coincidentally, almost exactly the same percentage of 11 37 percent had an LSAT at or above 600. 12 So that for the white applicant pool, the LSAT and 13 the GPA, at 3.25 for GPA and LSAT at 600, were capturing 14 about the same percentage of the white pool, so that we 15 could say in the parlance the testers would use that it was 16 equally difficult, because if you said I would like to get a 17 3.25 in college and go to law school, well, you'll be one of 18 the top 40 percent who apply, and if you say I would like to 19 get a 600 or above on the LSAT and apply to law school, you 20 would have been one of the top 37 percent of the whites who 21 applied. 22 Now, obviously, as Dr. Shapiro mentioned the other 23 day, it would be no point at all if the same people that got 24 the same good grades, got those kind of test scores, because 25 you would be wasting a half day on a Saturday. So different 135 1 people get grades and test scores and what happens is when 2 you require both, which actually, incidentally, law schools 3 do not do, they don't say you better have a 3.25 and a 600, 4 they use what they call a compensatory index, where you can 5 have high LSAT and low grades, or low grades -- low test 6 scores and high grades, so that's not what it's intended 7 to do, but it's the way the Law School Admission Council 8 presented the data. Twenty percent of the whites had both. 9 Q So in other words, 40 percent of the white people got 10 above a 3.25 or above? 11 A Right. 12 Q A different 40 percent, a different but overlapping 13 40 percent got the LSAT score of 600, and the 20 percent 14 captures the overlap? 15 A That's the group that was in both, yes. 16 Q But it didn't work that way for the black applicant. 17 Tell us why not. 18 A That's what was so fascinating for me. This Court and 19 this nation has heard a lot of information about the lower 20 percentage of black applicants who apply to law school with 21 a 3.25 or above, and there is no indication that that is the 22 top limit of the ability of black applicants, but if the 23 LSAT was working the same way for black applicants as it 24 was for whites, this chart wouldn't look the way it does, 25 because instead of 13 percent having a 600 or above, only 136 1 three percent do. And so instead of cutting that group in 2 half, the group is cut from 13 percent down to one percent. 3 Q In other words, when you contemplate the number of the 4 percentage of black applicants who had both the grades and 5 the LSAT scores in the highest range, it was down to one? 6 A Yes. And that's what I call the discriminatory impact 7 of the LSAT with respect to black applicants that doesn't 8 seem to be reflected in college grades. 9 Q Now, are you suggesting that grades are race neutral? 10 A Oh, no. Oh, no, but it's a benchmark for us to 11 measure the relative problem. And so insofar as there is 12 any problem with college grades, there is an additional 13 problem with the LSAT. 14 Q Has this problem continued, this additional problem, 15 for black applicants, black law school applicants? 16 A Well, in 1981 the LSAT changed from the 200 to 800 17 scale that it was based on, the same scale of the SAT and 18 the GRE, the 200 to 800, and they went to a 10 to 48 scale, 19 and I ended up at Testing for the Public. And then in 1991, 20 they changed again and went to a 120 to 180 scale. 21 Q And you're taking us to Exhibit 219, I think? 22 A And so in 1996, 1997, twenty years later, I looked at 23 that, as it's widely available, every law school admission 24 officer gets it, every prelaw advisor gets it, and if you 25 send an e-mail to the Law School Admission Council, you will 137 1 get a very polite response and get the data. 2 What we found is when whites were applying to law 3 school in 1996 and '97, twenty years later, 46 percent of 4 them had a 3.25 and above in college. Almost exactly the 5 same percent, 46 percent, if you round, had an LSAT at or 6 above 155. So it was working the same way that a 600 did. 7 Now, the Law School Admission Council will warn you 8 that a 600 and a 155 are not equateable, and I understand 9 that, but in terms of capturing the pool, the same number 10 of people managed to jump into each pile, and it went down 11 to 27 percent if you made it both. 12 Q And it was working in pretty much the same, slightly 13 moderated, but pretty much the same way, relatively 14 speaking, for black law school applicants as well? 15 A Instead of going down from 13 percent to three percent 16 twenty years ago, you now go down from 17 percent to eight 17 percent. 18 Q And combining the two? 19 A So the gap is not ten percentage points, it's nine 20 percentage points, and instead of one percent of the 21 students having both, now three percent of the students 22 have both. 23 Q So in other words, the independent discriminatory 24 impact of the LSAT has persisted now for at least 25 years? 25 A Yes. 138 1 Q Substantially unchanged? 2 A Yes. We have looked at the data before and after this 3 year and they are almost identical. 4 Q Now, you have done some work to try and determine 5 whether the SES factor on the LSAT is greater or lesser 6 than the race factor on the LSAT for a test taker. 7 A Well, that's one of the first questions that you get 8 when you present data to somebody. People will say, well, 9 the LSAT is biased against lower class students, and we have 10 heard testimony about the SAT being biased against lower 11 class students, and they said that's all you're capturing. 12 It wasn't until Linda Whiteman, Dr. Linda Whiteman, 13 who is the former Director of Research for the Law School 14 Admission Council, had previously worked at the Educational 15 Testing Service and is now a Professor at the University of 16 North Carolina Chapel Hill, published data about the most 17 extensive study about the law school, LSAT, Bar passage. 18 It was called the Bar Passage Study and as part of it they 19 gathered socioeconomic status data. 20 She published a chart in her article in the New York 21 University Law Review, which has been widely cited, that 22 shows the interrelationship between LSAT scores, 23 socioeconomic status and racial identity. 24 Sometimes pictures are worth a thousand words, Your 25 Honor. If you are white, as you can see, regardless of your 139 1 social class, you have an advantage on the LSAT. If you are 2 from the upper middle class blacks, or if you are from the 3 upper class blacks, and you compare your LSAT score with 4 lower middle class whites, your average score is almost 5 six points lower. 6 And so the next page, which I think would be 7 number 221 -- 8 Q It's 221. The bar graph was 220. 9 A Those were the pictures, and these are the numbers 10 that reflect the pictures. 11 Essentially, if you ask the question, does it pay 12 to be upper class if you take the LSAT? You say, yeah. 13 Depending on your racial group, it's worth about two to 14 three points. Now, this is on the 10 to 48 scale and we're 15 now on the 120 to 180 scale, so the numbers are not going to 16 be the same, but the ratios and the relationships shouldn't 17 have changed, because as Dr. Shapiro indicated, tests are 18 designed to be consistent. 19 For example, if you were white, if you came from the 20 lower middle class, your average LSAT would have been 36.24. 21 If you are upper class white, you would have 38.31. So it's 22 a little more than two points to be white if you're upper 23 class. 24 For blacks, it's about three points, 27 to 30, but 25 look at the difference. If you are a lower middle class 140 1 white, your average LSAT is 36. If you are an upper class 2 black, your average LSAT is 30. 3 I think there is one point I want to make on the bar 4 graph, since we put it in, is that they also told the number 5 of students that fall into these different groups. 6 THE COURT: Those are the numbers underneath there? 7 THE WITNESS: Underneath. And as you can see, 8 Dr. Whiteman actually made a point of this in her report, 9 that she tried to pick groups that were about the same size, 10 and I'm sure she also looked at what socioeconomic status 11 data meant, but she found for whites that that was pretty 12 true, about a quarter of the whites fell into each of the 13 groups. 14 If you look at blacks, though, far more than half of 15 the blacks were from the lower middle class. You had 937 in 16 the lower middle class. You had 280, 142, 488 in the other 17 three groups. So that when we talk about socioeconomic bias 18 on the LSAT, there's two points to be made. One is that 19 it's different than race, but it's also part of racial bias. 20 And so while it doesn't make sense to say that the 21 LSAT gap is a reflection of simply socioeconomic status, 22 because it's not, to say that the test is biased on the 23 basis of socioeconomic status is also to say the test is 24 biased on the basis of race. 25 Q There is an additional factor, though, which is race 141 1 itself. 2 A Absolutely. If you look at the relative magnitudes, 3 you have about two to three points -- 4 THE COURT: Which chart, 221? 5 THE WITNESS: The numbers -- 6 BY MS. MASSIE: 7 Q The numbers on 221? 8 A Yes. If you look at the numbers, class accounts for 9 two to three points within a racial group. Race counts up 10 to six points to eight points between blacks and whites. 11 Q Well, six to eight points if you're taking the lowest 12 socioeconomic status category -- 13 A Absolutely. 14 Q -- of white applicants and the highest socioeconomic 15 status category for black applicants. 16 A Yes. The smallest gap you can get is if you compare 17 lower middle class whites with upper class blacks. If 18 you compare upper class to upper class, that's a seven 19 and-a-half point gap, or if you compare lower class to 20 lower class, it's a nine point gap. 21 Q I understand that you have also done some work on -- 22 generated by the question that people have, as they do about 23 socioeconomic status, whether the additional gap produced 24 by the LSAT doesn't reflect different grades at different 25 colleges, less difficult or rigorous college classes, things 142 1 like that. 2 A That's a -- 3 Q So in other words, do the grades mean the same thing. 4 A That's exactly what you will get. When you look at 5 the national data and you look at the percentage of students 6 who apply with a 3.25 or above, people will say, well, they 7 went to different colleges, and so that reflects different 8 standards. 9 There was some very interesting data that we 10 reported as a result of the National Conference, Black 11 Lawyers Conference that's included in our report, Your 12 Honor, that actually shows that traditionally and 13 predominantly black colleges actually offered lower average 14 grades to their students than did the upper class colleges, 15 so that's the general world we're in, but we didn't want to 16 do that, we wanted to control for college, and so what we 17 did is we asked twelve law schools to give us their top 18 four feeder schools. They were the schools that Dr. Allen 19 mentioned yesterday that you're just tired of hearing from 20 students from, and we had those twelve schools give it to 21 us for three years, which meant that we had 36 cohorts of 22 students. 23 Q What year was this, Mr. White? 24 A The study was published in 1981 and the data was from 25 1978 to 1980. These were the applicant groups. We just 143 1 wanted to see what the pool looked like. We didn't want to 2 know whether they got admitted or anything like that, but 3 just what did the pool look like from the top four feeder 4 schools. 5 Then we said, now that we have got the people 6 applying accidentally to these law schools from the same 7 college, we're going to match each individual minority 8 student with all the white students who applied from that 9 same college and accidentally ended up in the same law 10 school application pile with the same GPA's. 11 And we said, what's the same? Well, plus or minus a 12 tenth. So if you're an African American and you have a 3.0, 13 we will look at all the whites who applied to law school 14 from your college that had either a 2.9 up to a 3.1, a very 15 narrow band of similarity. 16 Dr. Joseph Gannon from Boston College presented the 17 results of this study that included over 19,000 students. 18 Q Mr. White, I'm sorry, I just want to make sure the 19 record is absolutely clear, so you took each minority 20 applicant -- 21 A Yes. 22 Q -- and matched each applicant with all the white 23 applicants within a GPA range of plus or minus a tenth of 24 a point? 25 A Yes. 144 1 Q And then you averaged all the white people's GPA's, so 2 you had an average GPA that you were comparing the -- I'm 3 sorry -- you averaged all the white people's LSAT scores, 4 excuse me, so you had an average LSAT score for the white 5 people with the same GPA from the exact same school in the 6 same year as the minority applicant? 7 A That's correct. That's correct. Obviously, each 8 individual white student had their own LSAT score, so we 9 took the match between the black and that white, the black 10 and that white, the black and that white, and we got the 11 average gap that each individual black student faced 12 compared to all the whites that were the same. 13 And then I did have to ask some of my friends 14 that know a little bit more about math than I do, and 15 Dr. Joseph Gannon told me how to do it, that you weighted 16 that average so that you could come out with an overall 17 average gap that say black students from college X were 18 facing in that year. 19 Q And what were the results of that study? 20 A When we looked at the minority students' LSAT scores 21 and compared that to all their comparable whites from the 22 same school, we found that African Americans had 110 points 23 lower LSAT scores on average than the white students who 24 came from the same college with the same grades. 25 Q And this was on a 200 to 800 scale? 145 1 A It was on a 200 to 800 scale. 2 Q And a 100 point gap was -- how substantial was that? 3 A In terms of what psychometricians would call a 4 standard deviation, it's very significant. Tests start from 5 500 and go out, so one standard deviation is 400 to 600. 6 That means you'll capture a third of the applicant pool in 7 that group. And then another 200 points one way or the 8 other would be 300 to 700, and then 200 to 800. That's the 9 way tests are designed, to create that spread. 10 And so on average, you are having more than one 11 standard deviation in terms of what the testers would say in 12 terms of a gap. It's an enormous gap. It's 97 points for 13 Chicano-Latinos, 78 points lower for Native Americans. 14 And this is the point that I think is very 15 interesting to remember. This is data that we were doing in 16 Berkeley, California, even though the study was nationwide 17 in terms of our spread for colleges. We were very 18 interested to notice that there was a 36 point gap for Asian 19 Americans, so that when Asian Americans were taking the LSAT 20 and being compared to whites who applied from the same 21 college to the same law school, they didn't end up with the 22 same LSAT scores. They were behind 36 points on the LSAT. 23 They were relatively close, but they certainly weren't the 24 same. 25 Q So this is the information that's reflected on 146 1 Exhibit 222? 2 A Yes. 3 Q Minority to Non-Minority LSAT Score Differences, and 4 so on? 5 A And that was presented at our national conference by 6 Dr. Joseph Gannon, and then the results of the study were 7 published in the book and it's included in the testimony 8 and exhibits in my report. 9 Q So even when you control for school and for GPA, you 10 still end up with substantial and meaningful LSAT gaps by 11 race in the aggregate? 12 A Yes. Joseph Gannon put it best in his report when 13 he said, what was built up in four years was torn down in 14 four hours. 15 Q Have you -- has Testing for the Public recently 16 undertaken to update this research? 17 A Again, this was on the 200 to 800 scale and they went 18 to 10 to 48, now they have gone to 120 to 180. So it so 19 happened that one of my students, William Kidder, who I was 20 happy he took my LSAT course, then I was happy that he 21 decided he was going to teach the LSAT course for me, I was 22 happy for him that he got into Boalt Hall. I was flattered 23 that he read all my old law review articles, and I was 24 amazed that he took on the burden of actually trying to 25 reproduce Dr. Gannon's study. 147 1 He asked Boalt Hall to give him anonymous data 2 from their applicant pool and he reproduced the study that 3 Dr. Joseph Gannon had done twenty years ago. He did the 4 very same matching process, and this time we had the 5 identities of the school available to us, and you can see, 6 Your Honor, they are very famous schools, it's the top five 7 feeder schools to Boalt Hall, UCLA, Berkeley, Stanford, 8 Harvard and Yale. 9 And as you can see, the University of Michigan, 10 also, has a lot of people applying to Boalt Hall, not as 11 many as the other five, but we included the top fifteen 12 schools, down to the University of Texas of Austin, 13 major colleges, universities, applying to Boalt Hall. 14 What we found was when we control for grades at 15 those elite colleges, 9.2 gap for African Americans, 6.8 gap 16 for Chicano-Latinos, 4.0 gap for Native Americans, and 2.5 17 gap for Asian Americans. The numbers are different in terms 18 of absolute numbers, because you're now down to a 120 to 180 19 scale, but in terms of relative gaps, it's almost identical. 20 Q What is a -- just give us some idea, if you would, of 21 what a 9.2 gap means in terms of the average LSAT scores of 22 accepted applicants to different schools. 23 A We found it very hard to find law schools that were 24 nine points different in their average LSAT scores. We 25 actually put together a little table for -- 148 1 THE COURT: You mean admissions? 2 THE WITNESS: No -- yeah, in other words, if you 3 looked at, say, the University of Michigan Law Schools and 4 their student body and just for accident's sake I decided 5 to compare it to Ohio State -- 6 THE COURT: Right. 7 THE WITNESS: -- the difference between the average 8 LSAT scores of those two law schools is five points. 9 THE COURT: And within the law school itself, it's 10 going to be substantially less? 11 THE WITNESS: Well, that's the other thing that's 12 very interesting, Your Honor, and it's very disturbing, 13 because what the Association of American Law Schools does 14 now is they publish data for the benefit of students, 15 because they try to be as open as they can about their 16 admissions policies without making it seem overly 17 mechanistic, and so what they report is the 25th percentile 18 and the 75th percentile, so they are basically taking the 19 middle half of the class, because they admit some people 20 have very high grades or test scores and they admit some 21 people have lower test scores, and they don't want those to 22 be what people focus in on, but they say, here's what most 23 of the class looks like. 24 Most schools have a five or six point gap between 25 their 50th percentile. 149 1 THE COURT: Within that school? 2 THE WITNESS: Within that school. 3 THE COURT: When you compare the two, it was still 4 within that range? 5 THE WITNESS: Within the range; very difficult to 6 find a nine point gap across the country anyplace, yes. 7 So what it means in terms of the substance of the 8 argument that's made about law schools is that law students 9 are mismatched and somehow they should go to a lesser 10 prestigious law school or lesser selective law school. 11 The fact is, you're either matched or you're not 12 matched. There is not a nine point gap that you can go to 13 another law school and fit in. That is a much bigger gap 14 than you will find in the law school admission process. 15 BY MS. MASSIE: 16 Q Did you get any objections to the update of the study 17 that caused you to further refine it? 18 A Well, the other question that you often get is, yeah, 19 they went to the same college, but did they do the same 20 major. 21 Dr. Whiteman said in her study that when she looked 22 at the national data, the majors for the different racial 23 groups were pretty much the same, that if you're going to 24 apply to law school, it's not surprising that you studied 25 political science or you studied economics or you studied 150 1 history or you studied sociology, and I think there are 2 about eight major categories where you would be going into 3 law school, but it's possible that you're going to the same 4 college and taking different majors. 5 We had the data in the Boalt Hall study to actually 6 analyze that. There's obviously fewer students who are 7 matched at this point, because they may not be in the same 8 major, and so we said, okay, we're only going to include the 9 African Americans where we can find a white student who took 10 the same major and happened to apply to Boalt Hall from that 11 same college. 12 So we would be reducing down any variation that you 13 would get. Well, you were a poli-sci major and I was an 14 economics major, no wonder you got a better GPA, okay, so 15 just take history majors and compare those. Match them 16 again on GPA's, match them in the same major from the same 17 elite university. The gap from African Americans is now 18 down from 9.2 to 9.1. I'm not sure if that tenth of a point 19 is worth celebrating. Chicano-Latino goes up from 6.7 20 to 7.0. Asian Americans goes up from 2.5 to 3.6 points. 21 So what it means is when you actually match majors 22 and then match grades and match test scores, the gap goes up 23 slightly between blacks -- or between Chicano-Latinos and 24 whites and goes up slightly between whites and Asians and 25 stays virtually the same for African Americans. 151 1 So somewhere there is a nine point gap on the LSAT 2 that can't be explained by past educational performance at 3 the most elite colleges in the country. 4 Q Or by socioeconomic status? 5 A Certainly we looked at the other points, as well. 6 Q By the way, you have been referring to Exhibit 224 on 7 the majors; is that right? 8 A Yes. 9 Q And William Kidder, the research associate associated 10 with Testing for the Public who carried out the update at 11 your direction, just so the Court knows, he was an intern 12 at my law firm for a month last summer; isn't that right? 13 A Yes. 14 Q That was after he had undertaken all of this research 15 and so on; correct? 16 A Frankly, Miranda, as much as we have gotten to know 17 you, we had never heard of you at that point. This was 18 1998. 19 MS. MASSIE: I'm shocked. 20 THE WITNESS: This was 1998 when we got the data, 21 Your Honor. 22 MS. MASSIE: Still unthinkable. 23 BY MS. MASSIE: 24 Q So the LSAT adds to discriminatory impact on top of 25 the correlation of race with class? 152 1 A Yes. 2 Q And on top of the selection problems that result, 3 that we have heard a lot about, that result in lower 4 representation of blacks, Latinos, Native American students 5 on selective campuses that result in their getting lower 6 grades on the campuses, all the different forms that 7 discrimination takes, the LSAT adds something on top of 8 that? 9 A That's what is so surprising, is the Court has already 10 heard one example among many that could have been presented 11 of students who actually go to those campuses with very 12 different experiences on those campuses, which affects their 13 grades. 14 And so when you actually match the black student and 15 the white student at the same college with the same grades, 16 you're not matching in a way that is neutral, you're 17 actually taking people who had different experiences on that 18 campus and managed to get the same grades, and so for that 19 test bias to still occur is probably reflecting something 20 actually larger than we can capture with the numbers. 21 Q Have your interactions with students over the years 22 and your study of the testing literature and your analysis 23 of specific questions and all the work that you have done 24 over the last couple of decades on the LSAT and on testing 25 bias in general given you a sense of what some of the 153 1 contributing causes may be? 2 MR. KOLBO: Your Honor, I have to object on 3 foundation at this point, if this is more than a yes or no. 4 MS. MASSIE: I just asked if he had any knowledge 5 of it. 6 THE COURT: Could you repeat the question? I was 7 writing something down and I -- 8 MS. MASSIE: I will rephrase it. 9 THE COURT: Okay. 10 BY MS. MASSIE: 11 Q Do you have any opinions, do you have any basis 12 for knowing what some of the contributing causes might be, 13 gleaning from your work with students and your review of the 14 literature and your analysis of test questions over the 15 years? 16 THE COURT: The causes for? 17 MS. MASSIE: For the additional gap beyond what we 18 have been talking about. 19 MR. KOLBO: Your Honor, unless, again, it's a yes or 20 no, I object to the foundation. 21 THE COURT: Answer yes or no, first. 22 THE WITNESS: Yes. 23 THE COURT: Now lay your foundation. 24 BY MS. MASSIE: 25 Q Tell us what your basis is. 154 1 A When you go to National Council on Measurement 2 Education conventions for about twenty years and read as 3 much literature as you can, eventually you start to realize 4 what is essentially given wisdom in the testing community, 5 and the ways in which standardized tests are made by an 6 individual company or an individual test provider are 7 private information. They are sometimes made public, 8 certainly there are technical manuals that are available, 9 but the general approach of making a test is what's called 10 classical test theory, and that is part of the making of 11 the test. 12 There are other parts of the making of the test that 13 are actually made very public, and those include the types 14 of questions that are selected, the types of passages that 15 are included on the test, and then besides how tests are 16 made, it's how tests are taken. 17 Q What do you mean? 18 A That when Dr. Steele talks about stereotype threat, he 19 is not talking about anything unique to the test, he is not 20 saying that there was a stereotype on the test and people 21 felt threatened about it, he is saying that the test itself 22 evokes a stereotype. 23 So he chose his GRE items, which was the first 24 study, the one I read in most detail, because they were GRE 25 items, and he thought they were tests of verbal ability, and 155 1 he thought that he could do some psychological studies on 2 them. And he came up with a theory that he can identify 3 stereotype threat, but that's something that the test 4 company has no control over. They can't choose a different 5 reading passage. They can't choose a different logic or 6 reasoning question. They can't even choose different test 7 specifications and have any impact on what might be going on 8 when different students from different racial groups take 9 the same test. 10 And so it's the combination of how the test is made 11 and how the test is taken that results in what seems to be 12 about a nine point gap among people who have performed 13 equally in college. 14 THE COURT: But you don't know what percentage each 15 may play a part? 16 THE WITNESS: No, no. There is no way of telling 17 that. 18 I think that Dr. Steele has given some information 19 that gives you dimensions, and so what he was able to 20 manipulate in a laboratory setting, which is a much less 21 threatening setting than a law school admission test, 22 gives us at least, I would imagine, a lower bound of what 23 stereotype threat could be on the LSAT. And as I say, in 24 the real thing, that's different than sitting around in -- 25 THE COURT: The real world? 156 1 THE WITNESS: -- in a lab in the Stanford Psychology 2 Building. 3 THE COURT: Nobody obviously studied that, because 4 how do you study that? 5 THE WITNESS: That's the point. So we have the nine 6 points. And it's clear at least in theory, and I think 7 that there is documentation for at least a lot of the 8 information, that both the test development process that you 9 heard about from Dr. Shapiro and the test taking process 10 which you have heard from Dean Garcia affect the way in 11 which this nine point gap results. 12 THE COURT: When you teach the course, you talk a 13 little bit, I'm sure, about test taking? 14 THE WITNESS: Oh, yes, that's what we teach. 15 THE COURT: I shouldn't say a little, that's -- 16 THE WITNESS: That's what we teach. 17 THE COURT: And in your experience, I was going to 18 ask you this later, but in your experience, in the courses 19 that you give, what's the percentage of increase after 20 taking your course, if you know? I mean, I would suspect 21 you keep some statistics. 22 THE WITNESS: Your Honor, what I do is I give a real 23 LSAT that's obviously not official on the first day of class 24 and then I compare it with what students tell me their LSATs 25 were in the past, and at one point I averaged it out and it 157 1 was five points. 2 THE COURT: That's how they do it? I thought maybe 3 that they have students take one test and then take it 4 again. No? 5 THE WITNESS: Well, that's the trouble with the high 6 stakes test, that you don't want to use people as guinea 7 pigs, and since the Law School Admission Council will report 8 your average LSAT score - 9 THE COURT: You can't do that? 10 THE WITNESS: -- you don't want to do that to a real 11 human being, you know, but that's why we give an official 12 test, and obviously it doesn't have the same impact in the 13 same setting as a real test does, but it gives us some 14 indication of whether or not we have changed some people's 15 scores. 16 MS. MASSIE: I don't know if you have any other 17 questions, Judge. I'm just about to finish up. 18 THE COURT: No, he has answered my questions. 19 BY MS. MASSIE: 20 Q In your opinion, Mr. White, is there a level playing 21 field for minority law school applicants on the LSAT? 22 A Certainly not. 23 Q And if you didn't use affirmative action in law school 24 admissions, would using the LSAT be an unfair double 25 standard that discriminated against minority applicants? 158 1 A It would be an unfair double standard. One way of 2 seeing the dimensions of that double standard is to look at 3 the index that the University of Michigan used. Having a 4 nine point gap on the LSAT is like taking nine tenths -- 5 THE COURT: I'm sorry? 6 A -- taking nine tenths of a point away from your score. 7 THE COURT: The index? 8 THE WITNESS: That's right. In other words, they 9 give you a constant and then multiply a number by the LSAT 10 score and multiply a number by the GPA and add it up. It's 11 in the report that Dr. Larntz provided. 12 THE COURT: They say they don't really use it. They 13 say the computer -- 14 THE WITNESS: Oh, it's the index. They do it 15 mathematically or automatically once they get the index, 16 but if you want to see what the impact of a nine point 17 gap is -- 18 THE COURT: I see. It's obviously there. 19 THE WITNESS: It's like looking at the GPA of a 20 black student and saying, well, we'll just knock this down 21 nine tenths of a point. So if you've got a 3.5, we will 22 call it a 2.6. How's that? That's what the nine point 23 gap does for the index. 24 BY MS. MASSIE: 25 Q And in your view, Mr. White, to what extent should 159 1 race be taken into account in law school admissions? 2 A I think it's essential, and the thing that I was so 3 surprised about in terms of the response was not that the 4 Law School Admission Council had no explanation for the gap 5 when I presented it at the Society of American Law Teachers 6 on a panel with the President of the Law School Admission 7 Council just a month ago, this is not news, and it's not 8 news to admissions officers either. 9 I think that over the last 20 to 25 years admissions 10 officers have recognized that there is this independent gap, 11 and it's part of the folklore of admissions, so that people 12 recognize that this gap is part of the evaluation process. 13 And so just as you would like to know what school somebody 14 went to and what major they had, you would like to know what 15 race they are when 16 you evaluate their LSAT score. 17 Q Let me go back and ask the last question, because I'm 18 interested in your opinion on this as an expert on the test 19 rather than on the admissions process. 20 A Yes. 21 Q In your view, to what extent should race be taken into 22 account in admissions? 23 A It should clearly be taken into account in evaluating 24 the LSAT scores of the applicant. It is -- an aspect of 25 evaluating the information is knowing the LSAT score, and 160 1 knowing the race of the people who took the LSAT score is 2 part and parcel of evaluating that part of the applicant's 3 file. 4 Q And if you chose as an institution to take race into 5 account in a separate way, how much does it need to be taken 6 into account? 7 A You don't want to make it mechanical, but unless there 8 is a gap of more than nine points, I would say that the law 9 school -- between the student's LSAT scores -- I would say 10 that the law schools hadn't taken enough account of it. I 11 think that even though people recognize there is a gap, 12 people were surprised at how large it was. They were 13 surprised about nine points. I think schools underevaluate, 14 but I certainly think that they have to evaluate it with 15 that knowledge in mind. 16 Q We need more affirmative action in law schools 17 admissions, not less? 18 A Oh, certainly. The idea that people have been getting 19 a benefit on the LSAT is clearly not the case. 20 MS. MASSIE: Thank you. 21 MR. NIEHOFF: Your Honor, if I could have just sixty 22 seconds. 23 (Brief pause.) 24 MR. NIEHOFF: We don't have any questions. 25 THE COURT: Plaintiffs, any questions? 161 1 MR. KOLBO: Your Honor, we have no questions. 2 THE COURT: Mr. White, thank you very much. 3 MS. MASSIE: I'm sorry, could I reoffer Mr. White 4 as an exhibit -- as an expert, sorry -- and also offer the 5 exhibits into evidence? 6 THE COURT: Any objection to the exhibits? 7 MR. KOLBO: We have an objection only to the report, 8 Your Honor, for the reasons that I objected to his 9 qualifications. 10 THE COURT: What report? 11 MS. MASSIE: And I didn't offer the report in, and I 12 sure should have, so I thank Mr. Kolbo. 13 MR. KOLBO: You're welcome. I thought that's what 14 you were referring to. 15 MS. MASSIE: I was just referring to the exhibits. 16 His report is -- 17 THE COURT: Let's take them one at a time. 18 You have no objection to the exhibits referred to? 19 MR. KOLBO: I have none, Your Honor. 20 THE COURT: Those will be received. 21 MS. MASSIE: I apologize. I had it written down on 22 a post-it, which has disappeared, but I'll find it and make 23 it clear for the record. I would like to offer the report. 24 THE COURT: And the objection to the report is? 25 MR. KOLBO: Based on the foundation and fact that 162 1 the expert is not qualified in all these areas, Your Honor, 2 and the report addresses areas that he's not qualified in. 3 THE COURT: I will admit the report with the 4 understanding I'm only going to include those areas for 5 which he has expertise, and I think that's what he has 6 testified to, what he has expertise in. He has already 7 testified what areas he doesn't have expertise in, so if 8 the report includes those areas, then I will -- we won't 9 consider it. 10 MS. MASSIE: Excellent. Thank you, Judge. 11 It's 173, by the way. 12 THE COURT: Very well. Thank you, Mr. White. 13 THE WITNESS: Thank you, Your Honor. 14 THE COURT: Next witness, please. 15 MS. MASSIE: We actually had thought that, and I 16 meant to mention this before starting up again, we thought 17 that we were going to go through until 4:00. 18 THE COURT: Yes, we are. 19 MS. MASSIE: And so we thought that there was going 20 to be some cross examination, so we asked Professor Woo to 21 come back next week. 22 THE COURT: That's fine. I have no problems with 23 that. 24 MS. MASSIE: Sorry, I should have mentioned that 25 before. 163 1 THE COURT: No, no problem at all, none whatsoever. 2 I just -- that's fine. 3 Tomorrow morning, then, I guess is our next thing. 4 Let me just make sure everybody is on the same page in terms 5 of scheduling. Tomorrow morning we're going to put the 6 rebuttal testimony on, and Monday we're going all day. I'm 7 just looking at the schedule for Monday. We have the class 8 in, we talked about that, and at 11:00 I have just two, real 9 quick matters, nothing else in the afternoon, and I think we 10 talked about going into the evening on Monday, did we not? 11 MS. MASSIE: That would be fine. That would be 12 great. 13 THE COURT: It's up to you. I won't make plans 14 Monday. If you want to go, we can. 15 And then Tuesday we will start at 2:00, and I think 16 since we're starting at 2:00, if you want, I don't mind 17 going into the evening on Tuesday so we can get at least 18 a -- you know, I mean, I know it's hard for you guys, 19 probably harder for you than for me, so, you know, when I 20 say the evening, 6:00, 6:30, it's up to you. I don't want 21 to -- 22 And then Wednesday, as I said, unfortunately, I 23 can't be here. 24 And then by Thursday, I expect -- I'm just looking 25 at Thursday's docket. We have a pretty free docket. I have 164 1 nothing else, except Thursday we'll probably take lunch a 2 little closer to 12:30, and other than that, there is just 3 one other miscellaneous matter. 4 And I think Thursday, probably, unless there is a 5 witness that we haven't talked about, you can get everything 6 done without coming Friday, that would be great, but I'll be 7 here Friday, and Friday I have nothing else but this on the 8 docket. 9 MR. KOLBO: And just to make sure, as I mentioned 10 this morning, we're still working with, and I'll be talking 11 to Counsel about this over the weekend if necessary, we're 12 trying to figure out whether Professor Heriot should testify 13 Monday or later in the week, subject to her availability and 14 so forth. 15 THE COURT: And I will -- I haven't had a chance to 16 read it sitting right here. By tomorrow morning I will have 17 the answer for you, and I'll take it home and read it 18 tonight and listen to a little bit of argument and be 19 prepared to rule. 20 Let me just tell you one other thing that -- it's 21 not my intention, because I know Ms. Massie mentioned 22 before, really to do closing arguments. I'm not sure that 23 they are going to be fruitful here. 24 What is going to be very fruitful to me, at least, 25 is some kind of written summary, and what I have in mind, 165 1 and I would be more than glad to offer it at this time, and 2 when we can -- we don't have to necessarily discuss it 3 today, what I'm looking for is a concise summary of your -- 4 what you believe to be your best arguments and positions and 5 with specific reference to evidence or witnesses, and if 6 you can, to -- you don't have to quote things and that, 7 fortunately, you have ordered daily transcripts, so we all 8 have transcripts, don't waste your time, you know, and cite 9 it to a specific page or something of that nature, rather 10 than, you know, if there is something specific that you want 11 to cite it to, try to help us identify the most persuasive 12 evidence, you know, kind of like a closing argument, but 13 give me a little more opportunity to maybe direct our 14 attention to specific things and focus on the most 15 important things that you believe are important. 16 I have in mind maybe twenty pages or so, 17 simultaneous submission, and the same thing you would do 18 in a closing argument, but in a written thing, so that we 19 can digest it, and it's just more meaningful when you have 20 a little time to put it together and to focus. 21 And I would like it as soon as the trial is over, if 22 possible. These kind of trials, I have to, and I have set 23 aside a lot of time to start putting together things, and I 24 like to do it as quickly as possible because it's still 25 fresh, and credibility issues and all of those things, you 166 1 know. The longer it goes on, the harder it is to even -- 2 when you read the transcript, to -- you know, that's why I 3 have paid a lot of attention to watching witnesses and doing 4 the kinds of things that I, as a Judge, do in making a 5 decision. 6 MR. PAYTON: I think that makes a lot of sense. Let 7 me just report on some of the conversations that the three 8 of us have had about this, which may be changing. 9 I think we all agree with that, and we were talking 10 sort of informally among ourselves about, once the evidence 11 is fully received, we should be in a position to have 12 something to the Court, we said, in two weeks. Is that too 13 long? 14 THE COURT: That's fine. I mean, I was hoping ten 15 days, but -- 16 MR. PAYTON: We can do that. 17 THE COURT: I'm not going to quibble over ten days, 18 just because, as I said, my OCD. I set aside time to work 19 on this opinion and to read some of the transcripts and 20 so forth. 21 MR. PAYTON: Let me just keep going. I'm coming 22 right back to this. 23 The second thing we have talked about is whether or 24 not, in addition to that, it wouldn't be of help to the 25 Court, all three of us, wouldn't it be of help to the Court 167 1 to have brief closing arguments immediately after the close 2 of the evidence and do this as well, so if we finished on 3 Thursday, have closing argument on Friday. 4 THE COURT: I'm not opposed to it. I'm just not -- 5 absolutely, just again, part of my whole thing is not to 6 do things that are useless or -- and certainly it's not 7 useless. When you hear something and then you sit down and 8 read it, it sticks in a little bit further, so I have no 9 problems. Look, you know, but by the same token, I don't 10 want to -- if we can't finish it by Friday with closing 11 arguments, I would probably say economically and so forth 12 it doesn't make sense, but if we can, with some limitations, 13 maybe 45 minutes each, because after that, you know -- 14 MR. PAYTON: I think that's actually helpful to us 15 to have some limitation. 16 THE COURT: Yes, no more than 45, with the 17 understanding that we're going to get something in writing, 18 that will at least get our thinking going and we will be 19 able to take it and see what's going on. 20 MR. PAYTON: That's what I think. 21 THE COURT: I have no problems; and then a twenty 22 page submission after that just kind of focusing in 23 specifically on those things that, you know, that you 24 think that I should really take a look at. 25 MR. KOLBO: Do you anticipate us submitting post 168 1 findings of fact? 2 THE COURT: No, whatever you want to do in that 3 twenty pages, whatever you think is important. First I 4 thought, you know, that findings of facts and conclusions 5 and all, whatever you think is important in that twenty 6 pages, direct your attention to it. 7 I'm not going to tell you how to do it or what to do 8 or anything of that nature, just realize you don't have to 9 waste your time in terms of quoting things, we will all have 10 transcripts, and say, this is really important, read page 11 so-and-so, volume so-and-so, give us the page and the volume 12 and line and all that, we will be able to take a look at it 13 and whatever. 14 Kind of like you guys had the war rooms, now we're 15 going to set up a war room where we will all have all the 16 volumes. 17 I've tried many, many cases, so I know it just 18 shifts over to this side, so. 19 MS. MASSIE: Judge, that sounds very good, with this 20 exception, I think we're going to need a bit more than ten 21 days or two weeks. Can I talk you into three weeks? 22 THE COURT: You can talk me into it, but I'll be 23 very honest with you, that I've got to start putting it 24 together in ten days, you know. I'm not -- I will wait and 25 I will read it, and I'll pay attention to everything you 169 1 have to say, but as I say, I'm going to -- I have been a 2 judge long enough to know that the more time that goes on 3 after the trial, the less opportunity you have to really 4 capture it, and I'm just being real upfront with you and 5 real honest with you, you know, the essence of the trial and 6 the kind of things that are important. 7 So I'm not -- you know, ten days would really be 8 preferable, because I just -- as I said, may be my OCD 9 or something, but I'm one of these that I'm going to be 10 focused. I have set my schedule aside for this time, and I 11 have my schedule aside for as long as it takes to put this 12 together, because I think it's important and that's why I 13 kind of pushed you into a trial date and I pushed you into 14 this. And I told you, plus, we're getting letters and phone 15 calls and, you know, as I told you before, that we're 16 stacking up, and I think there is a lot of interest in it. 17 But most importantly, I guess it's the way I work, 18 and that is that I feel much more comfortable doing it 19 as quickly as possible in terms of at least starting the 20 process going, because I do remember what's going on, and 21 part of this whole case is the witnesses and what they have 22 to say and how they had to say it and all those things, and 23 the more time goes on, just the more it goes. So see what 24 you can do in ten days and we will go from there. 25 Tomorrow morning, then, we said the courtroom will 170 1 be open at 8:00 and the Marshals will be here to open the 2 building and everything at 8:00 and we will start about 3 8:30. Everybody agrees, or is that good or do you want 4 to readjust that? 5 MS. MASSIE: Judge, we just filed some papers 6 regarding Gail Heriot. 7 THE COURT: Oh, good. If you have them -- who did 8 you file them with? 9 MS. MASSIE: I think they have been filed on the 10 fifth floor. 11 I'm sorry, I thought they had been filed and served, 12 but they have not. 13 THE COURT: If you just give me a copy, you can file 14 them upstairs after, and if you want, I'll sign right on 15 there so you don't have to bring another copy down, and that 16 I have already gotten my copy. 17 MS. MASSIE: Great. 18 MR. KOLBO: And Judge -- 19 THE COURT: If Caroline was here, she is sick today, 20 I would file them, but we have nobody to file them. 21 MS. MASSIE: Oh, no, that's fine. 22 MR. KOLBO: Your Honor, we want to get the matter 23 argued tomorrow, but we probably won't have a brief 24 submitted in response to this. 25 THE COURT: You don't have to. I'll listen to oral 171 1 arguments tomorrow. We have plenty of time tomorrow and 2 I'll be more than happy to. 3 Okay, anything else? 4 See you tomorrow morning. Thank you. 5 COURT CLERK: All rise. 6 (Proceedings adjourned at 3:15 p.m.) 7 -- --- -- 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 172 1 CERTIFICATE 2 3 I, JOAN L. MORGAN, Official Court Reporter 4 for the United States District Court for the Eastern 5 District of Michigan, appointed pursuant to the provisions 6 of Title 28, United States Code, Section 753, do hereby 7 certify that the foregoing proceedings were had in the 8 within entitled and number cause of the date hereinbefore 9 set forth; and I do further certify that the foregoing 10 transcript has been prepared by me or under my direction. 11 12 _________________________ JOAN L. MORGAN, CSR 13 Official Court Reporter Detroit, Michigan 48226 14 Date:_________________________ 15 16 17 18 19 20 21 22 23 24 25