• September 3, 2015

Law Schools Resist Proposal to Assess Them Based on What Students Learn

Law schools would be required to identify key skills and competencies and develop ways to test how well their graduates are learning them under controversial revisions to accreditation standards being proposed by the American Bar Association.

The proposed revisions, which are being drafted by a committee of the association's Section of Legal Education and Admissions to the Bar, were a topic of heated debate here throughout the annual meeting of the Association of American Law Schools, which ended Sunday.

Instead of judging law schools primarily on "input" measures, such as faculty size and library holdings, the proposed revisions would look more at "outcome" measures, such as what students are actually learning.

Several law deans said they have enough to worry about with budget cuts, a tough job market for their graduates, and the soaring cost of legal education without adding a potentially expensive assessment overhaul.

"Given the fragility of the legal profession and the fragility of law-school finances, it's a tough time to think about taking on a major process of assessment," Lauren K. Robel, dean of Indiana University's Maurer School of Law, said during an interview here.

But supporters of the proposed revisions said law schools can no longer afford to churn out graduates who lack the skills employers want, especially when jobs are so hard to come by.

The blog Law Shucks reported last week that 12,196 people were laid off at 138 large law firms in 2009, the worst year ever for such layoffs. The job cuts included 4,633 lawyers and 7,563 staff workers.

Timing Questioned

"It's still unclear how all of the upheaval in the law-firm market will affect legal education," said Maureen A. O'Rourke, dean of the Boston University School of Law. While she supports a discussion of assessment based on student-learning outcomes, she believes it would be a mistake to make changes at a time when the legal landscape is in such flux, or to require law schools to enact costly reforms now. Law schools would probably have to hire additional staff members to collect data and develop testing metrics, and some curriculum changes could be expensive.

The proposed standards, which are still being developed, call on law schools to define learning outcomes that are consistent with their missions and to offer curricula that will achieve those outcomes. Different versions being considered offer varying degrees of specificity about what those skills should include.

Law schools would also be required to come up with a system to assess how well students are meeting those goals.

Some law deans question whether the kinds of skills that make a good lawyer can be measured through traditional assessment techniques.

"It is worth pausing to ask how the proponents of outcome measures can be so very confident that the actual performance of tasks deemed essential for the practice of law can be identified, measured, and evaluated," said Robert C. Post, dean of Yale Law School.

Despite the committee's assurances that law schools would be free to decide for themselves how to identify and assess desirable outcomes, Mr. Post said he remains concerned that the standards "will tend toward the imposition of uniformity and rigidity" in law-school curricula.

"At a minimum, the standards will require each law school to articulate a mission that can be assessed according to the science of measurable assessment," he said.

Calming Fears

Some deans worried that diversity efforts could suffer if law schools use measures like bar-examination passage rates to assess how much their graduates are learning. Members of underrepresented minority groups tend to have lower first-time bar-passage rates than do white graduates.

ABA administrators sought to assuage such concerns during a packed meeting on Friday with law-school officials here.

They said they will continue to work closely with law deans in drafting standards that will accommodate different law-school missions and avoid significant costs.

Among those who support the effort is James G. Leipold, executive director of NALP: the Association for Legal Career Professionals. He said hiring partners at law firms regularly complain to him that they have to spend too much time training new associates in skills they haven't learned in law school. But he added that there is a natural tension between legal educators who favor a more theoretical-based legal education and those who are increasingly pushing for a more skills-based curriculum.

NALP, which was previously known as the National Association for Law Placement, held a round-table discussion last month

in which Phillip A. Bradley, senior vice president and general counsel for Duane Reade, a large drugstore chain, likened law schools to car companies that are "manufacturing something that nobody wants."

Mr. Bradley said many law firms are developing core competencies they expect of their lawyers, but many law schools aren't delivering graduates who come close to meeting them.

"Some law schools are of the view that delivering law graduates who have been trained 'to think like a lawyer' is sufficient. It is then the law firm's responsibility to train the law graduate to 'be a lawyer,'" Mr. Bradley said in an e-mail message to The Chronicle.

In this market, clients are less willing to pay law firms the high prices they charge, in part to cover the firms' staff training costs, he wrote. He said appropriately designed outcomes testing could help firms identify graduates who are ready to hit the ground running.


1. rkrosenberg - January 11, 2010 at 08:46 am

This seems to me the perfect time to be having the conversation about what is taught and what is learned in law school. Times are changing. I went to law school 30 years ago and never felt that I was prepared to practice. However, many graduates of law school don't practice law and so there needs to be a conversation not just about what law firms need, but what makes a good lawyer. I agree that traditional assessment tools may not be good measures, BUT that doesn't mean that we shouldn't be thinking about what lawyers should be learning and what skills they need to be acquiring.

2. timlincoln - January 11, 2010 at 08:52 am

In defense of law school deans: yes, these proposals might press against variation in the mission and ethos of law schools in favor of irrational conformity (institutional isomorphism); yes, any system of assessment costs money.

In defense of proposed standards: do not law schools tell stake holders that they prepare individuals to engage in the practice of law? If so, then legal education should serve its intended end and law schools need information about how well they deliver on their promises.

3. mufc1tb - January 11, 2010 at 09:34 am

Have a look at the Law Benchmark statement in England. Active learning outcomes are the best way to see what a student "knows, understands and is able to do".

4. esselan - January 11, 2010 at 10:26 am

Assessment never has to be costly. It's just a matter of coherently recording what is going on in classrooms and using the results to grade not only individual students, but put together a larger picture.

5. dqualters - January 11, 2010 at 10:47 am

This argument about time, cost, and resources has historically been the way to side step the assessment issue. Unfortunately as other fields have found (business, medicine, engineering), it doesn't hold up for long. And as "esselan" said, doing it doesn't take sophisticated measures or bar pass rates. It's more about measuring classroom learning as students progress through legal education. Read Barbara Walvoord's work - she makes assessment easy, useful, and beneficial to faculty members. It's assessment for learning(which most professors in my experience are really interested in), not assessment to check off an accreditation box.

This will be a difficult transition but a necessary one for the 21st century. Added bonus, done right assessment actually makes learning easier for everyone!

6. 11272784 - January 11, 2010 at 11:40 am

This is the same conversation that is coming for ALL academic disciplines. It's time to start testing in class and assessing programs based on what has been learned, not what has been taught.

7. petermshane - January 11, 2010 at 04:42 pm

If this is to be taken seriously, then the bar should support law schools' creation of "teaching law firms," like "teaching hospitals," where students engage in the supervised practice of law with intensity resembling the work of medical residents. Currently, the training that law schools can provide in actual lawyering is typically limited to law school clinics that are confined to forms of legal practice that do not compete significantly with private firms for fee-paying clients. If the bar is willing to allow law schools to be competing centers in the provision of legal services, it would revolutionize legal education.

8. newyorkyankees - January 11, 2010 at 05:20 pm

How about law students taking required courses in the first year, a combination of requirements and electives in the 2nd year, and in the 3rd year undertake practical training in a law firm similar to what medical residents do?

9. edpm8504 - January 11, 2010 at 08:13 pm

How interesting that some of the law school opposition to such proposals seems to boil down to the precarious state of law school finances (and of all the un-or underemployed, debt-burdened graduates these schools keep producing). Interesting, too, that the ABA now seems to be seeing these proposals as introducing some reality to legal education, while it continues to ignore the underlying economic fundamentals: that the legal education industry has been producing way too many lawyers (however they may be assessed) for years, and the ABA has been complicit in ignoring the impact of this expensive and wasteful overproduction on the legal profession.

10. bbolson - January 11, 2010 at 08:15 pm

Why, pray tell, should law schools be exempt from accountability? Hasn't that caused the nation enough problems already??????

11. samanthak - January 12, 2010 at 12:03 pm

"Several law deans said they have enough to worry about" without having to worry if the students are actually learning anything?!?

12. peggybloom - January 12, 2010 at 12:03 pm

I am amused at all the protests. Our private Jesuit research university has a comprehensive student learning assessment system which involves all schools defining student learning outcomes and conducting ongoing assessment. Our law school has a well developed annual assessment cycle and the results have led to specific improvements, for example in first year student writing.

13. jillikg - January 12, 2010 at 12:04 pm

I agree with 7 and 8. In order for law schools to successfully produce graduates who can not only think like a lawyer but actually act like one as well, then they need to provide their students the oppurtunity to gain experience practicing law. Also, why is it that law school costs are continuing to rise? They are already astronomical and they don't have the need for a large budget for materials used in labs.

14. greenhills73 - January 12, 2010 at 12:53 pm

"Some deans worried that diversity efforts could suffer if law schools use measures like bar-examination passage rates to assess how much their graduates are learning. Members of underrepresented minority groups tend to have lower first-time bar-passage rates than do white graduates." Oh, my gosh, where to begin...

15. 11180037 - January 12, 2010 at 01:46 pm

How loudly does this statement speak to the diminishment of integrity in higher education: "Some deans worried that diversity efforts could suffer if law schools use measures like bar-examination passage rates to assess how much their graduates are learning. Members of underrepresented minority groups tend to have lower first-time bar-passage rates than do white graduates." What about the "suffering" of graduates who were lead to believe they were actually educated in something the job market sought???

16. haohtt - January 12, 2010 at 04:28 pm

"It is worth pausing to ask how the proponents of outcome measures can be so very confident that the actual performance of tasks deemed essential for the practice of law can be identified, measured, and evaluated".

Is the good Dean serious? If he and the Yale School of Law do not have the ability to identify, measure and evaluate the "tasks deemed essential for the practice of law," then how can they be sure that they know how to teach these unidentified, unmeasured and unevaluated tasks? Perhaps it is time for them to pack it in.

17. savoyard1 - January 15, 2010 at 03:30 pm

There is, I think, a middle ground between those practitioners who want young lawyers ready to go and those academics who think it largely the job of law firms to provide nuts-and-bolts training. The academics do have a point, though the ones quoted largely didn't make it. Law schools have a competitive advantage in classroom training in legal doctrine and theory, and to some extent in small-group simulations. They do not have a huge advantage in learning by doing, save in areas like legal research or the basics of legal writing. It's not that law schools can't provide that training; as was noted, they do, mainly through clinics. As it stands, most law schools provide a clinical opportunity to every student who wants one. But obviously that isn't enough, or the ABA would not be restive. (And if one looks at the history of legal education, current law schools do much more practical training than did their predecessors.)

On the other hand, the economics of law firms is changing. If a firm is to maintain its profitability, the senior lawyers can't spend as much time as they have in the training of junior associates, both because of the diminution in their own billable hours and the lack of billable work from the associate. At the larger firms, this to some degree reflects the huge increase in partner earnings over the last couple of decades, combined with a drop in client demand and a closer look at the bills. For partners to maintain their incomes, they have to squeeze every last billable hour from their staff, and lawyer training is an obvious place to start.

One possibility is, as was suggested, turning the third year of law school into a sort of giant clinic, with the students supervised by a large number of clinical faculty who would provide frequent feedback and teach courses in such topics as contract drafting, discovery, depositions, and so forth. I can imagine adding capstone courses as well that would build on the first two years of law school. Presumably the current curriculum would have to be compressed as well.

But there are some formidable costs associated with this sort of move. For one thing, clinical education is, as others have noted, very expensive if one takes only or primarily pro bono cases. For another, the practicing bar also demands junior lawyers who know more about the substance of their practice areas. It's not clear how that could be done within a compressed curriculum.

There are some possible answers, but at various times they have been proposed and have met with the opposition of the bar. (1) Clinics could take mainly fee-paying clients, much as dental schools provide low-price treatment to students supervised by clinical faculty. Especially in small markets, though, the bar has generally opposed this sort of thing bitterly on the grounds that they don't need the cut-rate competition. (2) Law schools could add a fourth year, or perhaps a seventh semester, devoted entirely to clinical training. For obvious reasons, that is likely to be DOA. (3) Law schools could simply raise tuition to pay for additional clinicians, facilities, and so forth. Another big winner. (4) Law schools could find the money internally by slashing faculty size and increasing teaching load. That would have some interesting effects on the demand for faculty positions and the quality of legal education.

Well, if there were any easy solutions, some start-up law school would have put them in place and blown away its market. As it happens, there is at least one law school that has made moves in this direction. Northeastern has a program in which students alternate semesters in the classroom with semesters doing supervised work in the field. I don't know whether their grads are particularly sought-after or competent; any Boston lawyers reading this story (yeah, sure) may be able to comment.

More realistically, law schools could, without changing their staffing greatly, do more to teach basic legal skills while still providing a sound doctrinal and theoretical basis for the employment of those skills. For example, courses could be accompanied by practicums (practica?) in which students would do some hands-on work in the pertinent subject. Clinics could be required. More schools could teach legal drafting. Upper-level courses could integrate more lawyering skills with the coursework. The curriculum could be structured better, with more capstone courses in the third year building on second-year prerequisites.

I can't imagine these changes would produce nothing but lawyers ready to try a case right out of law school. Based on my experience (and yes, I'm a legal academic with a fair bit of time in practice), you become a good lawyer over time, through doing different types of projects and seeing different types of clients and opponents. Still, students could come out of law school with a good basic knowledge of standard practice skills, which they could then build on over time.

And then there's the relative emphasis on theory over doctrine in legal scholarship . . . but this comment has already become book-length, so I'd better get back to grading. (I might add that the exam calls on students to give practical advice to their clients based on their doctrinal analysis and general sense of what's realistic, following on a course that presented a good deal of often theoretical material in a very practical setting. The students have generally been showing some real skill in applying what they've learned to real-life problems. I trust the ABA will be pleased.)

18. trterryjr - January 29, 2010 at 01:21 pm

This problem has three basic parts:

1. Few tenured law faculty have practiced law for more than 2 to 5 years. Thus, there are competency issues with them training people how to practice law.

2. Private clients will no longer pay law firms for the legal work of "juniors in training". After 38 years I can do something in 30 minutes that would take a young lawyer 4 or 5 hours. I may have some professional obligation to train juniors but the clients paying the bills don't. Some larger firms have training programs but economics dictates having new lawyers "do what needs doing to bring in dollars". Whether they learn anything by doing it is not a factor.

3. There has not been a serious effort to define what a lawyer's Basic Training should consist of. Medicine has long worked out what a fresh graduate's "internship year" should consist of. Also, from the first day of medical school the students have their hands on patients so that the intership year is more of the same with less class work.

Law schools teach how to practice law by replacing some substantive courses with clinical courses. Clinical courses are require much more faculty contact time than a substantive course. Some clinical course faculty have tenure but the teaching is mostly done by people who are not tenured full professors and never will be.

The complaint that law graduates "don't know how to do anything" is different from a complaint that "they don't know anything".

A post law school internship requirement would be the best answer. It could be done by the law schools and would take a year after the JD. Faculty with at least 15 to 20 years of experiance would have to be hired, given tenure, and paid over six figures per year.

Its doubtful that law schools would do this unless the bar admissions authorities required it. This means that "how to be a lawyer" teaching would more than likely be by public agencies much as post-degree medical education is done by teach hospitals.

"Teaching to the test" will beat most tests.

Add Your Comment

Commenting is closed.

  • 1255 Twenty-Third St., N.W.
  • Washington, D.C. 20037
subscribe today

Get the insight you need for success in academe.