• July 29, 2014

Many College Boards Are at Sea in Assessing Student Learning, Survey Finds

While oversight of educational quality is a critical responsibility of college boards of trustees, a majority of trustees and chief academic officers say boards do not spend enough time discussing student-learning outcomes, and more than a third say boards do not understand how student learning is assessed, says a report issued on Thursday by the Association of Governing Boards of Universities and Colleges.

The issue of academic quality is important for boards to consider because it is central to each institution's mission, and ensuring that that mission is upheld is as much a part of board members' responsibility as guarding the college's financial stability, said Susan Whealler Johnston, executive vice president of the association and author of the report. Board oversight of educational quality is even more crucial at a time of heightened calls for accountability and greater focus on the quality of educational services. (See The Chronicle's series on quality, Measuring Stick.)

The report, "How Boards Oversee Educational Quality: A Report on a Survey on Boards and the Assessment of Student Learning," is based on a survey conducted in November 2009 that asked 1,300 chief academic officers and chairs of board committees on academic affairs how boards oversee academic quality. The response rate was 38 percent, with 28 percent of trustees and 58 percent of chief academic officers participating. Almost one-quarter of the respondents were from public institutions, and three-quarters were from private institutions.

Results of the survey were mixed, Ms. Johnston said. While slightly more than half of respondents said boards spend more time discussing student-learning outcomes now than they did five years ago, 61.5 percent said boards do not spend sufficient time in meetings on the issue. A smaller proportion—38.5 percent—said enough time was spent on the subject in board meetings.

"There is plenty of room for improvement," Ms. Johnston said.

According to the survey results, business matters take up the bulk of board-meeting agendas. A majority of respondents—56.9 percent—said much more time was spent during board meetings on financial and budgetary matters than on academic issues, and 22 percent said slightly more time was spent on those matters.

That isn't a surprise, because many trustees have business backgrounds, Ms. Johnston said. There may also be a feeling among board and faculty members that teaching and learning assessment isn't something the board should be involved in (21.5 percent of respondents said focusing on student-learning outcomes was not an appropriate role for a board).

"I can't remember a serious discussion about this in my 12 years on the board," one private-university trustee commented in the survey.

2 Bottom Lines

The report refutes the idea that boards should stay away from questions of educational quality, saying that colleges have two bottom lines—a financial one and an academic one. While boards should not get involved in the details of teaching or ways to improve student-learning outcomes, they must hold the administration accountable for identifying needs in the academic programs and then meeting them, the report says. Boards should also make decisions on where to allocate resources based on what works or what should improve.

The survey found that many boards do not receive adequate information from college officials that would help them assess how students are progressing. The most commonly received information by boards was college-ranking data, which three-quarters of respondents said were very important or somewhat important in monitoring educational quality. Rankings may look at graduation rates and retention, however, but not at the quality of student learning. The second most common information source was alumni surveys, which Ms. Johnston said can be a good indicator of how well alumni believe their college education prepared them for the workplace or graduate school.

In the report, the association makes several recommendations for boards and college administrators, including that they devote more time during board meetings to discussions of what the college is doing to assess and improve student learning. Boards should expect to receive useful high-level information on learning outcomes, the report says, and should make comparisons over time and to other institutions. Training in how to understand academic and learning assessments should also be part of orientation for new board members.

Richard L. Morrill, president of the Teagle Foundation and a former college president, said monitoring academic quality was part of the active oversight that boards are responsible for. Boards should not tell faculty members how to do their jobs, he said, but they should hold the administration accountable for how the academic mission is carried out. Board members should receive information about learning and academic programs, ask questions about it, and monitor the institution's progress, he said.

Presidents and academic officers have a responsibility to provide information to board members that is easy to understand and monitor, so they can perform that oversight, said Mr. Morrill.

"When questions are raised around a board table, it has a powerful effect" throughout the institution, Mr. Morrill said. "I know that, having been a president and a board member."

Comments

1. butteredtoastcat - September 09, 2010 at 08:21 am

"Outcomes-based" learning, based on behaviorist assumptions and techniques, may not be the most appropriate measure for university-level learning. In fact, it may not be the most appropriate measure for learning period. The failure of outcomes-based education in the K-12 system (look at LA Unified School District as a big glaring example) should actually make higher ed administrators shudder for the future of the American college student.

2. impossible_exchange - September 09, 2010 at 08:47 am

This is part of the madness that is the American boss culture.
Those in charge do not have to get involved in the work of those who work "under" them. This Fordist model does not fit higher ed where the more talented, educated, qualified, and committed people in the system are the "workers" the professoriate. Admins and boards of directors are not experts in all fields, they cannot access the work of the university, they can only guide its success by hiring good people, and ensuring that the conditions for success continue thereby maintaining the college or university's vibrant culture.
It is not the job of board to look over professor's educational work.
Such concerns are for the department level.

The spread of assessment concerns in public ed amounts to micro-management of teachers by those not always qualified to do so.

3. bcagreenfield - September 09, 2010 at 09:27 am

If a board isn't holding university leadership accountable for program/school/university success then how can the university be successful? No doubt that professors and administrators can often be self motivated and hold oneself accountable, but isn't the job of a board to ensure the institution is achieving it's goals, whatever those may be?

I would assume the rollup of student performance against learning outcomes could be a critical data point against which the performance of a program/school/university can be measured and guidance offered by a university board who are indeed experts in strategy. They may not have deep expertise in subject matters, but they do need to understand are we doing well, not so well, and what are opportunities for improvement. Their perspectives on strategy could be invaluable to the institution despite lack of deep academic subject matter expertise. If the institution knows its assessment plan well, it is the data that is output from assessment the board is interested in.

It is the job of the board to challenge strategy and approach for an institution in achieving its goals. Assessment provides key points of data. If meaningfully assembled outcomes assessment data can be critical in the discussions related to program/school/university livelihood. I believe a discussion about outcomes assessment at the board level is entirely possible and meaningful without micro-managing.

4. chelley - September 09, 2010 at 10:06 am

If a university is run like a business, the students ARE your customers - without students the university wouldn't be in business.

Outcomes-based assessment and evaluation should most certainly be the concern of the board just as readily as it is the concern of the administration, faculty, and staff of the university. Student success (during their time at the university, and once they graduate) and the university's plan to achieve that goal, should be part of the university's mission statement. These tasks need not be considered micromanaging: the board doesn't teach, but it does need visibility of the status of student progress in order to make more effective decisions.

When the mission, outcomes, and goals are interlinked at the course, department, college, and university levels, the whole university succeeds. Accreditation becomes a motivator and not a chore, the university is better able to allocate resources (financial and otherwise) toward effective learning, and the university's reputation as a well-focused institution increases.

Obtaining assistance from the university's academic support unit is a great first step. Most units have evaluation teams that work with faculty, staff, and administrators to not only collect and analyze data, but help those entities to clearly report that information to boards and other stakeholders -something that is apparently not being done enough.

Academic support units are by their nature separate from institutional research departments in order to help the university build its capacity to evaluate student outcomes and understand how that information is relevant to all stakeholders, who in turn use that information to make more effective decisions - both business and academic.

Michelle Baron
Evaluation Strategist

5. 7738373863 - September 09, 2010 at 10:30 am

Boards act on inputs from the institution's senior leadership. The link to the learning process, as it is carried on by teachers and students, should be the provost or her/his designee. Or, if a board wants to get down and dirty, it should do so through its academic affairs committee or subcommittee. The board as a whole should keep a wise distance from the classroom.

More important than how the board informs itself of the successes or failures of the learning process is what it views the process in relation to. Most, if not all, institutions of higher education have a mission--either one that is self-proclaimed or one that is mandated by local, state, or federal authorities. The educational success of an institution should be measured in relation to its mission. Land-grant universities are answerable to the mission(s) stated in the Morrill Act of 1862. Denominational schools are answerable to the particular inflection their respective credal commitments give to the mission of higher education more generally. Institutions that stress their commitment to cooperative education should be assessing their success in relation to the effectiveness of student placements in educating and preparing students for their careers. And all institutions have--or should have a common core of courses used to prepare all of their students with a common educational experience that will support a lifetime of enlightened intellectual inquiry, growth and development. A mission can evolve or change over time, but the educational success of an institution must be viewed in relation to it.

The unfortunate legacy of No Child Left Behind is that we are leading with our behinds--we have gotten assessment ass backwards on the college level. Students may need a toolkit of specifiable essential competencies to walk in the door of a college, and if they do not have such competencies, then those students require the developmental education to attain them. What they need to proceed out that door and commence careers that are of value to themselves and of use to others is far more complex and diverse than assessment by the numbers can measure. In the Boston area, where I work, three students--one from Boston College, one from Harvard, and one from Northeastern--can each be success stories exemplifying the fulfillment of the university's mission, but the records of the three may have little in common, save for unquestioned excellence as reflected in grades and honors.

6. 22250655 - September 09, 2010 at 10:31 am

While Boards should not involve themselves in the mechanisms of the assessment of student learning, they should be setting clear goals for student persistence, success, and completion rates and holding the upper administration accountable for progress in those areas. My experience has been that Regents are more interested in capital projects and athletic success than in the actual mission of the university. An unexamined problem here is holding Regents accountable and assessing their performance in a way that has some teeth.

7. hawkeyecc - September 09, 2010 at 10:50 am

Because our board members are elected by the general public, and few have any educational background, they spend almost no time discussing student outcomes or assessment of learning. Their only concern is the budget. I tried, at one of our board meetings, to remind them about their obligation to support and ensure quality programing on our campus, but it was not well recieved.
Keep in mind, our board members include a car salesman, a plumber, a mail carrier, etc.. Some have no college education themselves, but because our board won a popularity contest at the polls, we are stuck with their leadership. Assessment of outcomes is not a high priority for this board.

8. 11220252 - September 09, 2010 at 10:52 am

"If a university is run like a business. . ." It is not a university! Let us start any discussion of these topices with three very old-fashioned assumptions: (1) A university is a community of scholars. (2) Students are not consumers, they are students. Consumers buy things. Students acquire the skills of scholarships. 3) Knowledge cannot be quantified. We don't sell knowledge at college.

9. thomaslawrencelong - September 09, 2010 at 11:07 am

One of the flaws in the "business" model for higher education is confusion about who precisely is the "customer." In a production view, the students would seem to be raw materials whom the intellectual workers turn into a useful product. So is the "customer" employers (for whom we provide "valued added" by crafting workers who derive profits for the companies)? Parents (since intellectual workers perform the useful function of preparing adult children to leave home and give their parents some leisure in their late adulthood)? Society (for whom we are preparing an educated citizenry, whatever that means)? There are competing interests among all these "customers." I'm also struck by the extent to which the business model has failed business in the long run; I don't know of any commercial business that has been as sustainable as the oldest universities.

10. clementj - September 09, 2010 at 11:42 am

Within some of the disciplines there are studies of outcomes and there is research on how to improve student learning. For example there is a branch of physics called PER (physics education research) that has developed diagnostics and methods for improving the scores on the diagnostics. The diagnostics of course are a moving target, but the various measures of outcomes tend to agree that certain types of teaching (not styles) do improve education.

The improvement not only results in better understanding, but also better problem solving ability. In addition there are some indications that student thinking is improved. So perhaps the boards could suggest that each school should do their own evaluations of student learning. But also that each school or department should collaborate with the cognitive psychologists and evaluators.

The current Outcomes-based assessment has the major problem that it has a focus which is too narrow. So using a one size fits all approach is not appropriate for universities. But the professors should be held accountable for learning about the educational research in their field of teaching. It is not enought to just know their field of academic research. If they are paid to teach, then they should be required to read the journals appropriate to their teaching.

There is abysmall ignorance on the part of college professors at both the 2 year and 4 year school with regards to the educational research in their field.

11. dav1dr0sen - September 09, 2010 at 11:47 am

Assuring outcomes are met and assuring quality are two separate issues. One is about achieving a goal and one concerns setting the goal. All professional accrediting agencies (NAAB, NASAD, AACSB, ABET, etc.) look at outcomes--the actual work being produced. However, they posit minimal quality standards. In those cases, it is clear that what controls the issue of quality is the end-user, the professions that students are being shaped to enter. But in areas like general education or "the generally educated student," what constitutes quality is not as clear.

By the way, in education the product IS the customer, as it is in a fitness center. And as in a fitness center the so-called product and customer must do the work to achieve the end.

12. williamcampbell - September 09, 2010 at 12:22 pm

I am currently a candidate for the board of trustees at Washtenaw Community College which is ranked as one of the top 100 community colleges in the country. Getting the board involved with requesting data from the administration regarding an assessment of student educational outcomes is at the foundation of my platform. As an inexperienced part-time Instructor in the physics department for 3 years, I was given freedom over content, testing, and grading methods as long as student evaluations were good. Mentoring was focused on improving numbers generated from Student Opinion Questionnaires. It seems to me that student opinions are what the President of the college is interested in as a measure of success. In March of this year I sent the following message to the board;

“What data does the Board of Trustees look at to ensure that students who transfer to four year universities have the skills and knowledge necessary for success?”

They have not answered my question as of yet. I am currently planning to meet with administrators at the college to discuss what is currently done to assess student-learning outcomes.

William Campbell
Campbell4trustee.info

13. betterschools - September 09, 2010 at 01:00 pm

A principal source of the confusion, although not always recognized as such is the fact that each person operates with a tacit conception of 'quality' that may or may not be congruent with those with one's colleagues. While this would seldom happen in discussions about scientific constructs, despite the fact that most of us hold Ph.D.s, we are not well versed in the contributions of philosophy to practical solutions resting on a construct of quality, in this case, academic quality.

There is no need to think in blunt terms when refined conceptual tools were invented and refined long ago. We need only learn and apply them to achieve desired solutions.

See: http://www.intered.com/storage/jiqm/v7n2_rhetoric.pdf

This article (13 years old from a 25 year old theoretical paper) contains a table of hierarchically organized 'academic quality' constructs (i.e., from least to most adequate, subsuming, range of application, etc.) Each construct is paired with a statement of its general characteristics and typical assessment measures. A second table analyzes common notions of 'quality' as applied to higher education. A third table identifies common conceptual and empirical criteria for evaluating the adequacy of different types of 'quality'.

While I may have pulled various issues together for the first time in that article, adding my own analytic perspective, none of the fundamental definitions of 'quality' were new at that time. All were then and are now immediately available to be put to use on this topic. NB: Before you apply the "not invented by me" criterion, take a look. I think you will find a level and type of quality that fits your philosophical presuppositions, tacit or explicit, and that you can apply immediately.

14. kurtx - September 09, 2010 at 02:06 pm

Hear, hear, 11220252. Clearly this needs to be said more frequently and louder.

15. betterschools - September 09, 2010 at 02:29 pm

A sometimes effective response to professors and administrators who assert that one cannot measure what they teach is this question:

"If no one can measure what you teach, how can you make a rational determination that someone has learned it, and how can you assign grades that reflect the inevitably different levels of learning of your students?"

16. qkoller - September 09, 2010 at 04:52 pm

Actually chelley, the students are not the real customers. Since American higher education has devolved into a euphamism for job training the real customers are the employers that hire the graduates. They are the ones complaining about the quality of higher education. What do they say about looking a gift horse in the mouth? They only pay a fraction of the cost for job training but do the lions share of complaining.

17. jesor - September 09, 2010 at 07:38 pm

I went to the mechanic the other day to get my car worked on. I had talked to my peers and looked at rankings online to figure out who was the best mechanic. I asked about prices and chose one that was well regarded, even though the cost for the repair would have been higher. I asked the mechanic what he was going to do with the car given it's problems and how those things would improve the performance of my car. He told me that it was none of my business and that because he was an expert in auto mechanics and I was not, that I shouldn't worry about it and just trust him. I went to another mechanic.

Any similarities?

18. samwise - September 10, 2010 at 07:32 am

to #4 chelley: The problem is that Universities should not be run as businesses because outcomes of actual education are not immediately measureable in total. Thus, from a business perspective doing what the customer demands or putting the customer's position as right because they are the customer waters down the educational process. Assessment of outcomes is a necessary part of education but boards that micro-manage a University to run it like a business simply ignore those concerns for easily measured outcomes that show a better bottom line. Students are the raw materials and the customers are those that eventually hire them. If you want to place students in a customer role then the choice of University is their customer function but once in the door the educational system must work to provide them with an education not a bottom line approach to getting them out the door. Boards that focus on Universities as a business choose easy yardsticks to the detriment of the students they are supposed to provide oversight for.

19. blogmerkezi - September 10, 2010 at 12:28 pm

So writes Rick Ulfik, organizer ogame of the “Make Waves for Change” event launching the Global Water sikiş Campaign today, in defining the current water crisis.

20. tcli5026 - September 10, 2010 at 12:42 pm

If you really want to use the business model (and I don't), it's better to see students as "employees." Yes, they pay for the privilege of "working" (so they're not really employees), but, like employees, they are responsible for fulfilling certain duties. They are given assignments ("work"), they are expected to produce a high quality "product," they are expected to on time for their "job," they are expected to follow directions and to pay attention to the "boss," and so on. If they fail to complete adequately any of their responsibilities, they are not "paid" their full salary (i.e., they are given a D or C); they may even be "F(ired)." If they do an outstanding job, however, they received a high "salary" and a slap on the back in the form of an A. They may even get promoted to much better place (i.e., graduate school), because they've demonstrated superior ability.

In this regard, we assess students all the time. And, this is one of my complaints about this "outcomes" and assessment craze. Why do we need to keep inventing new measures, when the ones we are currently using seeming perfectly adequate? To be fair, I understand the point of establishing some sort of "meta-assessment" measures, but so far, this process seems to driven by people such as Michelle Baron (#4)--an "Evaluation Strategist," whatever that is. And, I cannot help but think these "strategists" have an ulterior motive, which centers on taking more and more control of the educational process to serve their needs (whatever those needs may be--but, I am guessing, one of those needs is turn higher education into a business system first and foremost). But this raises the question, "Why?" After all, despite its problems (no system is perfect), the American higher educational system remains the envy of most of the world. We have tens of thousands of the best and brightest students from all over the globe coming here to study, and not just in the Ivies or top-tier R1s, but at all levels, including community colleges. By the standards of most corporate models, that must mean we're doing a brilliant job, right?

21. spearjh - September 10, 2010 at 12:58 pm

"While oversight of educational quality is a critical responsibility of college boards of trustees"

I don't know that I need to add to the din, but I stopped reading there. Has Ms. Masterson ever heard of a false premise?

22. der_gadfly - September 10, 2010 at 03:44 pm

Boards should not meddle in the daily activities of classroom learning or assessment, but they should support and provide the means by which this work can be done. They should ensure that their public perceives that the education delivered is of value. Beyond that, not much more to say.

Any entity can be seen in a business model, as all entities must pay attention to the inward and outward flow of money. In this pure sense, it is all business. An entity that consistently spends more than it makes is unsustainable in the long run. This is especially apparent as funding sources have dried up.

There is no confusion about who is the customer in a college environment. The matter is plainly obvious. In every 'transaction or exchange' there is an element of customer-and/provider. Therefore, we are at once, providers of services/knowledge and consumers. Suppose that a faculty member needs some clerical task done and goes to the administrative assistant and gets told to "go do it yourself". This is poor customer relations. The faculty member goes to the library to do some research and is told that the resources requested are not in because the check from the college bounced and the vendor will not send it without payment: who is whose customer here?

23. thunderboy - September 10, 2010 at 09:30 pm

Isn't the academic success of students the purview of university Senates?

24. 21wr12 - September 11, 2010 at 04:14 am

Well spoken #8. Students are products which employers hire. Once the employer discovers the product is unknowledgeable the product is fired.

Outcomes assessment should include surveys from employers regarding the performance of the graduates they hired.

Grades are not an indicator of learning. My Korean university has a policy of a maximum of 70% "As" and "Bs". This is intrepreted by the students as a minimum not a maximum. Consequently, professors comply with watered down syllabii and falsified grades or get fired.

25. nlasla - September 14, 2010 at 03:25 pm

Mr. Morrill recommends that presidents provide information about learning to board members. There is currently an initiative that asks presidents to ensure that at least once a year the governing board of their institutions receives and discusses a report about their efforts to assess student learning outcomes and to use that evidence to improve the quality of academic programs, co-curricular programs, and support services. To read more about this initiative, the Presidents' Alliance for Excellence in Student Learning and Accountability, go to: http://www.newleadershipalliance.org/what_we_do/presidents_alliance/

Add Your Comment

Commenting is closed.

subscribe today

Get the insight you need for success in academe.