Undergraduate education and the aura of expertise
Rufus, our rogue classicist, has written a smart response to that Chronicle of Higher Education article on a guy who makes his living writing students’ papers. In the original post, I kinda-sorta gestured at the distinction between acquiring practical knowledge and acquiring a patina of academic credibility, but Rufus nails it here:
So, here’s an inchoate theory: all large and hierarchical organizations produce both expertise and the aura of expertise separate from expertise itself. The college of medicine, for instance, both produces medical experts and maintains the aura of expertise that surrounds its members. Both expertise and the aura of expertise can correspond, of course, but they don’t have to. Corruption arises when the aura of expertise does not correspond to actual expertise, and is compounded when members of the organization believe themselves to have expertise that they lack. Such large, hierarchical organizations suffer other related problems, such as a lack of informational flow between levels- people lie to their superiors and vice-versa- and a tendency to circle the wagons to protect incompetent members and maintain the organizational aura of expertise. And I suspect that these problems pervade all such organizations, be they governmental, corporate, academic, religious, or other.
I think this highlights a real problem in undergraduate education: actual expertise no longer corresponds with the “aura of expertise” a college degree confers. A medical school or an auto repair class can’t coast on reputation alone – eventually, bad surgeons or bad mechanics will cost these institutions credibility with the public and prospective applicants. The baseline for the success of undergraduate institutions, however, is much less rigorous: enough graduates need to get decent jobs after they leave, but there’s no way to actually gauge their preparedness for careers in a given field.
I’m told that my undergraduate experience provides a framework for critical thinking that is readily applicable to just about any field, but of course this is impossible to quantify. If I was a cynic, I might suggest that perpetuating an amorphous “aura of expertise” in place of something more measurable is actually in these institutions’ interest – if there are no standards for success, there’s no way to actually hold colleges and universities accountable for what they’re teaching. And sure enough, some savvy administrators seem to have realized that perception and prestige are just as important as academic accomplishment:
For similarly shrewd reasons, in 2002 GW decided that it needed a varsity squash team. The only other colleges in the country with varsity squash programs for both men and women are Brown, Cornell, Dartmouth, Harvard, Penn, Princeton, and Yale. A GW athletic director explained to the Washington Post that the whole point of the GW squash program was to attract students who wanted to attend an Ivy League college and couldn’t get in.
The accreditation agencies are asking for colleges to perform assessments to determine whether or not students are actually acquiring the skills their classes are supposed to be conveying. Much scrambling, plotting, planning, and wailing and gnashing of teeth is occurring as university departments attempt to measure the efficacy of their programs.Report
And here I thought GWU was really a real estate investment trust disguised as a university for tax purposes!Report
Appropriately enough, I’m reading this while on campus, so I’ve got to be brief for now. But I should note that I’m really a rogue classicist, since I’m studying the classical era for fun and profit, while nominally a historian of the Early Modern period- actually, I’m much more of a generalist than people who are more successful in academia!
Secondly, I should note that I’ve sent fellow academics, particulalrly those who are tenure-track into fits of sputtering rage (almost) by calling for an exit exam for undergrads. It’s quite a controversial idea, which suggests to me that the demand is growing. What academics say if you use dirty words like “assessment” is always, “But you can’t quantify what we teach!” Sure. I know that. Nevertheless, I can usually tell when I encounter liberal arts graduates who haven’t learned what they were supposed to while in college. I’m guessing the people hiring them can too.
Finally, to some extent, it probably has always been the case that the student himself is the final determinant of how much a particular student learns. It just didn’t seem the way to us nerds when we were undergrads.Report
inominate one,
Regrettably, the assessment that the accreditation agencies are asking for measures only the aura of success, not the real success itself. I have been furiously objecting to the assessment my department has had to do for the North Central Accrediting Association because a) it is clear that all they are really asking for is that we give them a report that they can stick on their shelf, that says we did assessment; and b) nobody has yet been able to explain to me how to do real meaningful assessment of my program.* Not that folks wouldn’t wail and gnash their teeth if they were really having to undergo serious scrutiny, but we shouldn’t confuse the current wailing and teeth-gnashing as a reliable signal that actual scrutiny is occurring. Personally I can hardly even work up the energy to make a good fake product, since I know nobody’s actually going to pay any real attention to what’s between the covers.
As a policy person, I’d be interested in real assessment. I think we’ve made some great changes in our department over the past few years, but all I’ve got to go on is my gut. I’d prefer to either find out we’re on the wrong track and need to change, or have solid data I could wipe people’s noses with.
___________________________________________
* Here’s a real scam: a large number of people are calling themselves assessment experts because they write lots of articles about assessments. 99% of those articles have some version of the following, “there’s no one right way to do assessment, and each program should find out what works for it.” In other words, these self-proclaimed experts really don’t know anything, and will obliquely admit it, but rather than go to the effort to actually do real research and find out what works, they just study what people do without regard for whether what they’re doing amounts to anything more than painting the leaves green.Report
Unfortunately for us college folk it’s a catch 22, with employers complaining that we “don’t have the right skills,” while still requiring a degree from those scheming universities that confer them. What’s a grad to do?
And maybe I’m wrong, but I think I recall businesses and companies saying the three most important skills were something along the lines of communication, organization, and ability to work in teams. As much as we can bash the colleges for the amorphous knowledge they declare to bestow, I haven’t seen some clear cut list from people on the employer side making it very easy.
Should I get a college degree that teaches me skills specific to one specialized industry? What happen to Companies training on the job? Give me a job in the mail room and I’ll work my way up. If only though, as even those jobs no longer exist.
Where have all the “entry level” jobs gone?Report
In the 90’s, I remember seeing the occasional request for people with 10+ years experience using Linux. Given that Linux was kinda created in 1991, this didn’t make a whole lot of sense in 1996.
There’s also the whole “Job Requirements: Electrical Engineer. Job Description: Change lightbulbs, manage extension cords” issue that always shows up in recessions.
There is a significant part of me that wonders if this isn’t the evolution of discrimination against particular sub-cultures manifesting itself. “You want to demonstrate that your culture and the corporate culture can co-exist? You need a degree.” Once upon a time, a degree wasn’t necessary… but the things that happened under those circumstances were found to be illegal. So they evolved into something legal. That’s it.Report
Several years ago, I was working for a company in an XML programming department of about 20 or so where I was the junior manager of said department. We had some jobs to fill (paying XML programmers under $10/hr will do that) and the new job requirements were so stringent that nobody in the department, including the senior manager, was qualified for an entry-level position. The senior manager didn’t have a college degree. I was six months shy on “industry experience” (as in the industry our IT company served, not IT industry experience).
The end result was that nobody ever applied and we scambled to find people we knew that didn’t have the requisite experience but that we could convince the right people to hire anyway.Report
I’d love for a third party industry to crop up, and a lot of recruiting services do this in some ways, where you can go and get tested as to your ability or skills in certain areas. For instance I might be proficient with graphic arts software, or web programing, even if I don’t have the requisite experience. How nice it would be to go somewhere and take an “exam” for such things, bestowing upon me some amount of legitimacy that would otherwise require a transcript or work reference.Report
Mr. Hanley:
I can see where certain forms of assessment might only measure the aura of expertise rather than expertise itself, and I see and agree with your point that ideally the aura of expertise and expertise itself should have a correlation coefficient as near as possible to 1.0. Certainly assessment itself should be assessed as well, until we’re caught in a Mobius loop of meta-assessment ad infinitum.
That said, it’s a double-edged sword to allow departments to choose their own assessments. Some departments will choose assessments which measure only the aura and not the expertise proper. Hopefully, most will choose useful, meaningful methods which measure expertise gained and eventually, incrementally, cumulatively students will improve. The department in which I’m currently a grad student has a favorable ratio of young, idealistic faculty who are trying to make positive change, at least until they get tenure.
Realistically, though, shelves will fill up with binders containing TPS reports and assessments and no real change will occur.Report
Meh, I’d settle for something that was far below a correlation of 1.0 if I could just have confidence that there was some reliable correlation at all. The problem is not just that we’re choosing our own assessments. If we were choosing from some off-the-shelf set, all of which had some known validity, I’d be ok with it. The problem is that we’re all pretty much creating our own from scratch. That takes up my time and pretty much guarantees a crappy product. I think that’s what ultimately has me so tremendously pissed off about the whole thing–I’m being forced to invest large amounts of time in producing crap. It’s like working in a Soviet consumer goods factory, except I’m not even receiving any compensation for it.Report
I’m being forced to invest large amounts of time in producing crap.
Isn’t that the human condition? Seriously, though, this is a solid point.Report
“if there are no standards for success, there’s no way to actually hold colleges and universities accountable for what they’re teaching.”
I agree, though the standards should be decided either by the labor market or third parties like recruiter services, not the educational institutions themselves.
Again, there remains the problem, that Jaybird gives a theory for, about this disconnect from bother sides, that is, both employers and colleges. If employers gave colleges a clear list, of wanting graduates to be able to do A, B, and C, can you think of any reason why colleges wouldn’t seek to accommodate them? Yet I don’t see employers being very clear about what it is the college grads are lacking, even while the labor force shrinks, becomes more productive, and corporations are raking in record profits (Chait at TNR for more on this).Report
I’m thinking that, were internships given a more important function, and especially at the undergrad level, a lot of the problem would go away.Report
Ironically enough, I have a few friends who went to GW and their curriculum emphasizes getting internships in DC during the academic year.Report
I’d have to ask my wife, but when she was at Waterloo (Ontario) as an undergrad, she did internships, and I’m pretty sure she said that they (undergrads in Ontario) all do them, no matter the degree.Report
Everyone should work for free in order to obtain skills, at the very least, rather than pay thousands in tuition, you can literally work for your education. Of course, why not cut out the college all together, and just be trained on the job, full stop.Report
I agree, which is why I think universities with co-op systems ca not be praised enough, e.g. Drexel and Northeastern.
Of course those schools are extraordinarily expensive, despite costs being buffered by the intermediate semesters of paid work. I still never understood why anyone would ever do an internship for “course credit” where you not only work for free, but actually end up paying the university for the internship.Report
The above is directed at you Han.Report
The students in my department are required to do internships. That was one of the changes I made.
Re: E.C. Gach–The point of requiring internships is that we recognize classroom education isn’t the only way to go. Of course that cuts the other way, too, which is why throwing out the baby with the bathwater and doing only on-the-job education may be less than an ideal solution also.
Truthfully, the most well-rounded, intelligent, and thoughtful people I know almost inevitably combine a liberal arts education with lots of work experience.Report
Isn’t what an undergraduate education in general is supposed to accomplish a logically prior question to whether or how one might go about assessing a school’s success?Report
I suppose so. But isn’t that another problem with a liberal arts curriculum? Beyond a few rote phrases (“provide a framework for critical thinking” etc), I’m not sure what a liberal arts institution is supposed to do.Report
It’s training for you to teach yourself anything.Report
I needed some cheap credits for my degree, so I took a course in Industrial Management, along with night-class types who actually, you know, worked for a living. Making things and stuff.
One of the most beneficial courses I ever took, still use what I learned there. Mebbe Liberal Arts should “liberalize” a little.Report
Actually, I suspect most liberal arts college offer such courses, if not that specific one. See the courses available in my College’s Accounting and Business Administration department, for example.
I think rather than lack of availability, the problem is that most such colleges, including mine, are in small towns, so we don’t have much of a working-for-a-living-and-taking-night-classes crowd to add there perspective in the classroom for the 18-22 year old students. Short of moving our campus to the city, there’s not much we can do about that. That’s too bad, because the younger crowd benefits from working in classes and project groups with the older and more experienced crowd. That’s one area where a lot of community and regional publics have a real advantage over us. But generally those types of courses are offered.Report
I understand, James. All I was saying was that I took Industrial Management quite by accident—it never would have occurred to me as an English major.
Mebbe there should be a list of electives, a Column B, of stuff a liberal arts major would find completely alien. Real World 101.Report
I hear you. Requirements differ among schools, as does the role of advising, so the likelihood of students taking those classes differs from place to place and adviser to adviser, I imagine. Listening to a bunch of political consultants last week, I realized that any of my students who are interested in political campaigning should all be taking marketing and management courses–and probably could skip 90% (or more) of their political science classes.Report
I fully agree with this. The question is, how do we assess that? Or put another way, how would the theoretical third-party industry that tests for it (that E.C. Gach suggests above) manage to test for it?
And I don’t mean that as the typical professoriate snark that “what we do can’t be measured by mere quantitative methods.” I believe in quantiative methods, and I want this “learning to learn” skill to be quantitatively testable.
I think figuring out how to do so is crucial for the future of American education.Report
It would be expensive no doubt, but so is every evaluation, SAT, GRE, LSAT, psyche evals, physicals, etc.
I would propose something along the lines of going in to the evaluation, you are given a certain topic and set of problems related to that topic. You would then have 24 hours to prepare a presentation and memo regarding what you understand of the topic, what factors relate to the problems, how would you would go about solving them, what you would take into account, what solutions others in the field may already have tried, why those will or will not work, etc.
You would then go in day II for a group evaluation (something to evaluate your capacity for “group intelligence”) where the group would be given a topic and set of problems, that then must be brain stormed, memo-ed, and presented within a simulated 8 hour work day period. The group meetings would be watched by an evaluator, one that was in another room perhaps.
What do you think of that for a rough basis?Report
I can’t necessarily critique it substantively. But it’s simply not practical. I can see a business using it as a process for hiring executive talent, but not as something that could be applied broadly, like an exit exam. And I think we need to try to find something that can be applied more broadly. So it needs to be relatively inexpensive.Report
Why wouldn’t it be practical?
You don’t think something like that could be implemented for $500 a pop? And the above was crafted with regard to Jaybird’s “learning to learn” skill.
Thousands of dollars are paid per semester, $500 for an exit evaluation doesn’t seem to be a burden.Report
I’ll add that these third party evaluators would obviously have an incentive to format their eval criteria to the needs of employers, with all employers listing which eval agencies they accept “scores” from.Report
According to the Cambridge Handbook of Expertise and Expert Performance, there generally is NO transferable “general expertise” to be found.Report
Well, that settles it, then, Mr./Ms. Hyena. The academy hath spoke—a liberal arts education is useless except for teaching liberal arts, as we suspected all along. The snake eats its tail.Report