A Response to Anant Agarwal's Call for Teachers to Join the Revolution

This past weekend Anant Agarwal, the president of edX, published an op-ed in The Observer titled "Online Universities: it's time for teachers to join the revolution."   There is much in his rather self-serving op-ed to criticize (and parody, as Jonathan Rhees has done so brilliantly), but perhaps the most important thing to point out is the false premise of his title.  Current teachers are cast as Luddite resisters who have dug in their heels and are opposing something that is improving education.  This is a complete misrepresentation of practicing educators at all levels, many of whom regularly use a range of education technologies as well as tried and true pedagogical methods to produce learning in their students.  In fact, education technology has been a presence in K-12 as well as post-secondary classrooms for at least a decade, and has expanded tremendously, especially on college campuses, thanks to the availability of fast broadband connections.

To suggest that the pre-MOOC university professor was teaching with chalk and yellowed notes is an absurd caricature whose sole purpose is to position MOOCs as the shiny new thing that will revolutionize education.  Agarwal declares that the "days of the old ways of teaching are numbered".  While there are certainly examples of "the old way of teaching" on every college campus, I suspect that they are vastly outnumbered by the many innovative, technologically savvy, and dedicated faculty who are working every day to improve student learning.

MOOC supporters repeatedly point out that the platforms and price point (free) improves accessibility.  I will leave it to others to explicate why this isn't quite true (and here).  Certainly, by putting courses on an open access platform, their is the potential to improve access.  But it is far from clear that MOOCs really are doing much beyond making lifelong learning more convenient; and targeting gifted learners in third world countries.

The most significant source of my irritation with the self-promoting MOOC rhetoric is centered on the claims that the mode improves learning--and, specifically, that it can support better learning than can a campus-based, large enrollment course.  In basic terms, there's no evidence to support this claim, at least not yet.  Furthermore, given the demographics of current MOOC users, it seems to me unlikely that even extensive study of  data from individual courses or users is going to be generalizable beyond a particular course or a small subset of users.   This is a big claim to make, and it requires clear data, not anecdote and declarations.

Working teachers need to know how particular students at particular universities in particular courses learn.  Much more useful would be funding projects to study currently enrolled students, especially a public universities.  As well, we need to know how best to help our beginning students learn HOW to learn, especially how to learn in a post-secondary setting where instructors aren't teaching to tests.  Since the majority of  current MOOCs users are degree holders and experienced learners--that is, they are people who already know how to navigate a course, study, be self-motivated, etc--the data being collected is unlikely to yield much of value.  This will change as more of the students are actual college students with all the complexities that college students present, when the data sets can be narrowed down to "current UT students taking X course" but that is years in the future.  As Rob Reich explains, "If MOOCs promise to enhance student learning, they must show that they deliver at least as powerful outcomes as traditional lecture classes in universities and community colleges.  If not, their virtue is their democratizing potential; but they will only be better than nothing."

Agarwal writes, "MOOCs are also improving the quality of education.  Online learning promotes active learning, where the learning watches videos and engages in interactive exercises."   I agree that MOOCs are improving the quality of education--but they aren't doing it alone and they aren't doing anything revolutionary.  They are simply hacking the flipped class model--and replacing the in class part of the formula with a (generally chaotic and ineffective) discussion board.  Furthermore, the notion that interspersing a few questions into a video creates an active learner is absurd.  It's better than simply lecturing at students; but it takes a lot more than a few questions in a video to create active learners.  Likewise, it takes a lot more than a few i>clicker questions in a lecture to truly engage students.  I've observed many a lecture class where students are shopping for clothes but still able to stop and click in on questions.

Indeed, much of what Agarwal has to say about how MOOCs facilitate learning imagines an ideal learner who uses the course precisely as it was designed to be used.  Yet teachers  know that this isn't how it works in real life.  It's absolutely true that videos allow for self-paced learning, but that doesn't mean students will use it that way.  In fact, in my own class, I found out that very few of them did.  Sure, in an ideal world, they would watch the videos carefully, engage with the embedded questions, and review confusing points.  The problem is, students rarely behave like ideal learners and so do not get the benefits that they could from the technology. We educators may understand that a key to learning is practice with retrieval; but students don't know that and will often find ways to avoid doing it unless their grade depends on it.  But again, MOOCs aren't the first to recognize the value of retrieval.  Faculty have been using LMSs like Blackboard to do practice quizzes and the like long before the advent of MOOCs.  As Glance et al. note in "The Pedagogical Foundations of Massive Open Online Courses," "MOOCs are in essence a restatement of online learning environments that have been in use for some time." 

I am similarly puzzled by the claim "students have always been critical of large lecture halls where they are talked at and declining lecture attendance is the result."  This is not at all my experience.  Quite the opposite.  Students enjoy a good lecture, one where they can scribble down the notes on the PPT, memorize them, and do well on exams.  It is unclear to me what the benchmark for declining student attendance is--this observation is meaningless without more context and data.  Student attendance has little to do with the quality of a course, in my experience, and much more to do with time of day, time of semester, whether they are commuting, etc.  In truth, one of the major challenges in moving from a lecture model to a model that emphasizes active learning is having to train students to be engaged learners.  Still, many complain that they would rather just be lectured to.  Yes, the ideal student would recognize the value of being an engaged learner; but most of us are not teaching these ideal students.  In fact, most of us have to spend a considerable amount of time and energy teaching our students who to be students, teaching them (and then repeatedly reinforcing) good learning strategies.

After declaring that the revolution has arrived, Agarwal turns to the flipped class model.  Indeed, it's become increasingly clear that the aim of the MOOCs vis-a-vis brick and mortar campuses, ultimately, is to provide content for flipped classes.  I've written elsewhere about why this is misguided and misleading.  In a nutshell, because the difficulty of a flipped class isn't moving content outside of class, it's doing everything inside of class.  Yet, by fetishizing content delivery and the professors who do it, the MOOC Incs have encouraged the belief that universities can save money by hiring non t-t faculty to run these flipped classes.  This is a recipe for disaster and demonstrates how little the MOOC Incs understand (or care about) the pedagogy of the  flipped class.

The op-ed concludes with the declaration that "it's time for teachers to rethink learning methods."  It is unfortunate that Agarwal doesn't realize (or won't admit)  that this was happening long before MOOCs came on the scene.  MOOCs are doing nothing new, apart from experimenting with large scale delivery.  Certainly, they have encouraged an engaged discussion about technology and pedagogy; but thus far, they are using methods that have long been used on brick and mortar campuses--indeed, if anything, MOOC pedagogy feels a bit retro to most of us (breaking up lecture with questions; discussion board; quizzes).  I can imagine a time when MOOCs might advance our understanding of student learning and make real contributions to pedagogical theory, but they aren't yet there.  The MOOC Incs would do well to recognize this point and refrain from perpetuating ridiculous and unfair caricatures of teachers. 

Graphic: A Closer Look at MOOCs

This morning, the Los Angeles Times published the first part of what they are labeling a Roundtable Discussion about MOOCs. Of the participants, two are faculty who have or are preparing to teach a MOOC on Coursera (Al Filreis at Penn and Cathy Davidson at Duke); one is a professor who teaches online courses at University of Illinois, Springfield (Ray Schroeder). The fourth is a scholar and cultural critic who has regularly worked to draw attention to the labor issues that are raised by the push to accredit MOOCs (Ian Bogost at Georgia Tech). In this first installment, the four faculty members issued position papers. There was essentially no engagement by the participants with the particular points raised by the other papers (outreach and increased accessibility were the themes of the first three; economic/labor implications was the theme of the fourth). Hopefully, the follow-up tomorrow will include some efforts by the three supporters of various forms of online learning to engage with the real concerns raised by Bogost. And, in turn, one hopes that there might be a way to talk about ways that the outreach and accessibility issues--both real benefits of the MOOC model--can be addressed without contributing to the ongoing efforts to outsource teaching at public colleges and universities.

MOOCs are multifaceted, with clear benefits. They are also a kind of Pandora's Box that, rightly, many are afraid to open. We seem to have a good sense of what the pros and cons are, as well as the politics around each position (including the refusal to acknowledge that the decision to teach on a MOOC platform is, in a way, already taking a position in the debate). Now we need to figure out how to move the conversation beyond this outlining of issues, to find some strategies for managing the inevitable disruption that large-scale, platform-based teaching is going to cause at once state-supported institutions.

Thank you to Allison Morris for providing me with this wonderful graphic that outlines the pros and cons of MOOCs so beautifully (graphic produced by OnlineCollegeCourses.com).

My Summer MOOC: Archaeology's Dirty Little Secrets

I was delighted to learn that Dr. Sue Alcock would be offering a short MOOC this summer on Coursera's platform.  Dr. Alcock, a renowned classical archaeologist and gifted teacher, directs the Joukowsky Institute for Archaeology and the Ancient World at Brown.  The course, "Archaeology's Dirty Little Secrets" is well-suited to the MOOC mode.  Archaeologists have long been experts at public outreach, in part because their work is expensive to fund and very frequently depends on private donations.  As well, because they work on material culture, on stuff, their work is of far more general interest than, say, analyses of Vergilian metrics.  It is also a good course to run in the summer, when the teaching team is at work on their sites scattered around the world.  Besides introducing takers to some of the basic concepts of archaeological study, it is sure to inspire wanderlust!

The course is well-designed, not least because it doesn't even try to be a college-level course.  It does very well what it is trying to do: provide outreach to the general community; inform the public about some of the major issues at play in cultural preservation; raise awareness about (and, perhaps, funds for) Pompeii--an amazing site that the Italian government is letting fall into ruin; and give users a sense of what academic archaeology is all about and perhaps encourage them to take a real class at some point.  This course reminds me very much of the free lecture series that my department does from time to time for (mostly) retired adults.  It is meant to generate interest and provide an intellectually satisfying experience.  It is not meant to be a credit-bearing course even if it is full of interesting content.

As an aside, I wonder if part of the frustration and confusion about outreach efforts like "Archaeology's Dirty Little Secrets" is the fact that they are presented like actual, credit-bearing courses: there is a syllabus; assignments are due on particular days; there are quizzes; there is a professor lecturing on content.  Users are called students.  Certainly, some MOOCs really are designed to replace a credit-bearing course, particularly in math and CS.  In the humanities, however, they are generally not intended to do so.  Dr. Alcock's course certainly isn't--it's a shell of a course she taught for credit on the Brown campus and lacks much of the serious content and assessment that her for-credit course surely had.  That's not to say that the MOOC version is bad--in fact, it's a fantastic introduction to archaeology and the ethics of cultural preservation; and a great way to raise public awareness about the need to preserve such sites at a time when so many are being destroyed in North Africa and the Middle East.

So, some great things about this MOOC: first, it is not especially lecture based.  Each unit does have about 30 minutes of lecture from Dr. Alcock, but it is not the centerpiece of the unit.  The unit also involves demonstrations from Dr. Alcock, informal in tone and held in her very messy office.  There are conversations with various young faculty/postdocs about their own projects; and, finally, short slideshows about selected topics done by the teaching team.  This last feature, "Peoples, Places, Things" is my favorite and is something I am going to steal and integrate into my own hybrid (and, eventually, online) class.  The integration of different voices, different faces, is a big plus for the course (and, interestingly, is a tactic used by one of the few women chosen to use the Coursera platform).  I also liked that there are options for each assignment, ranging from writing an essay to interviewing friends and family to a more hands-on exercise.  Many people won't take the time to do these assignments, but they are well-conceived and a fantastic way to get users to engage (and another thing I am going to use to inspire my ongoing development of such activities in my own class).

There is amazing content in the course and it models some effective ways to teach material culture to undergraduates.  It lays out the major questions that any Intro to Archaeology course ought to include.  At the same time, the mode--a MOOC--means that it is a struggle to support the kind of engaged student to student interaction as well as instructor to student interaction that really pushes individual students to engage with the big questions on their own terms, with push back from content experts.  There is a discussion board and it has some interesting threads on it.  Overall, though, the board is used primarily for procedural questions.  It's a challenge to get students to discuss in a face to face classroom.  That challenge is magnified many times over on a discussion board--especially a discussion board for thousands of students from a wide range of backgrounds.  Dr. Alcock opts for an informal tone that works for her.  She comes off as smart, goofy, and accessible.  Yet, ultimately, she's not accessible.  I can't knock on her office door and have a conversations with her because she's in Petra, working on her research--as she informed me in a recent email.

All to say, this course is a wonderfully conceived way of using the Coursera platform to accomplish a range of important and useful objectives--none of which were to offer a course that replaced a credit-bearing course.  The motivated student will have a great time learning about archaeology and engaging in the various activities conceived by Dr. Alcock and her teaching team.  They might even have some productive discussions with others in the discussion forum.  At the same time, it is important to reiterate the point that this is not a college-level course on ancient archaeology.  Rather, it's a kind of teaser that, ideally, will encourage students to take such a course at campuses around them.  It might also encourage the general public to understand why archaeology matters and be more inclined to support the funding of preservation and research projects.

When a Butteryfly in Cambridge Flaps its Wings

A good friend and former grad school colleague of mine is an Associate Professor of Classics at a small liberal arts college in the Pacific Northwest.  Her college is tuition-dependent, has a very small but spectacular classics department, and depends on faculty to teach not only a 3-3 load but also to take on several additional directed readings courses as well as thesis students.  In addition, she is very involved in the college community through committee work as well as supervising several campus student groups.  During the academic year, she is incredibly, crazily busy.  My job at an R1, in a department with a strong graduate program, takes a very different form from my friend's job, most obviously a lighter teaching load (2-2).  Still, I would assert that I am just as busy as she is, albeit with different kinds of tasks.  Over the years, our phone calls during the semester always seem to revert to a kind of friendly competition over whose job is more exhausting.  At some point, I realized it was impossible to determine because our jobs are simply too different to compare--despite the fact that we are both Associate Professors of Classics.  To really understand one another's position, we would need to trade jobs for a year (at least).

In conversations with this friend and others in my field who teach at a range of different colleges and universities, I repeatedly observe just how different our jobs are--apples, oranges, peaches, and bananas.  This is an important point at this particular historical moment, as we all try to make sense of the MOOC phenomenon and do so largely in terms of our own limited departmental and institutional experience.  The tendency to assume that all jobs are basically the same, apart from the "research" vs "liberal arts" distinction, clouds the issue in some significant ways; and, I think, it's part of what is preventing a more constructive conversation about MOOCs among the academics most invested in the roles they now and will eventually play in higher education.

The conversation around MOOCs has become noticeably polarized in recent months, fueled by academics (including me) who are skeptical of the mode and concerned that it is being viewed as a magic bullet by state legislatures and regents in particular.  Faculty who are providing their courses on the Coursera or EdX platform have been criticized for their complicity in what seems to be a persistent move towards the outsourcing of teaching at public institutions.  They understandably feel defensive and unfairly criticized. The situation was not helped when the Chronicle published an article suggesting that MOOC faculty as a group had given little thought to the potential impact of a more widespread distribution of their courses to other institutions, where they would be offered for credit.  Many faculty, who decided to offer their course as a MOOC--for very little by way of financial compensation and motivated primarily by a desire to increase access to their course--are justifiably feeling victimized by the increasingly hostile, anti-MOOC blowback.

Cathy Davidson recently posted "Clearing up Some Myths about MOOCs", in which she reviews a number of criticisms of MOOCs and offers her response ("so here's an account of what I've learned about the MOOC myths from what I've done and learned first hand.  I don't know what of these things is true beyond my experience but the point of first-hand experience is to counter myths with at least some substance and reality, however limited").  She has several great points to make and many of these points could be expanded on by those who have been students of MOOCs or who have other kinds of experiences with larger scale online instruction.  Yet, at the end, the post only invites those with first hand MOOC teaching experience to contribute their views. Still, it is a far more reasonable and thoughtful engagement with serious criticisms of the MOOC model than the dismissive views of Northwestern Journalism professor Justin Martin.

It is worth pointing out that several of the particular anxieties discussed by Davidson (e.g. concern that MOOC faculty are going to get super rich) are not the source of most of the anti-MOOC anxiety/rage/well-founded concern.  Some of the myths are related to the major criticisms of skeptics--high dropout rate, unproven pedagogy that seems to rely largely on lecture in the majority of MOOCs--but they only skirt around the elephant in the room.  What is that elephant?  Accreditation. If MOOCs had remained in the realm of outreach and non-credit continuing education; if they had simply offered access to skills-based courses to those who would otherwise have no access, most (all?) with an investment in the future of higher education would be supporters.  The problem is, they are positioning themselves as "disruptive" to the traditional system of accreditation (and Congress is listening).

The problem is, the MOOC Incs need to return a profit to their investors, and they are not going to do this without a revenue stream.  That revenue stream requires accreditation that leads to some end (e.g. certificate, degree).  Whether it is by some sort of licensing agreement with less prestigious universities (San Jose St. and California more generally is ground zero for this model); by creating very inexpensive degree programs (e.g. the MS in CS at Georgia Tech, with corporate sponsorship from AT&T); or by forming partnerships with selected public institutions, the apparent intention of the MOOC Incs is to function as a middleman between the producer and consumer of course content. I agree with the defenders that the current MOOCs--many of them only 6-7 weeks long--are unlikely to be packaged and marketed to less prestigious institutions (though some may be).  Far more likely to happen: Coursera, Udacity, and EdX will trade on the hype generated by their elite partners in order to persuade less elite partners to select a small number of "superstar" faculty to use their platform to develop courses in house and deliver them to thousands of students at a time, across entire state systems.

Typically, faculty don't need to worry about what is happening at other institutions around the country or world.  What happens at Cal State Northridge has little to do with what is happening at Harvard or Oxford. Yet, in the specific instance of MOOCs, institutions (and faculty) do not operate in isolation but rather, as part of a larger and even global ecosystem. Suddenly, what happens at Harvard may well have quite a lot to do with what happens at Cal St. Northridge. It is not enough to assume that academic standards will dictate administrative decisions; and that face to face instruction will always remain the most common form of instruction just because it produces the best learning outcomes. Still, the extent to which individual faculty--much less institutions--are responsible to the community at large will be debated.  But it should be debated rather than sidestepped.

In the face of severe budget shortfalls and an apparent unwillingness of states to invest in education, public institutions are desperately searching for ways to cut costs.  The MOOC model (not necessarily the existing MOOCs themselves, but the model of large scale courses) is, at least at first glance, very appealing: expertly delivered content that only requires a teaching assistant to walk the students through the details of the course.  While this may strike faculty at elite, private institutions as an extremely unlikely outcome, I suspect it will begin to happen in various forms over the next several years at public institutions like my own.  Exactly what role faculty at Harvard and Yale and Penn and Duke are going to play in all of this is unclear (and, most likely, up to them).  More clear is the fact that public institutions are determined to cut instructional costs.  They have to continue to teach large numbers of students with fewer faculty than ever.  Expertise is expensive and public institutions have long led the way in scaling up class sizes as a way of cutting instructional costs.  It is inevitable that large-scale, platform-supported courses will become one way of addressing this problem, at least to start.  The effect this will have on important factors like student learning, retention rates, and  time to degree are entirely unclear at this point. 

I agree with Cathy Davidson, Al Filreis, Noel Jackson, and others that support cautious experimentation with MOOCs and other kinds of larger scale online instructional modes.  I appreciate the potential benefits of MOOCs in terms of increasing accessibility, but my immediate concern is more personal: the role that MOOCs will play in the curricula of public institutions like my own.  We need to know better how these work (and don't work); and we need to ground the discussion in evidence rather than speculation (one hopes that the MOOC Research Hub, funded by the Gates Foundation, will be useful in this regard).  We need to be developing a good sense of best practices for different disciplines and types of content.  We need to understand the differences between learners who come to a MOOC with a college degree and well-practiced learning behaviors versus those without any of these skills.  We need to better understand what we gain and lose when we shift to a large-scale mode of content delivery/instruction with little opportunity for student engagement (because, if we could afford the instructional support that is required for meaningful student engagement, we wouldn't be resorting to this format in the first place).  For all of this to happen, we need to be having a thoughtful, engaged, sometimes combative conversation that involves everyone who has a stake in the outcome.

******************
6/14/2013: Roundtable discussion of MOOCs in LA Review of Books.  Perfectly illustrates this phenomenon of supporters and critics speaking at cross-purposes.  Everything each person says is valid and true, but they do not advance the discussion of how to deal with the MOOC incursion into public higher ed.

Not to pick on CD, but this tweet captures my point brilliantly: there's no debate at all (and it's not live, it's in the very dead form of print).  These are four "position papers" in which each party carves out their space in the debate.  There's no real debate at all.  Now, hopefully, that debate will come in the follow-up tomorrow and in the form of comments from readers.  But thus far, because there's so little attempt by anyone to engage with the arguments of others, it seems more likely that people will endorse positions they already hold rather than truly collaborate on a set of approaches, best practices, etc.  But perhaps I am being pessimistic...

6/13/2013: Colleen Flaherty, "It's My Business" ("MOOCs – or massive open online classes championed by many as a way to democratize education and lower costs, and criticized by others for what they see as the courses’ potential to diminish educational quality and supplant the role of the professor – and other online education initiatives have forced an alliance between professors across disciplines, Nelson said, in their growing concerns about intellectual property and academic freedom.")

Peter Schmidt, AAUP Sees MOOCs as Spawning New Threats to Professor's Intellectual Property

The Myth of the Super Professor

Yesterday I wrote about the myth of the bad professor, a myth that has been a central feature of the narrative of a crisis in higher education.   The myth has been a driving force behind the daily calls for the reform of higher education--preferably by Silicon Valley VCs who are entirely outside of the system (and who, on the whole, seem to understand few of the complexities of higher education in the US).  The topic evidently hit a nerve with many because, in about 24 hours, the post had almost 500 hits.  Among other things, I think that points to the fact that so many of us are fed up with the constant attacks on our abilities and efforts.  It is enough of a challenge to inspire often undermotivated students to do the hard work of learning; the last thing we need is to defend ourselves from the politically motivated attacks of those who have never walked in our shoes and who have no idea what is actually happening on the ground.

Yet, to an extent, we have left ourselves vulnerable to these attacks because, in all the years that college teaching has been professionalized, we've never really developed a coherent and consistent system for measuring student learning in our classes.  In many respects, we ourselves are guilty of perpetuating the myth of the bad professor because we have not adequately challenged the myth of the great professor.  If the bad professor is phoning it in, the great professor is the charismatic sage whose students hang on to his (it's usually a he) every word.  When faculty are evaluated for annual raises, tenure, and promotion, student evaluations are consulted; teaching awards are often a result of student nominations and student recommendations.  At no point in this process is any significant attention paid to student learning.  The best professor is the one whose students love him and want to impress him (and, to be fair, are hopefully inspired to work hard in the course because of this love and desire to impress).  But shouldn't the best professor be the one who produces the highest learning outcomes in a cohort of students?  Shouldn't we be evaluating the quality of instruction not on the performance but on the results of the performance?

Most faculty will agree with the research that repeatedly demonstrates that teaching evaluations are not a very effective measure of instructor quality or student learning.  There are a range of age and gender biases at play.  One recent study found, unsurprisingly, a strong correlation between the attractiveness of the professor, the grade the students thought they were getting, and the positivity of evaluation of the professor and course.  We all know these things are true, yet we continue to play along.  One of my favorite stories about gaming course surveys goes as follows: during the semester, slip into the class discussion commentary on each of the survey questions, telling the students exactly how you are doing whatever the question asks (e.g. emphasizing repeatedly your accessibility).  Do this over and over.  By the end of the semester, the students will have internalized the narrative and will rate you and your course highly.  In my own case, I know that if I want to have outstanding course evaluations, I just have to let everyone think they are getting an A up until the time they submit the evaluation--and then use a high stakes final exam to sort them out.  I've never actually done this, but know plenty of people who do.

So what do we need to do to shift the focus from the personality of the professor--and the cult of personality more generally--to evidence-based arguments about instructional quality and student learning?  First, we need to assess our courses and students much more deliberately.  We need to be able to demonstrate in some terms the value added by our courses.  Yes, I know this plays into the rhetoric of the "outcomes" crowd.  It will inevitably undervalue all sorts of difficult to evaluate skills like critical thinking; and it assumes that a course's value is immediately known at the end of the semester (though this second issue could be addressed with follow-up surveys).  Many of us feel more comfortable with the current system, even while acknowledging its imperfections--including me.  At the same time, by essentially allowing ourselves to be evaluated based largely on features of our personality, we are laying the foundation for our demise (because there's always someone else who is even more accessible, entertaining, etc.).  MOOCs perpetuate the notion that good teaching is equivalent to skilled public performance rather than demonstrable learning outcomes.  An important first step in responding to the pedagogical claims of MOOCs is to insist that courses and instructors demonstrate student learning outcomes.

As part of the redesign of my Intro to Rome class this past year, I instituted a thorough system of assessments of the course.  Students had multiple opportunities to comment on all parts of the course.  This was interesting and instructive for a range of reasons, not least of which was because we also had data that allowed for a comparison between their self-reports and direct assessment of various behaviors.  It quickly became clear that there was a significant gap.  One place where this gap was enlightening to all of us was in the assessment of the implementation of an "ethics flag."  Three large enrollment classes implemented the flag in Fall 2012.  One class was lecture-based, 220 students.  The second was a mix of lecture and discussion sections, with 300 students.  Mine was flipped, with discussion as primary feature, with 400 students.  When the students were assessed, they reported the most satisfaction and learning in the lecture-based class; then the mixed class; with my class in last place.  When the flag implementation committee did a direct assessment of their work, the results were exactly the reverse.  They may have "liked" my class the least, but they learned the material much better--not surprising since I was requiring them to engage with it actively rather than passively.  This spring, the discussion section course and my course were offered again and the results were the same, but with a slightly smaller gap and enough student comments to be pretty sure that perceived workload was inversely correlated with student satisfaction. Alas.

When we saw these results, they made sense--but were also an important reminder that student self-reports often reflect their sense of comfort.  Many of them are still far more comfortable learning passively via lecture than in more active forms.  Indeed, as my campus has worked to "blend" a number of gateway courses through the Course Transformation Program, a consistent feature of the blended courses has been lower student evaluations.  There is a clear and indisputable inverse correlation between level of student engagement required and student satisfaction with the course.  What we are learning is that the techniques that produce the greatest learning gains and best prepare students to progress through degree programs and graduate on time aren't necessarily the courses (and instructors) that they love the most.  Often, this is because such courses require significant and consistent engagement.  At the same time, these instructors are clearly doing their job of producing significant learning gains in their students.

The time has come to abandon the myth of the great professor--the lecturer who keeps the audience rapt in their seats, scribbling down his every word, chuckling at his witty jokes, in awe of his brilliance.  Certainly, many professors are skilled performers and many of the students in their classes learn well. But there are plenty of boring, bad performers who are very good at designing courses and whose students learn at very high levels--even if those students didn't necessarily love the professor.   Teaching is ultimately about student learning.  If we continue to insist on defining our best professors as those who are the most gifted performers, but without any way to quantify what this means, we make it difficult to defend ourselves from the accusations of bad teaching.  If we shift to defining good teaching through demonstrable student learning, we make it very difficult for these outside (and sometimes inside) attacks to carry much weight.  Of course, this shift has to start at the highest levels of the university, by creating a system that evaluates and rewards student learning more than instructor likeability.  Likeability matters.  It matters that students enjoy a class.  But, at the end of the day, what matters most is that they learned the course content.

[disclaimer: my students generally like me and give me high course evaluations.  I'm not writing this because of sour grapes.  Rather, after a decade+ of teaching, I've seen repeatedly that the system is set up to incentivize a kind of teaching that does not always serve the students' best interests.  And, at the moment, this system has left faculty very vulnerable to charges that they aren't "good", with no clear way to rebut such charges.]

The Myth of the Bad Professor

Back in my youth (not that long ago...really!), teaching was generally viewed as a respectable profession, filled with dedicated and patient professionals who regularly went above and beyond the call of duty to look after the welfare of their students.  Sure, there was the pot smoking 9th grade American history teacher who had us spend more time doing guided meditations (nap time!) than learning about the Civil War.  But what I remember were the teachers who assigned several essays/week (the grading!) and taught me how to write before I went to college; the teachers who coached us in after school activities and served as advisers to clubs; the teachers whose doors were always open and who offered great advice for college applications. I grew up in California, at a time when California invested in public education at all levels.  For the most part, it worked pretty well and I arrived at college very prepared to succeed.

These days, though, barely a day goes by without some new indictment of the teaching profession.  Teachers are incompetent, unmotivated, phoning it in, living off the fat of hard-working Americans thanks to the power of labor unions.  At least for awhile, it was the schools who were underfunded and failing; but now the blame has been shifted to the teachers.  It isn't state legislatures who are failing students (by slashing education budgets); it is badly performing, unaccountable teachers who are failing.  If students are being left behind, we are told, it's not because of larger socioeconomic factors or underfunded schools; it's because teachers aren't doing their job.

If K-12 teachers are bad, college professors are even worse.  They only care about research.  They devote as little time as possible to undergraduate education.  Their "grading standards" are irrational. They only care about money--and especially about making more money than their colleagues.  They use grad students as pawns to carry out their personal, interdepartmental vendettas.  Managing faculty is like herding cats (ok, this one might actually be true).  These caricatures have given rise to a large set of personalizable "professor" memes on http://www.quickmeme.com.  It's not funny to praise a professor for doing a great job of inspiring and facilitating learning in trying times.  No, it's far more funny to seize on outdated caricatures of professors assigning their own textbooks to make a profit off of their students; of professors assigning too much work (or, alternatively, constantly canceling class).  I'm sure that every university still has a few professors who live up to these caricatures.  What is missed in all of this, however, is the extent to which classroom teaching, even in large lecture classes, has improved substantially on campuses across the country in the past decade.

The myth of the aged, rumpled (usually male) professor reading his yellowed, dog-eared notes is just that....a myth.  Sure, this character used to be common--I can think of a number of such characters who taught my undergraduate classes (including one professor who still used purpled exams produced on a mimeograph...in the early 90s).  But I suspect that, even on R1 campuses that put a bounty on research productivity, most instructors are, in fact, very good teachers.  Many are outstanding.  It is certainly true that many of us would benefit from additional professional development, in part because quality teaching can benefit from expertise with educational technology tools well beyond an LMS.  Many of us would benefit from a system that placed more value on high-quality undergraduate (and graduate) teaching.  Still, given the current state of affairs, the remarkable thing isn't how many bad professors there are out there, but how many good ones.

As Melonie Fullick so aptly articulates, "What I find deeply uncharitable (and inaccurate) is to generalize this experience even to the majority of university teaching faculty. I’ve given academic lectures in various classes, and I can assure you there’s no reason to assume the speaker is simply “transmitting” information. One of the main underlying issues here is the assumption of passivity in the students, and of a transmission model of communication that has long been critiqued by communication theorists. Another is the generalization about faculty approach, as if those doing the speaking aren’t working to make their presentation engaging and responsive—and as if there’s nothing but lecturing going on in a course."

This is an important point, especially when it comes to large enrollment courses.  The bad lecturer is the straw man invoked by the MOOC Incs and others who want to "scale up" classrooms in the thousands.  The argument goes roughly as follows: "If most classroom-based, large enrollment classes are terrible--because, according to our caricature of the bad professor, they must be--then we can at least select out the "best" lecturers (whatever that means), put them on video, and stream them over the internet."  So we take something that we have deemed bad, claim that it is bad because of the professor's incompetence rather than the context (one person trying to teach hundreds of students), and then purport to solve the problem by finding competent professors.  This logic completely elides the fact that, as research has demonstrated, students don't learn more or better from good lecturers (in fact, they sometimes learn less well because they have a false sense of confidence/mastery).  It ignores the fact that teachers don't create learning, we facilitate it.  The failings of the lecture class are, for the most part, a consequence of the large numbers.  Increasing those numbers exponentially is very unlikely to fix the problems and far more likely to exacerbate them.

Perhaps even more worse, though, is the damage that this "myth of the bad professor" does to faculty morale.  It encourages the general public, including students, to view professors as disinvested  narcissists who are a drain on taxpayer money.  Far from being a noble vocation, teaching at a public university leaves one open to the lampoons of politicians eager to justify their disinvestment in higher education.  In reality, professors (as well as K-12 teachers) are doing more with less, year after year.  They are working harder than ever, largely because they genuinely care about their students and believe that education is a social good.  In the meantime, legislators and even institutional administrators hang on to the myth of the bad teacher in order to justify a shift to very large-scale, lecture-based education using the so-called best professors.

If we can leave behind this myth of the bad teacher; if we can recognize and acknowledge all of the creative, engaged teaching that is happening at campuses large and small around the country, we might be able to begin the challenging work of brainstorming different solutions to the "large enrollment, lecture-based courses don't work very well to create learning" problem.  Specifically, in large enrollment classes, we might be able to experiment with ways to exploit ed tech to minimize the structural problems of the environment; through careful assessment and redesign, we might be able to find alternative models that work very well to teach larger numbers of students effectively.  To do this, though, will require the direction of money and technical support to campus-based teaching.  It will require recognizing that the teaching faculty are a university's greatest asset.  Furthermore, they are an asset that deserves to be respected and supported, not universally caricatured as incompetent, disinterested in undergraduate teaching, and easily replaced with videotaped (or live-streamed) lectures of select Super Professors from around the country (and world).

*************
If this post resonated with you, please also read Melonie Fullick's excellent post on the same topic (written back in October 2012).  In her wise words: " Do we even have a way of “measuring” student learning to show what works? The context of all this is not a neutral one, it’s not just “let’s improve learning by finding out what will help the most”. The loss of resources including government funding has created serious material pressures. The urgency of the rhetoric about “change” should also remind us that there will be winners and losers in the education game, and futurologists have stakes in predicting something that could become a self-fulfilling prophecy—because prediction inspires present action."  Exactly.  And, as Fullick points out, much of this is driven by the basic assumption that on the ground university teaching is of poor quality, but with no data to support such a claim.
**************
More on the connection between the myth of the bad professor and the replacement of professors by large-scale modes of course delivery in this post by The Homeless Adjunct.

The Starving Children in Africa Argument

Several days ago I tweeted that I was filled with rage at reading the Christensen Institute white paper on hybrid innovation.  I was a little surprised when, later that day, I noticed that someone had responded that they found the report heartening.  If the conversation had stopped there, I think I would have just assumed this was one of those "costs of playing on social media" moments.  Unfortunately, this person then suggested that I don't have any understanding of the state of education in Third World countries (I do, actually); attacked me for being elitist; and told me to come out of my ivory tower.  I was deeply offended and angry, as much because anyone who actually knows me know how unfair such accusations are.  At the same time, it started me thinking about something that I think is important: what is my responsibility as a professor at the University of Texas, Austin but also as a global citizen?  Where do my own priorities lie?

I've heard variations of the argument that MOOCs do and will dramatically increase accessibility to education, in the US (in places like California) as well as in Third World countries.  I have issues with this argument.  Certainly, inaccessibility is a real thing.  Still, I'm not sure that it's a good thing to make accessible something that, so far as we know, does nothing to actually improve learning on any kind of large scale.  I fear that, in fact, MOOCs might make the situation worse because it will *seem* like there is learning to be had when there isn't, really.  I'd also argue that it is a kind of bait and switch game: tell people that education is the game-changer, that they live in a meritocracy.  But then, even armed with significant learning, they find out that it's really about networks, connections, and other intangibles.  In some ways, this moment reminds me of evolution of graduate education.  It used to be that it was largely inaccessible to all but the most well-connected and wealthy.  Then it became much more accessible.  Suddenly the job market became impossible and the myth of academia as a meritocracy has been pretty soundly battered.  All to say, I'm not at all convinced that making learning more accessible is going to do a whole lot except raise the bar and put those invisible networks even more firmly in play for those who can afford to pay for access to real, live professors and a residential college education.

But back to my priorities....  I am tremendously concerned about the state of access to education around the country and world.  But, most of all, I am concerned about these issues in my own backyard of Texas.  MOOCs are wonderful for outreach and continuing education; their value as any kind of enhancement to a college education are far from clear.  I am deeply concerned that, in the name of outreach and continuing education (not to mention, making superstars of some of their faculty), "the best universities" will continue to produce MOOCs that threaten my own students' access to a high quality, meaningful education.  Accessibility is important, but so is maintaining the high quality of the education our public university students are receiving.  If we do not protect that, if we slide too far down the slope, we will find that degrees from our institutions are utterly unable to compete with degrees from any private institution.  That's already something of a concern.  With the hype around MOOCs, their recent incursion into several public universities, and other kinds of "scaled up" teaching experiments going on, it's a real possibility that we will permanently damage our "brand." 

It makes me terribly sad that children in Africa are starving for learning; but, for now, my responsibility is to fight for the interests of my students--current and future--in Texas.

Orienting Students to the Flipped Class: What Worked for Me

I have only seen very preliminary data about the student surveys we did in my flipped Rome class at the end of the spring semester.  One decision we had to make before we even administered the surveys was whether we wanted to include the descriptor "flipped class" in the section that asked for the students' reactions to their experiences in the class.  After some internal debate, I decided to go ahead and include the term, but also a short definition of the term ("flipped class, that is, a class that includes acquiring knowledge of some content outside of class and practicing course content in active ways during class time").  I knew that it would confuse my students because they had never heard their class described this way.  I didn't realize that the very first question would be "Did you know you were in a flipped class?"  Oops.  Apparently this generated a fair bit of discussion on this class FB group, wondering what was going on!  However, the semester was over; they had done well; and I think they realized that it didn't really matter what label someone put on their experience (in fact, part of why I agreed to use the more jargony term was precisely so that, if they took another flipped class, they would understand that it would be similar to what they had already done).  Ultimately, in order to be able to make good comparisons to the fall cohort, it made sense to ask the spring cohort the same set of questions.

I'm sure I'll have more particular things to say when I see the full range of feedback from the students on their experience in the class.  Interestingly, though, even though they couldn't have known they were in a flipped class, only slightly fewer of them indicated that this was the case than in the fall cohort (when we repeatedly talked about what a flipped class was, used the terminology, and even generated enough resentment for a group of students to start a "I hate the flipped class" thread!)  Additionally, the average ranking for every aspect of the flipped class experience was higher for the spring cohort than the fall, again despite the fact that they had never heard their class described as such.

I'm told that, on the whole, they had very little to say about the questions related to the flipped class aspects (outside of class videos, i>clickers, peer discussion, etc.) and spent most of their time responding to questions about the ethics component of the course.  In part, this was likely because many of them were finalizing their ethics portfolios around the time they were completing the survey.  But I suspect that there's another, more important force at work: the flipped class was completely normalized for them.  Because I never completely flipped it; because I eased into the flip gradually over the semester; because I always kept a bit of the "sage of the stage" in place even while putting a lot of the onus on them to keep the conversation going during class, it didn't strike them as particularly unusual.  If anything, they liked the fact that it emphasized active learning, that I regularly involved them in discussion with each other and as a group despite the rather large size.  Whereas the fall cohort had pretty strong feelings about the flipped class model--con and pro--the spring cohort treated the class structure as something unremarkable.

If, as I think is the case for post-secondary, large enrollment lecture courses (and, likely, even many smaller seminars), it is actually counter-productive to make a production of the course design to the students; if, it turns out, student buy-in can be achieved in ways other than making them fully aware of the pedagogical designs at work, then what *are* the necessary components for a successful student experience?  One word: orientation.  The students must know what to do, how to do it, and why they are doing it (at least in part).  Instead of spending my second class day discussing the flipped class with 400 students, I focused on orientating the students to the course; the teaching team; and the tools that we would be using during the semester (e.g. quizzes, worksheets, videos, discussion board).  I talked about each activity we would be doing, how it would work, and why it was a part of the class.  I discussed "best practices" for watching the pre-recorded lectures: take notes, shut down all other windows, sit at a desk.  I explained what peer discussion was and why they learned better and retained more information when they had to explain it to one another. 

Over the first three class sessions, I rolled out the class tools.  First I got them oriented to how a typical class would run and the sorts of things I expected from them, including classroom etiquette.  Then the TAs and I modeled an online discussion.  Only then did I have them try it--and on the first discussion, when a couple of students "trolled" other students, we jumped in immediately with rebukes.  For each activity the students did, they got substantial feedback from me or a member of the teaching (e.g. on the ethics worksheets, which we collected and my TA commented on).  Finally, the teaching team and I put clear boundaries in place: we were happy to help them find an answer to their question but expected them to be self-sufficient.  By the time they took the first midterm exam, it was very clear that the students knew what to do and what was expected of them.  It was not at all a surprise that, throughout the semester, the spring cohort far outpaced the fall cohort on midterm exams and final course grades.

In the fall, it was clear to me that a major source of student anxiety was simply disorientation caused by the disconnect of being in a massive auditorium yet not being the passive recipients of lecture.  Even students who did well, who followed instructions, reported feeling a sense of disorientation.  As an instructor, I was shocked at the amount of helplessness I saw.  Far from being empowered students, the flipped large-enrollment class seemed to cause many of them to be even less able to be independent, self-sufficient problem-solvers. It wasn't the flip per se, but rather, that the flip was happening in a physical space that many of them associated with an entirely different kind of learning.

Humanities professors like to say that we've been flipping classes since antiquity.  Well....yes and no. It's nevertheless true that humanities classes often involve the assignment of some kind of reading that is then reviewed or discussed in class. This works pretty well in graduate seminars and moderately well in upper division undergraduate courses. It doesn't really work at all in a large enrollment course (by which, for the current point I am making, I mean any course in which not every student can speak during a given class).  Once students have a bit of anonymity, they don't see the point of doing assigned work.  When students don't do assigned readings, class discussion grinds to a standstill.  Professors eventually give up and resort to spending most of class delivering content, with pauses for some discussion of points they have raised (the good ones do, at any rate; others will just lecture for the class period while the students dutifully scribble down notes....or, more likely, demand the PPT slides and study from those).  

The issue, as all faculty know, is one of class size (which is why there is such tremendous irony in the embrace of MASSIVEoocs as the solution to all that ails public higher education).  With a small enough class, we can hold students accountable--and they know that and so will often prepare assigned readings.  In large enrollment classes, on the other hand, it is nearly impossible to hold individual students accountable for thoughtful, daily preparation.  In such courses, in fact, students often take the view that they will EITHER come to the class OR do the readings, but not both (this view was clearly expressed in student surveys from the Fall 2012 version of the flipped Rome class).  Thus, it requires a significant shift in student expectations and habits to get them to, as it were, do the flip.  By relying on experiential rather than conceptual learning--in other words, by asking them to do something rather than understand exactly why they should want to do something--I seem to have found a way to get a class of 400 to make this shift. 

I can imagine a time when the flipped/hybrid model is more widely practiced in large enrollment classes at universities, when it would make sense to announce at the start of the class that the course will use that course design.  For now, however, that information means nothing to my students.  The only effect it seems to have is negative. It stirs up anxieties and causes students--to whom I am a complete stranger teaching content that is completely foreign to them--to be overly worried about their grades, about being subjects in a mad professor's teaching experiment, and worse. So, for now, I'll remain a stealth flipper.

Orienting Students to the Flipped Class: What NOT to do?

 

The research on implementing the flipped class model is in strong agreement that it is essential to explain the concept of the flipped class to students.  It is argued that, in explaining to the students the different pedagogical approach, they will be more likely to understand their own role in the teaching/learning dyad and give their "buy in".  Since the flipped class model has been especially embraced in K-12 education; and by math and science teachers, the research reflects the particular experiences and contexts of these teachers.

When I decided to flip my large enrollment Intro to Ancient Rome class, I read as much as I could about the flipped class model, in an effort to discern a set of "best practices" that I could use for my own implementation.  It was difficult to find any research that was directly applicable to my course, in large part because the research focuses almost exclusive on small class; on problem-based classes; and primarily at the high-school level.  Still, it made sense to me to spend some time introducing the flipped class model to my students and explaining to them how their learning experience would differ from a traditional lecture class learning experience.  I would tell them how much better this more active model was for their learning (sort of like those tablespoons of cod liver oil my grandmother forced my mother to take every day); and they would eagerly embrace the opportunity to be fully participatory agents of their learning.  It turned out that this was exactly the wrong approach for my particular group of students (I've written about this here).

In retrospect, I have a pretty good idea of why the orientation to the flipped class has to take a rather different form for college-aged (and older) students, especially for a course that is being taken almost exclusively by non-majors to fulfill a core requirement.  And especially in a state where nearly every student got to the University of Texas because they were in the top 10% of their graduating high school class and have internalized extreme grade anxiety.  Any whiff of change or experimentation raises hackles....and worse.  As well, especially for female professors, it's probably not a good idea to tell a room full of 400 undergraduates that you are trying a new pedagogical method.  They tend to assume that you are doing this because you don't know how to teach, and that you'd very much benefit from their advice.