Scaling Up

Teaching a large enrollment class (250+ students), whether in a classroom or online, is not unlike herding cats.  Actually, based on my considerable personal experience in both activities, I'd choose herding cats over the challenges of managing the logistics of a large enrollment class any day.  Shake a bag of treats and the cats come running.  Large classes, well, they take quite a bit more effort to prevent a decline into total chaos. 

Earlier this week, I watched e-Literate TV's latest installment on personalized learning at Arizona State University with quite a lot of interest. While I recognize the deeply concerning labor issues that underpin the ASU model, and hope that over time a more just staffing model is implemented, especially with regard to non-tenure track faculty; I also find Michael Crow's efforts to create the New American (Public) University compelling (see this condensed interview in the New York Times about his plans and motivations).  There's no question that the public university, even flagships like my own, are undergoing slow but fairly extreme transformations.  I like a lot of what Crow is doing, not least of which is emphasizing broad access without losing sight of the need to provide intensive support for many of these students who would not otherwise have had access to a four-year university.  I admire the clarity of his approach and the creative solutions he has found to create new revenue streams in the face of ever-shrinking state appropriations.

He is operating under extremely difficult circumstances, but takes that as a challenge.  He has a "take no prisoners" mentality, and he has the decided advantage of being at least five years further along in the process of evolution than just about any other institution.  He was already taking the first steps to re-imagine and re-invent the American (Public) University while most public universities (and their leadership) were unaware of how rapidly the landscape of higher education was shifting--and would continue to shift--even as the economy has begun to recover.  Many academics are critical of his vision.  For his part, Crow seems oblivious to his critics and powers ahead with his plans.  It seems that, about once a month, a new partnership or initiative is announced by ASU.

Crow has hired some very smart and experienced people to help him realize his transformative vision for ASU (and the people of Arizona).  In watching Phil Hill interview Adrien Sannier and others to capture ASU's approach to personalized learning, I was deeply impressed by several things.  But what stood out the most was the rejection of the traditional "small pilot" model of course development.  Phil Hill nicely captures Timothy Harfield's rejection of the standard pilot model--and his own observations about the general failure of this model to lead to sustaining innovations.  Typically, experimental course designs are piloted to small numbers of students, with the aim of identifying problems before "scaling up."  At ASU, when it came to implementing their remedial math course (and, I gather, other similar courses), they skipped the piloting stage and jumped straight to scale.  This is exactly the way to do it.  [But see Matt Reed's thoughtful response in Inside Higher Education, including his important point that--in some circumstances--it does make sense to start small, especially when taking on complicated projects that involve collaboration across several different parts of the institution.]

By accident, my own course development projects have both followed this same pattern.  The first time, when I flipped my Introduction to Ancient Rome course, I had planned to teach it to 200 students; but ended up with an enrollment of 400 students when my university needed more seats in introductory level courses.  I had no idea what I was getting into, and the extra 200 students added about 5 times as much work and complexity to the logistics of the course.  While it was, without question, a very challenging semester, the advantages of going straight to scale were quickly apparent: every problem became evident very quickly; we were pushed to maximize efficiency in every aspect of the course; we received a tremendous amount of feedback on every part of the course.  I spent the winter break retooling the course based on the feedback from students and teaching assistants; and my own experience of teaching the course.  I taught the revised version in the spring semester.  It went off without a hitch.  [as an aside: one other important factor in course development projects is teaching the course at least 2-3 semester in a row, in order to make the adjustments while everything is still fresh in the memory.]  I can honestly say that the first iteration of the flipped version was one of the most difficult teaching experiences of my life; but, by skipping the pilot phase, I think we were able to create a very strong course very quickly.  It was a painful first iteration, but there was a big payoff in subsequent iterations.

When we went live with the asynchronous online version of Introduction to Ancient Rome in Fall 2014, we assumed that it would be a small pilot.  We had not advertised the course at all.  It wasn't even listed and open for registration until four days before classes started.  We guessed we might get 10-15 random students register for it.  In fact, we ended up with over 300 students.  Once again, I found myself in a challenging situation.  Not only were we still doing significant development work on the modules; but I was also training an instructor who was new to the world of online education (and who had never taught a class over the size of about 30 students).  Once again, though, we found that the large size of the class immediately highlighted the design flaws and encouraged us to figure out how to maximize efficiency in everything.  It was a stressful semester for me as well as the course instructor, not least because we were rushing to finish modules and release them to students in the course of the semester.  However, we learned everything we needed to know about the course's design flaws.

I spent this spring semester revising some parts of the overall course design (e.g. adding more structure, including a graded quiz at the end of each module); and then making revisions to the individual modules.  As with the flipped version of the course, I was able to avoid a drawn-out process of identifying and eliminating the design flaws by jumping straight to teaching it at scale.  In one semester, we figured out what parts of the course either didn't work at all or needed to be revised; and also identified the features that were crucial to the course's success (e.g. clear expectations; an extended orientation to the course; strong online presence of the instructor; integration of weekly f2f review sessions on campus).   In addition, because I'd had the experience of making this scary leap before, it was a lot easier to manage the chaos and remind myself that nobody was going to die.

In theory, pilots make a lot of sense.  It seems like a way to contain the inevitable problems of a new course. In reality, however, it doesn't actually make much sense to pilot courses that need to be scaled to small groups of students.  A small class and a large enrollment class are entirely different in their character and their challenges.  Many of the challenges of large enrollment courses are logistical and are a direct result of scale.  In order to identify and remediate them, the "pilot" of the course has to be run at scale.  It is a bit scary to take this approach, to be sure; but one quickly learns that students are resilient and reasonably forgiving of our learning curve.  Most of the problems are not devastating to student learning; they just get in the way.  When I taught the flipped class for the first time, the course evaluations were mixed--about half the students loved the class and half hated it.  But their hate was a result of the flipped model, not problems in course design.  Interestingly, the student evaluations on the first iteration of the online course were much more positive.  While they identified parts of the design that needed more attention, the responses were in general very positive.  Behind the scenes, it often felt like total chaos poised on the edge of disaster.  But we were pretty good at keeping all of that behind the scenes (including our own high stress levels).  As a result, the students had an overall good learning experience while also helping us figure out how to make the course better.

One of the most significant challenges for public universities in the coming decade will be figuring out how to support learning at scale.  In particular, we have to figure out how to design and implement online courses that can be taught at scale.  This is not to say that there is not an important place for smaller, seminar-style online classes--those are also going to be an essential part of any future curriculum at a public institution.  But at least some courses will need to be large-enrollment, probably asynchronous (though could be synchronous) in order to meet growing demand, especially as institutions increase the size of their student population.  So how do we teach at scale while supporting high levels of student learning?  F2f, large enrollment lecture classes are not the answer, as universities are slowly coming to realize. The next several years will probably see a shift from faculty delivering content in the classroom to faculty building online courses that have the capacity to personalize learning (if adequately and appropriately staffed).

ASU has a good model in place for teaching at scale--a model in which the roles of the instructor and TAs in particular are re-imagined.  One of the many positive things about teaching at scale is that, with a well-designed (and tested) course, it is much easier to identify the students who need our attention.  The majority of the students are actually very capable of working on their own (often with a group of peers).  They benefit from feedback on their work but don't require frequent interventions to keep them on schedule or help them understand content.  With an adequately staffed course and skilled teaching team (and, ideally, an easy to use dashboard), members of the teaching team can focus on those students who most need intensive mentoring to advance successfully through a course.

The keys to success in teaching at scale?  A good course and, most importantly, a skilled teaching team with clearly defined roles.  ASU has figured out that the success of ALL students and not just the top 30% requires a significant structure of support, from an instructor to graduate assistants to undergraduate mentors.  Too many people, including those in administrative positions, only pay attention to the first part of this equation--the quality of the course design--and don't realize the importance of the teaching team to student success.