Greek and Roman Mythology MOOC: Quiz #1

I finished watching the lectures, about 90 minutes in total, yesterday and attempted the first quiz.  The lectures were a lot of background information explaining what myth was for the ancients as well as some modern theorists; and then an introduction to the world of Homer and a close reading of the first 10 lines of Homer's Odyssey. I did not take notes on the lectures, in part because the information was either the sort that I can quickly look up or already knew.  As a professional classicist (albeit one who has never taught myth and whose research doesn't tend to touch much on aspects of mythology), I felt pretty confident.

I scored a 16.2/20 on the first quiz.  I was a little embarrassed.  I quickly realized that I should have taken notes on the lectures about ancient and modern views of myth.  There were several very detailed questions about that information that I struggled to recall.  Partly, I was a little surprised to be asked such detailed, factual questions that weren't really about my ability to think but purely recall data.  I don't tend to ask these sorts of questions so was unprepared for them.  A great reminder that it is important to prepare students for the way you are going to test them (in this case, we could retake the quiz 3 times with no penalty). 

Today I reviewed the two lectures with the details I had missed on the quiz and took notes. I also reviewed my first quiz.  I am pretty sure there was a problem with the quiz--or at least a very poorly worded question.  I missed it the first time but, when I retook the quiz, I gave the same answer the second time (because I am sure that it is the correct one), and still missed it.  So got a 19/20.  I retook the quiz a third time, just for fun, and had several new questions.  Most of them were the sort of details I know from my general work in classics.  But again, I was taken aback by a question about the size of Homer's army.  I am not sure what the correct answer is and I didn't remember it from the lecture.  It seemed like a nitpicky question and not a hugely important one.  It also didn't reflect what had actually been the focus of the lecture, which was Helen as the face that launched a thousand ships. 

I did learn some helpful things: a. having questions with multiple answers is a good idea (I will do this for my own exams, perhaps next fall); b. it is essential to think about what skills you are trying to encourage your students to develop.  After this quiz, students will be scribbling down every word because it is too difficult to tell what is and isn't going to show up on the quiz.  c. retaking quizzes with a randomizer is a fantastic idea and a very effective learning tool.  If I do end up designing an online class, I will definitely make good use of weekly quizzes like this.

Student Altruism in a Large Class Setting

One of the more surprising things about my flipped class is the extent of student altruism at play.  Perhaps it's just that it is more visible to me because of piazza and the indirect reports I hear about the class Facebook group.  I've had students create study guides for every pre-recorded lecture and distribute those.  They extracted the embedded review questions, put it into a single document, and posted it on google docs for everyone to complete and view.  Others made e-flashcards and made them available to the class.  Study groups abounded.  The persistent mantra on the Facebook page was something along the lines of "let's team up and all get As."  I don't grade on a curve so there's no incentive NOT to work with others.  Still, I've been really surprised at the displays of altruism.  I mean, I can understand small groups sharing work on study guides with one another; but why with the whole class?  I'm really interested to hear about other instructors' experiences with this, especially in a large class setting where students don't know everyone.  And interested to hear thoughts on the psychology of this (and whether there are studies of this phenomenon).

Managing a Team of Teaching Assistants

It's one thing to design a fantastic, student-centered course.  It's a whole other thing to make that course work in a real-time setting.  To make it work, at least for my large course, requires a team of teaching assistants.  For just under 400 students I was assigned 4 TAs.  Initially, I was given two TAs and I had to campaign vigorously for the "additional" two.  I found it deeply irritating that I was having to organize letter-writing campaigns to the deans in my college so that there would be enough classroom help to actually implement the student-centered class that I had worked so hard to create (and that the college had spent at least some money to support).  Sure, two TAs might have been sufficient when I was standing in front of the class lecturing and encouraging the students to be passive vessels; but the whole point of the redesign was for them to be active and engaged.  Beyond classroom logistics "stuff", I needed TAs to help me by walking around and dropping in on the "turn to your peer" discussions.  As well, the design called for a senior TA to lead a weekly review session of the material the students had viewed online.

In the end, I did get the team I had asked for: two relatively senior grad students who could take on a lot of responsibility; two first year grads who would be very helpful with classroom and other course logistics while they learned the ropes; and two student graders.  I made one of the senior TAs the "grading czar" and it has been his responsibility to manage all things related to grading, including overseeing the team of graders who are marking the short answer questions on the midterms (the two first year grads + the student graders).  The other senior TA was put in charge of the Friday review sessions.  I have worked intensively with her on how to prepare a student-centered review that consists primarily of i>clicker and "turn to your peer" instruction.  We have also worked on picking out the sorts of details that it will be helpful to emphasize; and how to choreograph and presentation so that it runs smoothly, doesn't have you jumping ahead of yourself, etc.

A ratio of 1 TA/100 students is an absolute minimum for a student-centered, flipped class; better would be 1/50.  The students are more engaged and therefore asking more questions; there are more moving pieces and many of these can be handled by TAs (especially after the first iteration of the class).  TAs are especially helpful in facilitating peer instruction.  In my classroom, because of the design of the room and the fact that there are so few extra seats, I have two of the TAs stand at the doors and help late-arriving students find seats.  That has minimized the distraction to me and the rest of the class and prevented students from plopping in the aisles.  On exam days, I recruited an extra grad student to help us hand out exams.  We put the scantrons into the exam so that we only needed to hand out one item.  We then divided and conquered the room.  Next time, I will have one person stand at the back and hand exams directly to students who are coming in after we start handing them out.  But, using this method, we were able to hand out 400 exams in about 5 minutes and everyone was able to start on time. 

I found out on Friday that the TA who was leading the Friday review sections was going to be transferred to another class with discussion sections (a medical emergency had left that course without a TA to do sections).  This was not welcome news, but was the best solution for my department as a whole.  It now means that I have to take on those review sessions in addition to everything else.  This TA did not do grading; but she was slated to grade part of the final exam with me.  I am working to ensure that she will be paid an extra stipend to do that.  This whole episode reminds me of another mantra associated with teaching, but especially teaching student-centered classes: remain flexible and adaptable at all times.

There was a fair amount of logistical work to do at the start of the course, as I figured out how to divide the different duties among the TAs.  Fortunately, I have a fantastic group--reliable and competent in every way.  They have been really good at helping me figure out solutions to course logistics and have done what was asked quickly and without complaint.  When teaching this sort of a class, especially for the first time, it is essential to have an excellent team of teaching assistants.  They are the sine qua non, and it is their contributions that make or break the experience for the students.

A Bee in my Bonnet: Why discipline matters in designing a flipped class

This will be the first post in a series I am going to call "A Bee in my Bonnet."  What I mean by this is that these are posts where I take up an issue and rant a bit. These rants aren't directed at anyone in particular, though they are often stimulated by conversations (and disagreements).   Mostly, this is me thinking out loud about my disagreements with blended learning orthodoxy.  At the moment, most of the bees swirling around in my bonnet relate to ways in which I'm finding that a. flipping an intro-level humanities class works differently from a chemistry or calculus or statistics class; b. even within humanities, there are distinctions to be made. Intro  English has very different objectives from courses in fields that students generally don't study in any depth in middle school or high school (e.g. economics or classics); and c. those who are teaching the teachers (i.e. instructional design and instructional technology folks) would do well to be open to the fact that the current research, while helpful, doesn't always work very well for a class like mine.  And, if the aim is to encourage other faculty like me (i.e. faculty teaching courses that have some emphasis on delivery of declarative knowledge) to undertake blended learning, it will be helpful to be attunded to disciplinary differences, differences in student audience depending on the course and its particular function in a university curriculum, etc.  I am delighted with the preliminary results of my flipped course; but am also learning that the "best practices" I read about don't always make a lot of sense or even apply at all to the course I am teaching.
**************************

I am quickly learning that, for all that blended learning and flipped classes are a commonplace of conversations and faculty workshops at campuses around the country, we are still very much on a pedagogical frontier.  Our understanding of how this model works and, perhaps more importantly, why it works, is very much a work in progress.  As well, most of the preliminary research that is being used to create "best practices" guidelines for faculty interested in blending or flipping their classes comes out of the natural sciences/math/computer science disciplines.  Courses on logic taught out of philosophy departments are the rare instance of a liberal arts discipline using blended learning successfully.  All of these types of classes have something in common: they are concept based.  Teach a concept (how to calculate mass) and have students practice the application of that concept.  Concepts are reasonably discrete or, when they are cumulative, it is in a very logical, additive process.

But what about courses like my Introduction to Ancient Rome, where one of the explicit learning objectives is to master a body of declarative knowledge (a basic narrative of Roman history from 1000 BC-476 AD)?  It is all well and good for people to do seminars, write blog posts or journal articles, and record youtube videos heralding the death of the lecture.  That probably is and should be true for courses that don't require instructors to first teach students the basic facts of the topic of the course.  So, for instance, nearly every (all?) students taking Intro Chemistry at a university have had some chemistry before.  They might not have learned much, but they at least know what the field is about.  Likewise with English or math.  The same cannot be said for teaching students a dead language like Latin or Greek. Or teaching a course on the history and culture of ancient Rome.  For the most part, my students know where Rome is but they know very little else.  This semester, I did a content pre-test that used items from previous exams (so they were tested and we knew something about the likelihood that students would know the answer by the end of the semester).  These items were just basic facts.  The highest score was 50% and most students got 4-5 out of 24 correct.  In other words, about what you'd expect from pure guessing on every item.

I can't start out the semester having them apply or practice knowledge they don't have.  As well, the nature of my material is such that it is not well-suited to 3-5 minute mini-lectures.  It's not a series of concepts that come together to form the course; it's the story of a culture, with all sorts of interwoven complexities.  So I had a couple of big challenges when I decided to flip my Rome class.  First, how do I teach them basic declarative content without lecturing.  Can I assume that they can read the textbook, digest it, know the significance of various events without being told, etc.?  Well, no.  So I need to do that for them.  But I can't do it in 3-5 minute videos--that would be jarring and would never teach them enough unless they watched 20 of them for each week.  I agree that the 50 minute lecture is too long and unnecessary (though, ironically, many of the webinars telling us this are doing so in a 50-60 minute lecture). 

I decided to do 20-25 minute lectures, with a clear break in the middle for review (I have a slide with review questions and tell them to press pause).  I also remind them that they can always pause the lecture, just as they do when watching a movie on Netflix or a TV show on their DVR.  Nobody will require them to sit still for the entire 25 minutes.  But 20-25 minutes was a good way to "chunk" my material--not too long, not too short (and it could always be subdivided by the viewer).  That is the time it takes for me to present a concept and explain it.  My material is less dense in its initial presentation than a mathematical formula, though, which is why students will watch 25 minutes of a Roman history lecture but probably not 25 minutes of someone lecturing about math or chemistry or theoretical physics.

I absolutely agree that the lecture should not keep the same slide up for more than 5 minutes.  Slides should have an image as well as words, and the words should be minimal.  It helps the students focus if the slides change about every 3 minutes.  Faster than that might be too fast for most students, but much longer and they start to zone out/want to press fast-forward.  The trick is to find a pace that is steady, swift without being frenetic, and variable enough to keep the students focused.  It is very possible to do this with longer lectures.  I would not recommend a lecture longer than 30 minutes, however, and feel like somewhere between 20-25 minutes is the golden mean for a class with my content and constraints.  It is important to speak quickly (you will feel like you are chattering a mile a minute but it will sound completely normal); to have elegant and visually interesting slides (but not excessive amounts of animation); and, in my view, to move as little as possible.  The animation and charisma has to come from your voice, not from pacing around the video frame.  Your students may be able to tolerate your pacing around your classroom (I do this too) but on a screen it looks awkward and can be vomit-inducing.

What way forward?  The most important thing, particularly for learning specialists in campus centers for teaching and learning, is to keep in mind that there are no hard and fast rules.  It really does depend a lot on the course content and student audience.  Certainly, with so much research now on flipped classes in the natural sciences, especially physics but also chemistry, we know a lot more about "best practices" for those disciplines.  But it's still a very new frontier for liberal arts and our subject matter is rather different from the natural sciences (not least because, in the introductory level courses that are most likely to be candidates for flipping, it is not problem-based).  Furthermore, one humanities class can't stand in as the model for all of them.  The key thing as we all venture out into the wilderness will be to have a good knowledge of what has worked in individual situations (and why); have flexible ideas about "best practices"; and, most of all, to think hard about what is going to work best for a particular class and its particular mix of students (and level of student experience with the subject matter).  Discipline matters!

BlendKit2012: Week 1

I am participating in BlendKit2012, an online course about blended learning.  It is my plan to write a weekly blog post in response to issues or questions raised by the readings for the course.  This week, the readings focused on some of the big questions that underlie the redesign of a course to incorporate elements of blended learning.  I wanted to respond to two of the "points to ponder" in detail.

"Is it most helpful to think of blended learning as an online enhancement to a face-to-face learning environment, a face-to-face enhancement to an online learning environment, or as something else entirely?"

Before I started teaching my blended (actually, flipped) class, I'd have answered the first option, an online enhancement to a f2f learning environment.  Certainly, I embarked on the redesign of my traditional lecture-format course with the thought that I would move some/all of the content delivery to outside of class so that we could devote class time to learning and applying the basics of Systematic Moral Analysis to case studies from roman history.  Now that the class is underway and we are 1/3 of the way into the semester, it is clear to me that, while the class is combining f2f learning with online learning, the end result is something entirely new.  I feel a bit like a chemist in her laboratory, mixing beakers full of chemicals and expecting a certain reaction to happen when, in fact, an entirely unexpected reaction happens.  Something I've been reminded of: students are not predictable creatures (or at least, not entirely predictable); and group dynamics are very hard to explain and predict.  I have been really surprised to see all the ways that changes in my behavior as an instructor have resulted in some predictable and many unpredictable changes in student behavior.  It is clear to me that we are still very much on the frontiers with blended learning and that there is a great need for additional research, more data, before we reach firm conclusions about what it is and how it works.

"As you consider designing a blended learning course, what course components are you open to implementing differently than you have in the past? How will you decide which components will occur online and which will take place face-to-face? How will you manage the relationship between these two modalities?"

I went about my redesign in a pretty straightforward way.  I figured out what my objective was, beyond getting them to master the course content, and then thought about what parts could be done without me.  So, first, I decided that my main aim was to reinforce with every element of the course a basic idea: the mastery of course material is about student learning and not instructor teaching.  I needed them to take responsibility for their learning, to become self-sufficient in their use of the tools for learning that I provided (textbook readings, pre-recorded lectures digesting textbook readings, piazza discussion board, Friday reviews of recorded lecture material, in class discussions, i>clicker questions, etc.), and to view me as an experienced guide rather than a fount of all wisdom who would magically fill their minds with knowledge.

When I determined which components of the course to move outside of class (e.g. pure content delivery, announcements about course logistics) and which to retain, I asked myself one question: for what parts do they need me?  What parts can they reasonably do on their own, especially when they can post questions to piazza?  How do I make sure that at least 80% of class time is spent with them talking, clicking on questions, and otherwise being actively engaged?  I have been fairly ruthless about finding ways to do everything apart from i>clicker and peer instruction outside of class.  When I hand back their first midterms, for example, I will direct them to watch a short video of me reviewing the exam, talking about trouble spots, telling them the class grade distribution, etc. If they have questions or want to review their exam, they will be directed to the TA who oversees the grading team.  I refuse to waste 15-20 minutes of a 45 minute class period conveying information to them that can be conveyed in other ways.  By being so ruthless about preserving class time for student engagement, I am able to preserve the integrity of our classroom as a place for them to talk, not me.  They seem to like this.

Piazza.com

UT uses Blackboard as its LMS (Learning Management System), though they are currently piloting Canvas as an alternative.  Many faculty are not fans of Blackboard but I don't really mind it.  It does what I need it to do--store course documents and make them relatively easy to find; provide an e-gradebook and calculate grades.  I have never used its discussion board features, though I've heard from both students and instructors that they are clumsy.  Early this fall, just before the first review session of some content they had watched at home, I opened a thread on the discussion board for them to post questions for the TA leading the session.  Immediately, a student emailed me asking if I would create a class site on piazza.com.

I have tried to approach every aspect of this course with an open mind and so figured that I should see what this piazza thing was.  I browsed around and decided that it couldn't hurt anything and so created a site.  It took about 5 minutes (though I now know I should have imported the class roster myself rather than letting piazza do it).  The site itself is pretty simple and intuitive: students post questions which they can tag (thereby organizing posts and making it easier to retrieve posts on a certain topic).  Other students or instructors can answer the question.  I'm not sure why--perhaps the mere fact that it isn't a UT LMS--seems to have encouraged the students to feel some ownership of the class site and to participate pretty energetically.  I get an email when a student has posted something (I've set it to be a two-hour digest).  Typically, by the time I get to the question, at least one other person has answered it and often done a more thorough job than I would have (e.g. pointed the student to page # of textbook where topic is discussed or provided links to external sites).

Students certainly use it to ask basic questions that are, in fact, covered in the readings and recorded lectures.  They use it to ask questions about course logistics (e.g. when will attendance grades be posted?  when will we know our exam grades?).  They also use it to ask questions that the readings and lectures provoke but do not address.  That is, they use it to sate their curiosity or to extend a discussion.  At times, I have used it to extend a conversation that started in class or to raise a related topic.  I frequently get several takers who are willing to contribute their thoughts.

My absolutely favorite part about piazza?  By encouraging students to post there (they can do so anonymously if they wish), my email traffic for the course has dried up almost entirely.  I answer questions once rather than 34 times.  I can answer their questions quickly, while they are still fresh and I can do it in a few words.  It takes much less time to answer 4-5 questions on piazza than to answer 1-2 emails.

Preliminary Feedback

I find that one of the biggest challenges in teaching a large class full of students I can;t really know (there are 400 of them!) and don't interact with by name in every class period is the absence of feedback.  In general, I am quite good at reading a room and making adjustments.  Yet, for the most part, I can't do that in a large class. I am still pretty good at taking the temperature of a room's vibes and I like to think I know when something is going really well or really poorly.  Where I struggle is in the middle--when they aren't furious but also just putting in their time.   I was especially stressed about the feedback problem with the flipped class.  Other faculty and administrators as well as the research warned that students don't like flipped classes: they resist; they grumble about the perceived increase in workload; and they give the instructor and the course poor evaluations.  I tried to anticipate some of these objections at the start of the semester by preparing a short video I titled "Teaching and Learning."  In it, I did a brief overview of the the origins of the lecture model and gave an introduction to the flipped model, the reasons for it, and its objectives.  I also reviewed lists of pros and cons.  In other words, I tried to anticipate and address their objections before they could ever take shape in their heads.

We are now 1/3 of the way into the semester and, so far as I can tell, there is no palpable resistance and certainly no more grumbling about the workload than in previous semesters (which is to say, it is minimal despite the fact that it has indeed increased a bit, perhaps by 15-20%).  Honestly, I'm a little surprised.  I worked hard to get the students to buy into the design of the course; I made clear to them why they would be doing everything that we are doing, and why each part was inside or outside of the classroom space.  Still, I expected more resistance (fortunately, faculty are ever willing to provide the resistance to pedagogical innovation).  Instead, what I've found from the students is an eagerness to learn about ancient Rome (I had 111/387 students watch the videos for the next week in the 48 hours after the midterm exam; 250 of them showed up for a voluntary review of video content today.  I am pretty sure *I* wouldn't have showed up if I were a student!).

With the first midterm exam earlier this week, it was a stressful week: the exam had to be written, proofed, all the logistics for its administration and grading worked out.  My brain hurts.  Yet I also feel energized by this amazing group of students and their response to the flipped classroom.  On Wednesday night, mere hours after the midterm exam (and long before grades will be posted), I received an email from one of my students.  In it s/he wrote,  "Just wanted to let you know that i thought i would never be able to remember this kind of information (ancient rome stuff) in this amount of time. It is thanks to the echo lectures, all of the iclicker Q's, piazza, and of course your lectures as well as the TA reviews on fridays. Keep up the GREAT work, i'm really enjoying the class and the discussions that we have! :)"  I am sure I speak for all teachers when I say that there is nothing more rewarding than knowing we were able to make a difference for a student.  I was especially pleased to see the pleasure of mastery that I was able to help this student experience.

This afternoon, while waiting outside of my classroom for the previous class to finish, I had a conversation with a different student.  I asked him how the class was going for him and he launched into an energized rave about how much he enjoyed the fact that he got to apply what he was learning and that it wasn't just about memorizing a bunch of facts.  At that moment it hit me full force that a huge reason for the noticeable increase in student motivation to learn the "facts" part of the course (i.e. the content delivery that I pre-recorded) is because the course gives them the opportunity to do something with that knowledge in the form of evaluating the justifications for different literary and historical characters' ethical choices.  They have clearly recognized that they can't evaluate these case studies if they don't know the facts.  At the same time, they know that there's a point to learning those facts.  In earlier versions of the class, I failed to provide intrinsic motivations for learning the course material.  Sure, there were regular exams that served as extrinsic sources of motivation; but my students never really got to experience the kind of excitement and satisfaction that comes from higher-order, critical thinking.

This student highlighted for me just how essential that is (and yet how difficult in a class that requires the instructor to ground the students enough in the facts for that high order thinking to be in any way meaningful).  I have been delighted to see how willing my students have been to accept that content delivery had been moved out of the classroom space.  Before this conversation, though, I didn't quite know what that was.  I get it now: they'd rather talk to each about the intricacies of our Roman history case studies than listen to me regurgitate the textbook readings.  Sure, most of them need that regurgitation for at least parts of the content; but they sense that what they are paying for and what they are getting out of bed and coming to the space of our classroom for is the chance to talk and interact with the material.  In my conversation with this student, I seized the opportunity to observe that we were able to do the application of facts because of the pre-recorded lectures and made a point of sharing how much fun it was for me to teach the class this way.

I still feel like I won't really know what "the room" thinks until the end, when all the votes are tallied.  Still, it's been great to hear such good things from a few of them.  In my experience, students will be quick to tell us what they don't like but they aren't always so quick to tell us what they do like.  I'm pleased that these two took the time to reinforce for me my sense that, as a class, my students were on board and enjoying the ride thus far.

Going Back to School

One of my favorite modes of procrastination is to go on Coursera and browse all the different free classes I can take.  I've even registered for a few, including a course on Greek and Roman mythology taught by Dr. Peter Struck.  Penn hired Peter when I was a senior graduate student and he was a member of my dissertation committee.  He's also an award-winning instructor, prize-winning author, and all-around smart guy.  I was intrigued to see what the experience of a MOOC would be like from the point of view of a student, and thought this was a good place to get started.  I'm also taking BlendKit 2012, an online class about blended learning.  BlendKit 2012 is a smaller group, looks to be more instructional design folks than professors, and is more likely to be people of relatively similar knowledge levels.  Because I teach during the weekly, live webinar, I'm what they are calling an "asynchronous" participant.

Some first impressions from my return to the "classroom":

a. the quality of filming for the mythology MOOC is very high quality.  Unlike many free online courses (e.g. Open Yale Courses) these lectures are filmed explicitly for the MOOC audience.  The instructor is front and center; there are no random backs of students' heads.  Perhaps most importantly, the framing of the shot creates the impression that I'm a part of things and not a voyeur lurking high up and far away from the instructor and the rest of the students.

b. in a typical class, the instructor makes a point of underscoring his/her accessibility and availability to the students.  We encourage them to connect with us, particularly in larger classes where they might not otherwise feel a connection to us.  In the MOOC, it is all about repeating that students should not expect contact from the instructor/instructional team because there are so many students (50,000 in this particular course).  On the course syllabus and in the introductory lecture, it is made clear that the instructor's job is to deliver content.  Everything else connected to the course will be done by computer (quizzes) or peer instruction (papers, discussion).  On the one hand, I absolutely understand the need to make this clear.  On the other, I found it weirdly off-putting.  I'm not used to my instructors telling me not to bother them.  I'm wondering how effective peer-instruction will be when the students don't know each other in person and, having done a lot of peer review of papers in my courses, I'm pretty skeptical about the ability of students to grade other student papers.  I'm willing to be shown that my concerns are misplaced, however.

c. The MOOC is *a lot* of work.  Several (5-10) lectures that are 15-20 minutes long + substantial readings of difficult ancient texts.  It seems to pretty much Dr. Struck's regular Penn mythology class broken down into shorter but more frequent lectures.   His Penn course may include more theory (I'm not sure) and obviously has exams, but otherwise the MOOC course seems pretty similar.  As the course goes on, I'll be interested to see the level at which the lectures are pitched.  So far, it seems pretty high.  I'll also be interested to see what the completion rate is, especially relative to other Coursera courses; and the demography of the people who finish the class.  I am going to try to be one of them.

d. In the BlendKit introductory webinar, I was struck by how many of the participants were instructional design folks.  It is interesting to see the extent to which blended learning is still such a new frontier at colleges and universities, but also the extent to which the teaching specialists are working to become informed advocates of its principles.  The sort of networking that BlendKit 2012 is facilitating is crucial to the wider implementation of blended learning techniques and to the development of "best practices".

e. I had to giggle at one aspect of the BlendKit course: badges.  It seems that people never outgrow their need to have some sort of outward mark of their accomplishments!  I am knee-deep in implementing my own blended learning course and mostly interested in lurking, doing readings when I can, and just hearing what people have to say.  I will try to participate in discussions when I can, but I'm not sure how often that will be.  I'm pretty sure I won't meet the requirements for my weekly badge and I'm ok with that.  But I gather that collecting badges and certificates is going to be the Next Big Thing.  Alas.  [a great post on the disruptive power of badges: http://hastac.org/blogs/slgrant/2012/10/05/questions-about-badges-higher-ed#comment-20505]

An Exercise in Peer Instruction

I will confess that, before this semester, I had a lot of doubts about the utility of peer instruction.  Like many faculty, I've used small group discussion in smaller classes pretty regularly; but often, it seemed more an opportunity for students to lose rather than gain focus on the course material.  Still, I was willing to give peer instruction a go, particularly because it is the only kind of discussion possible in a 400-student class.  In addition, I felt that I had enough training from learning specialists to find ways to make it more meaningful.  Peer networks are at the heart of MOOCs and there's some good evidence to indicate that students can do a very effective job of teaching one another.  They may not have the expertise that the instructor has; but once the content has been delivered by the expert, peer instruction is very effective in getting that content to stick and encouraging active engagement with and curiosity about that content.

On Monday in the Rome class, the students had been asked to watch a recorded lecture covering the 1st and 2nd Punic War (up to the Battle at Cannae).  This is complex material and, since they have an exam on Wednesday that includes this material, I wanted to be sure that they got a chance to practice it and ask questions about it.  I prepared a 45 minute review consisting of i>clicker and "talk to your neighbor" questions.  Just before class, I got an email from the Echo guru at UT letting me know that about 25% of them had at least opened the lecture.  Of course, this information is not a perfect indication of how many were prepared: I don't know how carefully they watched the lecture; I don't know if groups of them watched it together; I don't know how many read the textbook but didn't watch the lecture.  Still, it is a good rough estimate of what I can expect when I walk into the class.  So I knew that about 25% of them had some idea about the material.

I started with a few i>clicker questions, some easy ones and some more difficult ones.  They were all intended to highlight "stuff you should be sure you understand."  The class was quite good at narrowing the answer down to two options but on several polls, it was a close vote.  In those cases, I had them break out into peer discussion for 1-2 minutes and then re-take the poll.  Inevitably, there was a substantial shift to the correct answer.  The students all oohed at this and I smiled at the power of peer instruction at work.  I also made a point of pointing out to them how effective it was to talk to their fellow students.

By the end of the class, we had reviewed all the key points of the lecture.  Students who had done no work prior to class were forced to engage actively at every moment.  I kept my direct comments about the material to a bare minimum and used the Socratic method as much as possible.  I also did everything I could to keep them on their toes, moving from i>clicker to peer discussion to short class discussion and back to i>clicker.  In this way, those who had done some preparation before class got to practice the content and teach it to others.  Those who had not done any preparation got some exposure and some sense of how well they will need to know the material by exam time.  Most of all, though, everyone came out of the class knowing that they could turn to their neighbor (see Julie Schell's blog Turn to Your Neighbor for much more on the power of peer-based education).

Creating Communities of Learners

In their origins in the 12th and 13th centuries, the university--a name derived from the Latin word universitas--were not places; they were groups of people.  Specifically, they were guilds comprised of students and their instructors.  Sometimes students paid the instructor directly (as they did in the Roman period.  St. Augustine, for instance, traveled to Rome from Carthage in the hope of finding less rowdy students who would actually pay their tuition bill.  No such luck...).  Othertimes, the Catholic Church or the state served as an intermediary between students and teachers.  Not surprisingly, it was this latter method that gained a foothold and serves as the model for our current system of tuition paid to an institution rather than directly from student to instructor. The key bit of information that history has to tell us, though, is that universities are not about the brick-and-mortar campus; they are about the people who inhabit them.  Their aim, originally, was to bring together communities of learners with the instructor serving as an expert guide/personal trainer.

Somewhere along the line, higher education has lost sight of the fact that its primary function is to provide a physical space for communities of learners to form (and re-form in each class and in every new semester).  Particularly at state-funded (or, really, poorly funded by the state) institutions like UT Austin, the university and its classrooms have tended to be places for students to crowd into lecture halls for 50-75 minutes at a time, to be talked at by an instructor.  Now, many of our instructors are extraordinarily talented at delivering content clearly and in an entertaining fashion.  The problem is, once students are done listening, they go off on their own and try to do their homework.  They may well not know anyone from their class or have any idea how to get help.  The Course Transformation Program at UT is addressing this issue head-on in some of our most important "gateway" courses--courses that teach several thousand students each year and serve as prerequisites for upper-division courses.  In these courses, instructors are working closely with teams of learning specialists to provide extra support in class and encourage peer learning outside of class, among other things.

My Rome class, which has been dubbed the first of what will perhaps be a series of slightly different courses under the auspices of the CTP program (CTP 2.0), has at its center the belief that the classroom should be a place for supporting and encouraging the creation of communities of learners.  From the first day of class, I have incorporated daily "talk to your neighbor" questions.  I have encouraged them to use Piazza, a discussion board that strongly promotes peer-to-peer interaction.  On their own, the students formed a FB group.  Still, I really wasn't sure how much this overt shift in my teaching style, away from lecture ("sage on the stage") and towards a more student-centered and peer-focused approach was working.

The class has its first midterm exam tomorrow.  In some sense, it is also the first real opportunity I have to see the effects of the "peer-centered" approach I have taken.  I have taught large (100+ students) classes for several years at UT, including this Rome class twice before.  About a week before the exam, I am usually inundated with emails and office visits.  Typically, the students are either asking factual questions whose answers can easily be found in their textbook or in my lectures (but that would require them to have taken good notes); or they ask some version of "what do I need to know for the test".  Test anxiety is particularly palpable here in Texas where standardized testing has had such a prominent (and, to my mind, harmful) status.  This semester, I girded myself for the barrage and warned my TAs to be prepared.  After all, I was now dealing with twice as many students as earlier semesters.

To my amazement, I have had no students ask me any version of "what do I need to know for the test".  Not a single one.  Neither have my TAs.  They already know what the questions will be like because they've been practicing them in class every single day.  I had one student come to my office hours with a couple of questions.  She told me what she thought the answer was and then asked me to explain a couple of things that had piqued her curiosity.  It was just the sort of exchange an instructor longs to have with his/her students.  Instead of lining up outside our offices or shooting off emails, the students are posting questions on Piazza.  Typically, someone has already answered it before I ever get to it and they are all smart, reasonable questions that show them thinking and really grappling with the material.  It is truly remarkable. I am seeing a depth of thought, a willingness to wrestle with course content at a level I can't say that I have ever seen in a class other than a graduate seminar.

The students are forming study groups; they are egging each other on to work hard and ace the exam.  There's no curve so there's every incentive for everyone to help each other.  They have worked through the review questions I embedded in the recorded lectures as a class (it is posted on google docs).  Two brothers used an app they created to generate study guides for every recorded lecture; other students did study guides for readings.  Another student made e-flashcards for a list of terms I gave them and shared them with the class.  I am fascinated by the altruism at work, but also at the spirit of collective "let's work hard, learn this material, and do well".  The exam is written to reinforce this kind of behavior--after all, it is an introductory level course.  The key is putting in the work.  It's a lot of material, and I expect them to really know it.  But it can be learned with some elbow grease.  From all indications, they are doing just that.  The thing is, some kids may not do the reading and may not watch the lectures.  But they are still going to learn a ton from the distilled notes of their classmates.  They aren't getting away with anything, really.

What I've witnessed in just a month of teaching a flipped class is a complete shift in student behavior.  They have been trained to turn to each other for answers, to trust that their peers can teach them--and that is exactly what is happening as they prepare for this exam.  Sure, I am still holding a review session.  But honestly, I am feeling a little bit superfluous and left out.  I'm not complaining, to be sure, but am shocked at how little I am needed once I have given them the tools to learn.  These aren't the passive, helpless undergraduates I'm used to seeing (and helping to create when I, too, lecture at them).  They are active, self-sufficient, curious, and engaged.  It hit me that this is what a true community of learners looks like.  It's a beautiful thing.

Student Engagement in a Large Class Setting

The aim of flipping a class--taking content delivery out of the classroom and devoting class time to application of content--is to foster a student-centered, active learning environment.  Even the most charismatic lecturers will struggle to keep a class of 400 on their toes, actively thinking and engaging with the content of the lecture, for 50 or 75 minutes.  Oftentimes, from our perspective on the stage, it will look to us as if the students are eagerly listening to our carefully crafted lecture, laughing at the jokes, answering the questions, and nodding in agreement.  A few of them are, in fact, staying engaged; but one consequence of observing my colleagues in their large classes (and having others observe me in mine) is that it is evident from the perspective of the student in the audience that our perceptions are just that...perceptions.  Students who look like they are taking notes are doing their math homework or buying clothes online or IMing with their friends.  Students who sit on the edges of the room or in the back rows, out of my sight line (many of our auditoriums are very poorly designed), are napping or reading the paper. Of course there are always those devoted students who populate the first few rows who really are engaged. The challenge, though, is getting the rest of them engaged in a meaningful way (and I'd argue that simply tossing a few questions to the audience or having them talk in groups about something isn't necessarily meaningful engagement).

The central goal of every class I am teaching this semester is engaging every student in the room at all times and as much as possible.  As the instructor, my focus is completely on class.  Yet, I know that it is easy for student attention to wander when they aren't "on the hot seat" (mine certainly does when I am in workshops or lectures that require me to sit still and listen to a presentation for 45 minutes).  In my other class, an 8 student course for graduate students during which we read large amounts of Latin prose and work on improving their skills and speed, there are some relatively easy ways to do this.  In the Rome class, with 388 students, it's a major challenge.  I>clickers are a tremendous help, both in engaging students at a particular moment and in keeping them ready to be engaged.  They never know when another question will be put up.  I also use peer instruction at least once/class.  I am working on ways of blending i>clicker pools and peer instruction so that they play off of one another.

We also do "class" discussion, during which students will offer answers or thoughts on a topic.  There are myriad problems with this mode of discussion, however.  First, it is difficult to hear.  Sometimes I can't hear what they are saying and have to ask them to repeat it.  I then have to repeat the comment for everyone else to hear.  Often, that still isn't sufficient for kids in the back of the room.  They can watch the recording after class to find what they missed but it is difficult for them to stay engaged during class itself.  As well, the layout of the room makes it very difficult for me to see people on the sides of the room who are raising their hands.  The stage is very shallow and a large swath of the room is completely outside of my peripheral vision.  Moving off of the stage and around the room helps a bit but then I have my back to some section of students.  Finally, "class discussion" is more of a 1-1 form of conversation.  It works well (or at least better) in small, seminar-based classes.  It is less than ideal in a very large class and I will be avoiding it as much as I can.  It is too easy for the majority of the class to tune out, even if briefly.

I will be experimenting with some other ways of keeping everyone involved, such as using colored cards that they raise.  In future semesters, I might allow them to tweet questions or responses.  Figuring out how to keep everyone involved is the biggest challenge of teaching a large class, but it is essential to the flipped design.  It will be the part of a course redesign that frustrates and stymies faculty like me who are accustomed to a lecture-based mode of teaching.  It requires us to completely re-think what we do in class; and requires us to discover and arm ourselves with an entirely new bag of tricks.  This is where support from a campus Center for Teaching and Learning will be crucial.  I have been the lucky recipient of great support from UT's CTL staff in the form of one-on-one consultations and workshops.  This fall I am participating in a Collaborative Consultation group on the flipped classroom.  A group of faculty meets with CTL staff members twice/month for 90 minutes to talk about our experiences with flipping our classes, tools we have found, etc.  I am expecting that this will be an invaluable resource, particularly for finding ways to keep every student as engaged as possible during class meetings.

Close-Captioning of Echo Lectures

One of the coolest elements of the pre-recorded Echo videos is the close-captioning feature.  Once we had a final cut of the content lectures, UT's Liberal Arts ITS department sent them out to be close-captioned (a not-inexpensive process).  This way, all of the videos are accessible to hearing-impaired students.  In addition, all students can keyword search the close-captioning of individual lectures.  This allows them to find specific places in the lectures and review a particular concept.  So, for instance, if they are sure what a term means, they only need to know what lecture to search in order to find my discussion of it.  They cannot do global searches (so, they cannot search for all mentions of Caesar in the lectures).  This is an amazing feature, and will be especially useful when they review for exams.  I will be very curious to learn more about how they are using the close-captioning to access and review content and, more generally, to increase their mastery of the material.

I>Clickers

My colleague Deborah Beck has a great rule.  She incorporates new teaching tools one at a time.  This is her very wise way of ensuring that her teaching stays fresh but that she doesn't go nuts trying to learn and juggle several new techniques or technologies at once.  I have tried to follow her advice with the Rome class.  A lot has changed, but those changes happened over the course of many months.  In class, I have changed my mode of teaching from primarily lecture to entirely student-centered.  Thus, what I put on my PowerPoint slides is very different: now, instead of names and dates, it is questions for them to answer, talk about to a peer, think about on their own.  The technology hasn't changed, however, except in one notable way.  I am now using i>clickers. 

Faculty at UT have been using i>clickers for several years to take attendance, get student feedback, and to advance discussion during lecture.  I had thought about using them in the Rome class, particularly in 2011, but opted not to.  Last fall was about learning the Echo360 technology.  When I redesigned the Rome class to make it discussion-based, I realized that I needed to have a. a quick way to take attendance; and b. a way to keep 400 students simultaneously engaged.  The obvious solution was to incorporate i>clickers.  More and more classes are using them, particularly in the natural sciences and math/engineering.  They are relatively inexpensive ($40 for the 1st generation model I require) and students can borrow them from a friend who is not in the class.

UT's Center for Teaching and Learning offers excellent support for faculty who want to learn how to use i>clickers.  I met with Mike Wallace a couple of times to talk about the kinds of questions that work best to stimulate thinking and discussion; and for advice on how to use polls and peer-discussion together.  Mario Guerra walked me through the technical side of things and helped me get everything set up for my class.  When I walked into the classroom on the first day, I felt confident that I had the necessary tools to start experimenting with i>clicker polls.  Three weeks into the semester, I am a total convert.  I>clickers are excellent tools for keeping a large class thinking and also for getting discussion going or pushing it in new directions.  I especially enjoy using questions to start a discussion and then asking the same question after they have discussed their views with one another.

I have experimented with how much time to give them on questions.  They need more time on the first question, because they have to get their clickers out of their bags.  For short,  factual questions, 30 seconds is usually sufficient.  For most questions, 45 seconds is enough time for even the more deliberate thinkers.  My goal is to give everyone enough time to read the question, think about it if need be, and answer; but to avoid 30 seconds of watching the seconds tick away.  One thing I repeatedly forget to do: start the poll.  I assume that this will become second-nature eventually, but I seem to forget at least once/class and am reminded when the students start rumbling.

Next semester I will be more clear about a few things: first, it is their responsibility to have a functioning i>clicker by the second class meeting.  Unless their is an obvious technical malfunction, it is their problem.  Second, I will include specific instructions about how to use it, what frequency for 2nd generation clickers, how to delete a clicker registration and register a new one, etc.  By far the most student questions about the course up to this point have concerned the i>clickers.  I suppose this is to be expected since i>clickers are a relatively new technology.  Still, for all the hassles, I can't imagine teaching a student-centered large class without them (or something like them). 

I can't quite imagine adopting a tool that requires them to have a web-enabled mobile device (like learningcatalytics.com) at the moment.  I love the idea of being able to ask questions in forms other than multiple choice, but fear that the technology problems would outweigh the benefits--at least in a very large class and at least at this moment in time.  My thought: if pressing a single button on a simple remote is so difficult for some of them, how would they manage a phone or tablet or laptop?   I think I can require that they have such a device, but it will be some time before I feel like students can be counted on to operate those devices using interactive learning tools in a classroom setting.  That said, I can't wait to try out the Learning Catalytics tools in a smaller class setting (e.g. a discussion-based seminar of 25-30 students, where tech problems are easier to manage).  Perhaps, with some experience on my side, I will feel more comfortable introducing these more sophisticated, personal device-based tools in a larger class setting.

Lots of Moving Parts

Teaching a 100+ student class is a challenge.  It's not the teaching part that is difficult for most of us but all of the classroom management issues, the degree of organization that is required to ensure that everyone and everything is in the right place at the right time, the time it takes to answer the inevitable logistics questions that arise for the students (despite the fact that you covered all of it in posted course documents and multiple times in class).  Make it 400 students; add a team of 4 graduate student TAs and 2 undergraduate graders; and have both in class and outside of class activities to harmonize and the challenges become all the more apparent. There are a whole lot of moving parts and a lot of potential for something to go wrong.  Three weeks into my adventure in flipping, I find myself mentally and physically drained by Friday afternoon (and, in truth, by Thursday morning).  Am I getting old?  Well, yes; but that's not the real cause of my bone-aching exhaustion.  Rather, it's the fact that there's so much--and so many people--to keep organized.  BlackBoard, UT's current LMS (Learning Management System), helps a lot; so does our class site on Piazza and the Echo recordings of every class session.  Still, there's a lot to do this first time through.

A brief overview of the course's set-up: The class meets three days/week for 50 minutes (really, 45 minutes because our room is on the southside of campus and most of the students have class in the next hour on the other side of campus; rather than deal with a mass exodus at 45 minutes after the hour, I lopped off the last 5 minutes of class).  The students are asked to do three types of work outside of class: read selected pages from the course textbook (A Brief History of the Romans); read selections from ancient primary sources (Livy, Vergil, Sallust, etc.); and view pre-recorded videos in which I digest, explain, highlight the content delivery of the textbook.  During most weeks, class time on Mondays and Wednesdays is devoted to looking closely at a momentous event or action (e.g. rape of the Sabines; Tiberius Gracchus' land redistribution legislation).  The assigned primary text reading generally provides an account and/or analysis of the event discussed in class. 

Mondays are typically spent reviewing the major elements of the event (i.e. carefully working through the primary text to extract out important plot details and also identify potential reporting biases).  The goal on Monday is to highlight the aspects of the case study that will help them to evaluate the ethical justifications for an ethically-questionable action.  Typically, this means teaching them how to combine textbook "facts" with the close reading of a primary text.  On Wednesdays, we engage in ethical analysis. Active and engaged learning is the focus of class both days and I use a variety of methods to support that (i>clickers; peer discussion; class discussion).  On Fridays, a senior graduate student prepares a review of the textbook/video content.  I work closely with her to have her review mirror the student-centered ethic of the class.  She selects certain "high priority" content to focus on, and then writes i>clicker questions or poses questions for peer discussion.  Her review is designed to elicit learning from the students and help them identify gaps rather than simply to re-deliver content.  The material that they review on Fridays provides the historical background for the case they will work through the following week.  Ideally, I'd have Mondays be devoted to review of content and the case study on Wednesdays and Fridays but at UT, during football season, well, that seemed unrealistic (though I do take attendance and attendance is 10% of their final grade).

Harmonizing: When I wasn't filming lectures this past summer, I was working on redesigning the in class part of the class.  I had decided early on in the process that I would be doing case studies, but I struggled a lot with exactly how to do that, and how to create a clear connection between the Roman history component and the ethics component.  I knew that a key to the flipped class's success was creating a clear relationship between the out of class and in class parts of the course.  Still, it was difficult to see exactly how this might work.  As the class has gotten underway, I've made a point of regularly and deliberately reinforcing that connection, whether by asking i>clicker questions about material from the lectures/textbook or by posing questions for discussion that require them to put together the specifics of the case study with the larger historical picture (e.g. Livy says the Tarquins were bad and yet we know that, under their rule, Rome became the most powerful city in Latium).  I also made the decision to have the review session be held during regularly scheduled class time.  I don't require attendance, but about 75-80% are attending.  Thus far, I've been extremely pleased with how seamlessly these out of class and in class activities are coming together, reinforcing one another, and leading to much deeper learning and more sophisticated student insights.

It's Not Actually About Me

This afternoon I met up with Dr. Michael Sweet, a fan of Roman history and the director of Instructional Development in the Center for Teaching and Learning at UT.   We chatted about flipping courses, teaching attitudes, team-based learning (one of his special interests), and discomfort in the classroom.  It was a great conversation for me and I wanted to write about some of the things that emerged for me.

We talked a lot about how the flipped class can create instructor discomfort.  Experienced instructors are pretty comfortable with standing in front of a group of students and delivering content, perhaps breaking up their content delivery with a few questions for the audience.  Of course, we weren't always comfortable with this mode of instruction.  It took practice, trial-and-error, and a lot of discomfort.  We looked to our own teachers as models to imitate.  After teaching a lecture-based course a few times, however, we realize that it's not so bad.  We have our Power Points (of course we fiddle with them each semester); we have our jokes; we have our "teaching persona".  We walk in, we entertain and instruct, and we leave.  The audience of students is often pleased with our performance.  Throughout, we are in almost control of the script the class will follow.  Sure, a few students may ask questions, but for the most part we say exactly what we had planned to say.  After a few semesters, we even know what questions the content will elicit and have well-crafted answers ready to go.

When I walk into my flipped classroom, armed with a PowerPoint and a plan for the discussion, I am aware that I have no idea what is going to happen.  We will start where I choose, but we will end where the discussion takes us.  It used to be that I ran the show, scripted the performance.  Now it's my students who do that.  Sure, I set the topic for discussion and get the ball rolling.  I stage interventions from time to time.  Often, though, I feel like I am trying to herd a clowder of wild cats.  This is a good thing.  This is what student-centered learning is all about.  It's an adjustment.  When the class discussion does not unfold exactly as I imagined it would, planned for it to do, I sometimes leave class feeling like I didn't do my job.  I have to remind myself that this sense of slight disorder is what a flipped class is all about.  I don't want to create the impression that my class is chaotic and discussions jump all over the place--not at all.  But it's certainly true that discussions take turns that are unexpected.  One of the big lessons for me in these first few weeks is that it is my job to come to class with a plan; and it is my job to be ready to abandon the plan and improvise at a moment's notice so long as learning is happening.  I have to remind myself of this every Monday and Wednesday as I make the walk from my office to my classroom.

One thing that has helped me in the transition from a professor-centered to a student-centered classroom has been to remember that, in a way, I've already taught the class.  The recorded lectures in which I deliver content are the backbone of the class.  If that's all they ever get from me, they've gotten a better version of this class than any other group of students.  Whatever else they get is a bonus; it's value-added.  If we don't cover absolutely everything I intended to cover in a particular class meeting, no problem.  Whatever we did do was a bonus.  They are also getting several other bonuses: an active discussion board on piazza.com, a Twitter feed with weekly Tweetchats, a class Facebook group.

In student-centered teaching, it's really not about me, it's about them.  It's not about what I am doing but about what they are learning.  I have long been a critic of the notion that good teaching can be equated with inspirational, charismatic performances at a lectern.  It's nonetheless been a challenge to rid myself of the belief that I can inspire students to learn through a witty, well-delivered lecture.  Even though I've sat in enough classrooms of very good teachers--watching their students surf the web, work on their math homework, and shop for a dress--to know that the focus has to be on student learning rather than faculty teaching, it's still difficult to fully embrace the consequences of that realization.   Good teaching has less to do with us (the teachers) and everything to do with our ability to increase student learning, however we may do that.  Dr. Eric Mazur addresses this in his talk "Confessions of a Converted Lecturer."

There is a lot of evidence-based research on student learning.  One very clear conclusion: active, engaged students learn and retain knowledge better than passive students listening to a lecture, no matter how talented the particular lecturer might be.  The challenge for an institution like UT Austin: turning a class of 400 students (or 200 or 800) into active learners.  The other challenge: persuading faculty, especially in liberal arts, that we can let go of our lecture-based mode of content delivery (or, really, remove it from the space of the classroom) and turn the classroom into a space for engaging our students in discussion and critical thinking--even if this means that, sometimes, we are going to feel a little out of control, uncomfortable, out of our element.  Because, after all, it's not about us; it's about our students.

Some Challenges Along the Way

I would be remiss if I did not address the challenges that came with flipping the Intro to Ancient Rome class.  I will write about the challenges of the flipped classroom in a separate post at a later date.  In this post, I want to touch on the challenges I faced during the filming of content delivery.

The Emotional Challenges:  I am a shy person by nature.  I don't like to be "out there" and I don't like feeling vulnerable (great TedTalk on vulnerability by Brene Brown) and exposed.  Yes, I'm a professor and I make myself vulnerable for a living, but it's still scary and uncomfortable.  Over the years, I've learned to tolerate and even feed off of the challenges of teaching a 200+ class of students.  I've learned to deal with the inevitable judgments and criticisms that are part of publishing one's research.  I tolerate the discomforts because I truly believe that my teaching and research benefit others--at least, I hear this often enough to keep me going.  Still, like many others who spend a lot of time in vulnerable positions, I don't always find it easy to put myself out there time and time again.  For someone who is naturally shy and reserved (yes, colleagues, I really am reserved!), the prospect of standing in front of a camera is utterly terrifying.  I managed to not really think about it until the weekend before we started to film.  I spent most of that weekend working on PPTs, watching the Women's College Softball World Series, and burning off my panic on my elliptical.  Seriously.  I was really scared.  I couldn't believe that I had agreed to let myself be filmed while lecturing and then let students watch me talking to them.  I would have no control over how and where students watched me.  I imagined them playing drinking games ("every time she says 'um', take a shot"); mocking me; and worse.

By the time I showed up for the first session of filming, I was terrified.  I am pretty skilled at hiding my panic and I don't know how apparent it was to Mike Heidenreich.  Fortunately, Mike has the most soothing and calm voice and manner of anyone I've met.  He immediately put me at ease and made me feel comfortable in the confines of the audio studio.  Pretty quickly, I realized that I needed to force myself to be completely present in the moment (my athletic background suddenly became very useful).  No thinking about anything except whatever I was speaking about at that very moment.  In fact, I soon came to look forward to my days in the studio.  Before we started and between takes, Mike did a great job of keeping me loose and laughing.  I haven't laughed so much, ever, as I did during the 4 weeks we worked together.  I am pretty sure that, with a less skilled "handler", I'd have done a much worse job.

I still can't watch myself on film for very long.  I watch for about 5 minutes, press pause, do something else, watch for another five minutes.  I still feel vulnerable and exposed.  Now that the class is underway, though, I also get to see all the ways that these pre-recorded lectures have enhanced student learning and made it possible for me to create a classroom that promotes active learning.  The payoff is completely worth the discomfort.

Teaching injury!

The Physical Challenges:  In part because of the intense filming schedule that I chose (because I knew I needed most of July to focus on redesigning the in-class side of things), the process of filming was hard on my body.  Most of us are used to walking around when we are on our feet for long periods.  But filming required standing completely still.  My feet and legs ached tremendously by the end of the day.  I slept with an ice pack on my lower back and took anti-inflammatories like they were Skittles.  I got a massage every week, something I strongly recommend if you are doing a lot of filming.  For the final two weeks, Mike got an anti-fatigue mat for the audio studio--this made a huge difference to my legs.  Within a few days of finishing, most of my aches and pains went away.  My right foot, however, was very sore and a bit swollen.  It turned out that I had fractured a small bone on the bottom of my foot (sesamoid).  It hurt like crazy and took about 6 weeks in a walking boot and a cortisone shot in the bottom of my foot to finally calm down (the bone won't actually heal; the aim is just to reduce pain and swelling).  Lesson I learned: barefoot is a bad idea.  Always wear supportive shoes when standing still!

How I Flipped Out: Pre-Recording Content Delivery

In early June, I focused my attention on reworking the presentation of the content of my Intro to Ancient Rome course.  Once I made the decision to pre-record the lectures using our Echo360 lecture capture technology, I also had to revise and reorganize my Power Point slide presentations.  Even if I had had the time in the spring to start work on revising the Power Points--time that I didn't have--there was no way I could have understood how to do it until I actually did a day of test shots in front of the camera.  It was really only when I was in front of the camera that I was able to intuit the ways that the medium of lecture capture (and shortened lecture times) demanded a different kind of presentation than what I had typically done during a regular class session.  In fact, it becomes clear that, about a week into shooting, I really "got" how to work with the medium of lecture capture and maximize its strengths.

After a weekend of frantically preparing the first 4-5 PPTs, I showed up at the audio studio for the first real day of filming.  We filmed on Monday, Tuesday, Thursday, and Friday from 10-1 pm (often more like 2 pm).  We usually filmed 4 lectures/session.  We did two weeks on, a week off, and then two more weeks on.  There were two additional one-day sessions in late July and mid-August to film (and re-film) some lectures orienting students to the course, to blended learning, and to the ethics approach.  We filmed a total of 51 lectures (some of these were re-filmed a few times!) plus the three "introduction to the course and its approach" lectures.  The lectures were about 15-25 minutes in length, and most of them were right around the 20 minute mark.  We ended up re-recording about 15-20% of them, for a range of reasons: the first cut was too long; there was a typographical error on a slide; I was tongue-tied (this happened exclusively at the starts: every once in a while, I would have trouble getting my momentum going); there was a technical problem (one day we lost all of our work because my new computer power cord was creating some kind of interference with the Echo Box); or I just didn't feel like I had done the lecture as well as I could have. 

Now, I'm a perfectionist (like many academics!) and I had to let go of a lot of that during this process.  Still, there were times where I really felt like I could do better and it was worth a second shot.  With the Echo360, we could edit things out but we could not splice things in.  To make this work, one has to stop after making a mistake, wait about 45 seconds, and then start again so that the editors can cut out the mistake.  I could never train myself to do this.  Once I lost my momentum and train of thought, it was gone.  I always found it easier to start over.  Fortunately, most of my troubles came within the first 2-3 minutes so it was easy to just start anew.  In the end, we did keep a few lectures that had tiny errors in them (saying 507 BC instead of 509 BC for the foundation of the republic, but in a context where they have already learned the correct date and it's clearly just a slip), with the thought that we will surely be doing more recording for other versions of the class and so can replace them then.

Reviving the Art of Close Reading

Perhaps the greatest challenge for instructors of large courses (100+ students) is being able to move beyond the delivery of textbook facts and engage in the careful analysis of primary sources.  In my discipline, Classics, the close reading and analysis of literary texts or material artifacts is what we do.  In Latin class, we translate passages, we parse forms of nouns and verbs, we talk about the nuances of specific words.  My own research methods are very dependent on the skills of close reading of literary texts.  I write about big ideas, but my arguments regarding those big ideas are based in the careful analysis of ancient written sources (most of them in Latin).  Yet, in our large introductory courses on Greece and Rome, it is a challenge to get students to engage deeply with ancient primary sources (in translation).  My colleagues who teach our Intro to Ancient Greece course do a splendid job of incorporating primary sources into their classes.  In the Intro to Ancient Rome course, however, we have a more difficulty finding primary sources that are student-friendly.  There is no Latin Plato or Sophocles or Herodotus.  Roman comedy is dense and culturally specific.  Vergil is moderately accessible as is the ancient historian Livy; Sallust has some fun bits.

In the Intro to Ancient Rome, I always assign several selections from Livy to the students during the first section of the course.  Livy is pretty accessible when he talks about Romulus and Remus and the foundation of Rome; and then the regal successors to Romulus.  When he gets to the Tarquins, however, things start to get a bit tricky--in large part because this is when Livy's own historical biases become especially prominent.  His presentation of Tarquin the Proud, Tarquin's son Sextus, Sextus's rape of Lucretia, the expulsion of the Tarquins by Brutus and his followers, and the foundation of the Roman Republic is dense, complicated, full of unfamiliar characters and places and, overall, very difficult for students to understand.  In previous incarnations of the course, I did my best to highlight the major events and characters and one or two themes (e.g. Sextus' violation of guest-host relations), but did not say much about the complexities of the scene.  I had too much basic content to deliver and no time to linger.

In the current flipped class, I don't have to worry about content delivery--it's already done.  This week, the Monday and Wednesday class meetings focused on this episode, with the aim of analyzing the ethics of Brutus's use of Lucretia's rape and suicide to inspire a rebellion against the ruling family, the Tarquins.  In order to begin to evaluate Brutus's actions, it is necessary to engage in a careful reading of Livy's report of the events.  On Monday, we went through several key bits of Livy's account in order to highlight elements of the "case" that might bear on their analyses (e.g. that both Brutus and Sextus engaged in acts of deception, but to different ends).  We did a bit more of that on Wednesday, but then jumped into serious discussion of Brutus's behavior and the arguments that might justify his actions.  We chatted a bit as a class and then I had them "talk to a neighbor".  In order to elicit justifications, they had to be able to know and have some understanding of Livy's account.  That is, they had to engage in the careful reading of a primary source--a kind of reading that went well beyond mere plot summary.

If I needed any confirmation that my current students are learning to read Livy at a depth that I've never seen before in a large, introductory level course, it came shortly after class when a student posted a question on our Piazza.com discussion board.  He was confused about an i>clicker question I asked near the end of class about Brutus's display of Lucretia's body in the forum (i.e. town square) of her hometown of Collatia (15 km Northeast of Rome).  He thought the forum was only in Rome itself.  I quickly answered that most cities had a forum.  He then followed up with a thoughtful and very detailed question about a passage in the Livy text.  He offered his own interpretation and then asked for other opinions.  By the time I saw the follow-up question,  three or four students had already offered various (correct) answers and added useful information.  I was then able to add a couple of links to outside sites.  What struck me was how, in a matter of a few hours, these students hanging out on the Piazza site were able to use this passage to open up a very interesting perspective on the very question we had been considering a few hours earlier in class.  Close reading at its best!

Becoming The Talent

In early June 2012, we began the filming of the lectures that the Rome students are watching outside of class. Initially, I had wanted to record only my voice and the screen capture (as happens when Echo360 is used in class).  I had no problem going to the audio studio to do this since the sound quality would be much better than what I can get recording on my laptop with the cat meowing in the background.  I was not, however, especially eager to be filmed.  I pushed hard not to be filmed, in fact, but the Assistant Dean who runs the Liberal Arts ITS department, Joe TenBarge, was insistent.  In part, filming me would allow them to collect data about student viewing habits.  Students often  say that they would like video on the in-class Echo recordings; now they could see how true that was, at least in one case.  Of course, in my own head, I am thinking, "hmm, maybe you'll just find out that they don't want to watch *me*!"  Now that I have real students watching these videos, I am more persuaded that it was a good decision.  I can see the ways in which having me present and speaking directly to them creates an interesting kind of intimacy.  This is an important antidote against the inevitable depersonalization that occurs in a 400-student classroom.

For reasons that seem silly in retrospect, I had imagined myself teaching in an empty classroom, with a camera somewhere out of sight filming me.  I thought that the weirdest part would be getting used to talking to nobody.  Well, yes, that was weird--but definitely not the most disorienting part of the process.  Before I began filming, I was given an orientation to the audio studio and introduced to the student interns with whom I'd be working, in addition to the UT Echo guru Mike Heidenreich.  We also did some test shots.  One thing that quickly became clear was that I needed to make some important decisions about my appearance: did I want to wear the same thing every day? a new outfit every day?  or something in the middle?  As well, I realized that I would need to real wear make-up because of the bright studio lighting.  For the test shot, I wore a navy blue shirt with some decorative fabric folds on the front--pretty simple as far as women's clothing goes.  On camera, however, it looked distracting, especially because I have dark-rimmed glasses.  The image was too busy.  I also realized that it was not a good idea to wear any sort of dark eye-shadow because, every time I looked down at the computer screen, the viewer got a full view of my eyes.

It didn't take long to arrive at a couple of important decisions: if the video is just a head/upper torso shot, it's probably a good idea to wear something very simple if you want to blend into the background and have the students focusing on what your are saying rather than on you (of course, if you want them looking at you, wearing a new and outrageous outfit every day is the way to do it!).  Wearing foundation and powder (for women, at least) is a good idea.  If you wear eyeshadow, make it a neutral color.  Simplicity was my guiding principle and it seemed to work well.  I ended up wearing a long-sleeved, pinkish t-shirt with a decorative but simple neckline.  My glasses are what stand out the most, but otherwise I coordinate with the Power Point design.  It never occurred to me that a. I would need to think about how I looked on camera; and b. think about what my objectives were in creating an image (have the students think I was a cool dresser?  get them talking about my different "looks"?  fade into the background as much as possible to help them focus on the content rather than the person delivering it?).  It also never occurred to me that I would have to walk around in the hot Texas summer sun with foundation and powder on!  My favorite part of the day was getting home and scrubbing my face.

A screenshot of what the students see when watching the lecture

Another big surprise to me about the filming process was learning how to speak to a camera.  Yes, as it turned out, I wasn't going to be filmed in an empty classroom but rather, the audio "cave"--a soundproof room with a microphone, a camera, a stand holding my laptop, and me.

Bobby the bat enjoyed his time in the audio cave so much that he has become a permanent denizen

I did bring an "audience" with me--some fuzzy friends who inhabit my office--but otherwise it was just me.  And it was weird, disorienting, uncomfortable, scary, etc.  I am accustomed to standing in front of a crowd and lecturing (aka delivering content).  But talking straight to a camera?  I quickly realized that all of my usual teaching habits--making eye contact with as many students as I can; speaking at a reasonably slow pace so that everyone can process what I am saying; and moving around the room--were precisely what I could NOT do when being filmed.  On film, the lecture looks best when one stands completely still (some hand gestures are ok), stares down the camera, and speaks as quickly as possible.  For the first few days, I felt completely out of sorts--here I was doing all the things that would make me a disastrous lecturer in person.  Yet, when I watched the rough cuts, it looked good--and completely normal.  Perhaps most surprising to me was the disconnect between my sense of how fast I was speaking and how normal it looked on film.

Finally, it became immediately obvious that I would need to completely re-work the Power Points I had used to teach the course in the previous fall.  The ideal length of a recorded lecture is about 15 minutes and certainly no more than 25.  In addition, because the students could press pause, I realized that it would be helpful to insert review questions at the midway point and at the end of each lecture.  That way, they could quiz themselves on what they had listened to and rewind if necessary.  They could also use these review questions to guide their preparation for midterm exams.  All of this meant that I completely re-shuffled the arrangement of material.  Each lecture covered less material, but was also more focused on specific topics.  Particularly when it came to lecturing on the complexities of the last century of the Roman Republic (the period covered by HBO's Rome), it was a big advantage to be able to present the material in smaller and more digestible chunks.

Teaching Ethics in a Large Class Setting

In 2004, the Commission of 125--a collection of the top civic, educational, cultural, political, and business leader in Texas charged with studying the University of Texas and offering advice for improvement--released a report of its findings.  Among their recommendations was the exhortation that all University of Texas students "examine questions of ethics and the attributes of effective leadership."  In the wake of Enron and other corporate scandals, among many other ethically questionable happenings across the US, it has become clear that universities have a responsibility to ensure that their graduates leave with an awareness of what are generally considered moral imperatives (e.g. don't cheat, keep your promises, do your duty); an ability to recognize when a moral imperative has been violated; and well-developed skills in ethical analysis.

In response to the recommendation of the Commission of 125, the College of Undergraduate Studies began to offer the possibility of "flagging" certain courses with an Ethics and Leadership Flag.  The Ethics and Leadership flag is one of six flags in a new core curriculum currently being instituted by individual colleges and schools at UT.  To fulfill the core curriculum requirement, students must take (and pass!) at least one course that carries an Ethics and Leadership flag. To flag their course, instructors submit a description of their course objectives, sample readings and activities, and possibly a syllabus to a committee charged with ascertaining that the course meets the requirements of the Ethics and Leadership flag ("at least one-third of the course grade must be based on work in practical ethics, that is, what is involved in making real-life ethical choices"). 

A recent initiative of the College of Undergraduate Studies has focused on redesigning large lecture courses (100+ students) to carry the Ethics and Leadership flag.  In Fall 2011, Jess Miner approached me about adding the flag to my Introduction to Ancient Rome course (Classics was one of the "target departments" for this initiative, and also the field in which Jess holds a PhD).  I was intrigued--the history of ancient Rome is rife with ethically questionable actions, after all!  At the same time, I was unsure how I would be able to add more content to a course that was already overflowing.   I realized that, in order to integrate the ethics component fully and devote enough time to practicing ethical analysis with the students, I would need to shift at least some of the content delivery out of the classroom. 

I was a bit reluctant to change the design of a course that, frankly, was working very well and seemed to be well-received by students.  At the same time, I loved the idea of integrating pragmatic ethics and getting students to see how the lessons of ancient Rome might have applications to their own lives.  After a lot of mulling and consulting with different people, I finally settled on a plan for redesigning the class: I would pre-record the delivery of the Roman history content (essentially, the in-class lectures) and have students watch these outside of class.  In class, we would work though a set of case studies involving ethically questionable actions (e.g. Rape of the Sabines; Brutus' deception and misuse of Lucretia's body to end monarchy and found the Roman Republic).  In other words, I'd "flip" my class.

I did most of my flipping from June-August 2012.  The redesign was funded jointly by Liberal Arts ITS and the Provost's office at UT and I benefited tremendously from the advice of Mike Heidenreich in LAITS and several members of the Center for Teaching and Learning staff (in particular Stephanie Corliss, Mike Wallace, Erin Reilly, and Mario Guerra).  I consulted frequently with Jess Miner to select my case studies, get tutored in the basics of Deni Eliot's Systematic Moral Analysis, and to work through my approach to teaching the case studies.  Among the many lessons I learned this summer: designing and creating a good class is a team effort. 

We started our second case study in class today--Brutus, Lucretia, and the end of the Roman Monarchy.  On Mondays, I use i>clicker questions to review material from the out of class lectures and assigned reading of a short selection from an ancient primary source.  The aim is to bring out key points for our detailed ethical analyses on Wednesdays (on Fridays, one of the TAs leads a review of the out of class material that I don't cover).  A couple of the key issues today: the fact that Brutus himself was a member of the Tarquin clan; and the topic of deception. I continue to be impressed by the students' perceptive comments; their eager participation; their preparation (about 75% of them get the i>clicker questions right); and the natural way that they are using ethics to deepen their understanding of the complexities of Roman history.  It's not just that they are learning about ethics; it's that their learning of Roman history/culture seems to be significantly deeper than that of previous students in this course precisely because the ethics component encourages them to think harder about the subject matter.

On Saturday, in the aftermath of the bomb threat and evacuation, I posted a note on Piazza.com (a site we are using for discussion) suggesting that they could apply what they knew about moral laws and ethics to the UT administration's late evacuation of campus.  Within an hour, a pretty active conversation erupted, with a range of viewpoints.  I was so impressed to see how quickly and easily they were able to highlight the important issues and make arguments pro and con.  It was an impressive demonstration of student learning.

Lecture Capture: The Student Perspective in December 2011

At the end of Fall 2011, Liberal Arts ITS at UT Austin conducted a survey to get feedback from students users about their experiences with the Echo360 lecture capture technology.  98% of my class responded to the survey (they were promised two extra credit points on the fourth midterm if the rate of response was greater than 95%).  The results were pretty interesting to me and I wanted to share them (my favorites were the students who recognized the way it might discourage class attendance; if I ever teach a typical lecture format again, I will put these quotes on the syllabus!).  The first number is for my class; the number in parentheses is the average for all classes using Echo360:

Did you access the lecture capture recordings during the fall 2011 semester?
Yes - 99.1% (84.8%)
No - 0.9% (14.7%)

Overall, I was satisfied with my experience using the lecture capture recordings.
Strongly agree - 51.6% (45.0%)
Agree - 41.2% (44.8%)
Neutral - 4.5% (7.3%)
Disagree - 0.9% (1.4%)
Strongly disagree - 1.8% (1.5%)

When did you primarily view/listen to the recordings? Check all that apply.
Same day as lecture - 17.2% (14.9%)
Same week as lecture - 59.7% (62.2%)
Shortly before exam - 76.9% (68.1%)

What was your main reason for viewing/listening to the recordings?
To review information after the lecture - 7.7% (10.0%)
To add more information to my notes - 6.3% (16.3%)
To prepare for an exam - 48.4% (37.3%)
To get information I missed when absent from class - 29.0% (31.5%)
Instead of seeking assistance from a TA or instructor - 0.5% (0.6%)

Viewing/listening to the lecture capture recordings helped me learn.
Strongly agree - 55.9% (51.8%)
Agree - 36.8% (40.0%)
Neutral - 5.5% (6.2%)
Disagree - 0.0% (0.5%)
Strongly disagree - 1.8% (1.5%)

Viewing/listening to the lecture capture recordings helped me understand the lecture content.
Strongly agree - 52.5% (49.7%)
Agree - 37.1% (40.7%)
Neutral - 7.7% (7.4%)
Disagree - 0.9% (0.9%)
Strongly disagree - 1.8% (1.4%)

If two sections of this course were offered with the same instructor, one with lecture capturing
and one without, which section would you take?
Section with lecture capture - 91.0% (83.5%)
Either - 8.6% (15.4%)
Section without lecture capture - 0.5% (1.2%)

Please provide any thoughts you have about lecture capturing. (here are some of their
responses):

I think this is a great tool for the classrooms! My professor adds a lot of content to
her lecture that isn't found directly on the powerpoint, so hearing her words
allowed me to copy down the extra details that are so important in this course.

The service is convenient for both the instructor and the students, however it may
result in a perverse incentive: not going to class.

I thought it was great. I would listen to them before the tests and all of the
information would be refreshed in my mind and I would do well on the tests.
Amazing tool, helps whether or not you attend lecture. University should not be
hesitant to expand the program just because of concerns about students skipping
lecture.

Extremely useful if you didn't understand something, had to miss a day,
or simply wanted to review. Would love to see this offered in more classes

i loved the lecture capturing, it helped me review information that the professor
had rushed through during class.

It makes me NOT go to class. It's evil!

it is such an amazing tool that helps the most in times of need, both for the
student and the professor. I, as a student sometimes woke up late, or had a test
that same day that required more studying and being able to study an extra hour
and go back and playback the lecture is just like I didn't miss anything! For the
professor, when she isn't feeling her best she just recorded the lecture and
posted it up without having to go to the class, that way we don't miss a class and
fall behind on material either! Love this!