Wednesday, December 5, 2012

Extra Credit

In general, I'm not a fan of extra credit.  Students, of course, love it.  For them, it's a kind of second chance.  The problem is, they also tend to think of it as "free" points.  They don't view it as a second chance to learn something; or an opportunity to learn content beyond the course syllabus.  Rather, in my experience, they often treat it as something they will sort of take a stab at and then expect to get the maximum number of offered points.  Over the years, I've started offering some extra credit on exams and also to final grades in large enrollment classes.  Partly, it's my way of making up for a question on the exam that may have been unexpectedly difficult for most students; and it's a way of telling them that I won't negotiate on final grades.  It gives them the chance to cover those small gaps between letter grades and leaves me giving them whatever grade they earned without feeling bad for the student.

This semester, I have been amazed to observe the way my students behave at the first mention of extra credit.  They remind me of my kitten, a food hound, who runs as fast as he can to his food bowl when he hears me pouring in his kibble.  Part of what amazes me is the fact that these are the same students who, as a group, won't follow any of my advice about how to prepare for exams.  I can tell them that preparing for class regularly, attending class, and participating in the class activities will raise their grade, on average, 10 points and they don't want to hear it.  But they will expend a tremendous amount of effort for two points that are guaranteed.  Among other things, my observations have reassured me that the way to approach this group is to reallocate all those midterm points to quizzes, homeworks, and other "small stakes" assessments.  They have to see the immediate payoff (woot, I got 2 points for doing my  homework today) in order to behave rationally.  If I simply tell them that doing their homework daily five classes in a row will add ten points to their grade, they wouldn't do it.

This seems to be a generation whose pleasure and reward circuit is wired entirely different from my own.  I find it difficult to relate to an audience of students who require constant, tangible incentives to behave rationally.  At the same time, it's my hope that, in a way, I can train them in how to do this and then, as they progress through college, they will not require quite so many immediate rewards (though half of my students are already upper division students).  I also suspect that I am encountering a large number of students from the sciences who are accustomed to daily homework sets.  They have never been taught how to learn in a humanities class and, for various reasons, aren't inclined to listen to the advice of the instructor.

Failure, mine and theirs

It is that dreaded time of the semester when students in danger of failing the course (but for various reasons unable to do a late drop) come to my office to plead for mercy.  They ask for opportunities to do extra credit (there have already been several during the semester) or for a change in how their grade is calculated (utterly oblivious to the fact that I can't give them special treatment without offering the same treatment to every other student).  Rarely do they take personal responsibility for their failures unless I ask probing questions that force them to confess their lack of study, class attendance, etc.  Often, they will assert that they can only learn in a "normal" lecture format and that the flipped classroom harmed their ability to learn the material.  This might be more persuasive if I didn't also have a lot of data about these students and their learning habits.  After they leave my office, I look at their attendance; I look at the specifics of their i>clicker answers.  What I see confirms my suspicions: they didn't do well because they missed a lot of classed and, when they were in class, they clearly had not prepared and missed most of the i>clicker questions (and surely were not able to make heads or tails of the discussion questions).

It also frustrates me that they wait to come see me until the point in the semester when, in fact, I can do nothing for them.  It is too late to change their learning habits (at least for my course). It is too late for me to suggest that they work closely with me or a TA in the weeks leading up to the exam.  Indeed, I suspect this is precisely the point.  They come to ask for special treatment and favors, but they don't want to be asked to do anything in return.  The same lack of effort that got them in the position of risking a poor grade is precisely what drives them to take action only when it is too late for me to do anything to help them.  They leave my office feeling dissatisfied and I am left feeling disheartened and frustrated (and telling myself that I need to stop taking their failure to engage so personally). 

As classes wind down and the final exam looms ever closer, I feel nervous for them.  The final is worth a significant amount (25%) of their final grade.  I am sure that many of them are counting on doing substantially better on the final than they did on earlier exams.  If history tells me anything, it's that they will do worse.  Students almost always end up doing worse on final exams because they get distracted with end of semester festivities and blow off studying--despite the best intentions.  I've seen this repeatedly over 10 years of teaching at UT and it's one reason I only reluctantly give final exams during finals week to large enrollment classes.  The final exam isn't difficult.  It will be an abbreviated midterm (albeit covering some pretty complex content) and then a set of ethics-related activities, including an ethical analysis.  If they put in the work, they should do extremely well. If they wait until the night before the exam, they will not do very well.  I am letting them talk about the ethics part with one another and I expect to get many exams that are ok but largely students trying to repeat what they read from a Facebook group but didn't really understand.

I worry that I am going to end up failing more students than usual in a class that is supposed to improve student learning.  The reasons for this are clear--they didn't hold up their end of the deal.  They didn't learn despite an abundance of learning tools at the disposal, largely on demand.  The exam questions are handpicked from in class discussions, which are based on the recorded lectures and textbook readings. There are no curveballs.  Still, I can't help but feel like I also have failed as an instructor and motivator every time I fill in that F bubble on the grade sheet.  Rationally, I know the argument about leading horses to water; but it is difficult not to feel that I should have been able to find some way to make them want to drink that water.  Hopefully, once we do all the math, it will turn out that most of them were able to squeak by with Cs or Ds.  I also suspect that it's generally true that, the larger the class, the higher the failure rate--another reason I don't really like teaching 400 students.

As I worry about their grades, I have also been thinking about how I'd grade my own performance.  Perhaps I am being hard on myself, but I'd say an A+ for effort but about a C- for execution.  I made several rookie mistakes in the class design, but the most significant was designing in class activities around the expectation that the students had done their assigned homework but with no grade attached to that homework (and no way of checking, even with a quick i>clicker quiz at the start of class). I assumed an ideal, self-motivated student as my audience (aka me when I was in college).  Of course, that vast majority of my students didn't behave that way.  This meant that in class review and discussion was virtually impossible and led me to eventually remove the required attendance policy.  It was a disaster to have 400 students, 300 of whom were unprepared and totally lost, sitting in a room.  They chatted with their neighbor, left to get a snack at the student store across from the classroom (and then returned, sometimes bearing ice-cream cones), and were generally disruptive.  I felt like I was teaching 9th graders.

In retrospect, I was totally unprepared for the audience I was teaching.  In earlier, lecture-based versions of this course, I could envelop myself in a bubble of denial.  I could focus on the students who had genuine interest in the course and ignore the fact that most of the class was there to get an A or B and check off a core curriculum requirement.  I persuaded myself that I could get these students to engage simply by providing opportunities for engagement.  I didn't grasp that I would have to force that engagement, at least at first and probably for much longer.  I did not understand that many of them would take pride and pleasure not in learning but in bragging on Facebook about how little effort they are putting into the class.  I was utterly unprepared for their strong reaction against change (never have I heard so many praises of the class lecture; typically I hear students say that they CAN'T learn that way because they can't follow the lecturer, etc.).  I was  caught off guard by the way they used the flipped model as an excuse for not working.  It seemed to me that they decided that they couldn't learn from watching pre-recorded lectures and so therefore didn't need to watch them.  And when they subsequently did poorly on the exams, well, that was the fault of the course model and not a consequence of their study habits and lack of preparation. Over and over again, they are given opportunities to improve, but many of them would rather complain that my expectations are ridiculous (I mean, who can expect a lecture class to include discussion?); that the grading is to harsh (because we expect the actual, correct answer and not some vague approximation of it); and that they should be doing better--even when some of them are also proud of how little effort they are expending. The logic baffles.

I couldn't make the kinds of mid-semester adjustments that I wanted to make (and that the class needed) thanks to the Texas legislature, which demands that syllabuses be published on the first class day and remain unchanged.  This is largely a good thing; but it does mean that, when instructors are testing out new course designs, we are stuck with a flawed design for the entire semester.  This is tough on instructors as well as the students, especially the students who wonder why I am not making changes to a model that is clearly not working for a lot of their classmates.  I was motivated to change the structure of the course in the first place because I hated feeling like I wasn't really teaching them much of anything; and they certainly weren't learning to do more than memorize a bunch of facts and then dutifully spew them back to me on exams.  I totally failed to understand that I couldn't force students to learn anymore than a coach can play the game for her team.  All I can do is create a learning environment that motives student learning.  On the whole, I didn't do that as effectively as I had hoped, in large part because I failed to understand how much structure and grade-incentivized assessments would be required to get them to abandon their "cram for the exam" mode.  On the bright side, I have a very good sense of what steps need to be taken to shift the focus to daily preparation and away from exams.  It remains to be seen whether this shift will lead to increased student participation in and enjoyment of class discussion.  It also remains to be seen what it will take to persuade students that, to do well, they will have to actually put in some work and perform at a reasonably high level.  To my mind, these last two issues are probably much more deeply rooted in pre-college preparation and are going to be much more difficult to change.

Tuesday, December 4, 2012

Midterm #3: A New Record

My Rome students made a significant contribution to a new record for number of views of Echo recordings on a single day.  Unsurprisingly, the day they set this record was the day before the third midterm.  I wasn't surprised.  After the second midterm and in part in response to a lot of grumbling from students that they didn't find class a good use of their time (unsurprising since 75% of them were unprepared and so unable to really participate in the various polls and discussions), I decided to make attendance optional.  I expected a big drop off: November is a very busy month for students (and faculty) and they knew that everything for the Rome class was available on demand.  In fact, most days saw about 25% of the enrolled students in class.  It was largely the same students who came for every class (not coincidentally, several of these same students have test averages over 100).

I wasn't that surprised that students decided not to come to class.  When I recorded my lectures last fall, attendance dropped to about 30% of the class most days.  I wouldn't say that there was a significant difference between the two cohorts in this respect.  If given the opportunity, a lot of students will opt not to come to class, regardless of what is happening in the classroom (lecture or review/discussion).  This semester I saw very clearly just how poorly most of my students manage their time; and the depths of their denial about how much work they have to do to catch up.  When I offered the attendance "opt out", I expected that 70-75% would take it.  But I also expected that most of them would view the recordings of class at some point, perhaps even regularly.  In this respect, I was completely deluded.  While I don't have data from last fall showing me WHEN the students viewed the recordings of class lectures, I do have it for this semester.  What I saw was disheartening.  Despite repeated warnings not to save everything for the last few days; and despite being told repeatedly that 90% of the exam questions were taken from the material covered in class (about 50% are taken word for word), they apparently did no work whatsoever for 2.5 weeks and then attempted to cram everything in at the end.

I have a handful of altruistic students who have made a number of different study aids intended to help students review the lectures.  They also collect all the iclicker questions from class.  I am sure that many students skimmed the posted PPTs of lectures and then went to these study aids.  But, in various ways, these study aids were insufficient (notably, they didn't include images of architecture, which played an important role in the last third of the class).  The exam was on the Monday before Thanksgiving.  Starting on Wednesday, I received daily updates of the viewing stats; and over the weekend I received them every 12 hours.  As expected, they skyrocketed in the 24 hours before the exam.  Also as expected, many of the students did not make it through the entire list of assigned lectures.  This ended up causing them a lot of problems since the short answer portion of the exam put more emphasis on the later lectures (I wrote the exam before seeing these stats and had done this in part because I assumed that these would be easier questions for everyone, and especially for those who had been in class).

The stats for this exam aren't really comparable to previous cohorts because I also had a reasonably large group of students who were earning As or Bs blow it off (I allow them to count their lowest midterm for 5%--something I won't do again).  But, overall, the performance on the exam was dismal.  Typically, students score about 10 points higher on this exam than on the second midterm.  This group actually scored lower on this third midterm.  The reasons for this poor performance were apparent from their learning habits.  As several of them have subsequently explained to me, they had exams for their "important" classes (calculus, chemistry, computer science, etc.) during the same period and so put the Rome class on the "back burner".  Interestingly, though, they think that the problem is that the class is too hard, not that perhaps they should accept the consequences of not keeping up in a reasonably demanding class.

If I ever had any doubts about my plan to overhaul the assessment structure for future cohorts, they were laid to rest when I watched the train wreck that was the third midterm.  I also realized that I will never be successful in changing student learning habits by talking to them and trying to reason with them.  I have to simply put a structure in place that gives them constant feedback and forces them to engage or drop the class.  If I don't want them to cram for midterm exams, I need to put less weight on midterm exams and start attaching grades to their daily assignments.  Many of my students come from the natural sciences, engineering, vel sim.  They are used to doing daily homework.  It will be a change for them to do that in a large enrollment, humanities class, but I think they will settle into the habit relatively fast.  One of the biggest lessons of this semester has been that students will repeatedly make bad decisions if the negative consequences of those decisions are not immediately apparent.  In some sense, they WERE apparent when they came to class and couldn't engage, but their response was to devalue the classroom experience rather than to change their behavior. 

Monday, December 3, 2012

Changing Student Learning Strategies

In a recent op-ed in the New York Times, David Brooks took on the challenging subject of how people change (  It has generated a large number of comments from readers (in part because he chose to begin the column with the controversial story of a largely deadbeat dad who got fed up with his children's failures and so sent them a letter in which he disowned them).  One particular paragraph jumped out at me, as I wrap up Phase I of transforming my Introduction to Rome class from lecture-based to a student-centered class.  Wrote Brooks, "People don’t behave badly because they lack information about their shortcomings. They behave badly because they’ve fallen into patterns of destructive behavior from which they’re unable to escape."  This describes perfectly what I've observed about my Rome students, much to my frustration.

At the very start of the semester, I had the students watch a short recorded video titled "How to Avoid your Own Decline and Fall"--basically, tips for success in the class.  I discussed in detail how to watch the videos (take notes!  watch them in a quiet space that allows you to focus!) and gave advice on a range of issues like attendance, keeping up with the assigned work, etc.  We then duly discussed this video in class.  Yet, when push came to shove (i.e. when midterm season hit in about week 6 of the semester and, suddenly, the students' time management skills came under the gun), they ignored every single thing I had told them about how to succeed in the class.  As soon as we handed back the first midterm (on which scores had been well above the usual median and average, in large part because of the extraordinary effort put in by the students to prepare in addition to the extra practice with the course content during class), I let them know that the second exam was going to be significantly more challenging.  I told them the historical stats on the exam, in part to challenge them.  I reminded them that it would not be possible to cram and do well on the exam for all but a few of them.

75% of them ignored me.  They convinced themselves that what I was saying wouldn't happen to them.  Perhaps they thought they were the exception.  Mostly, though, I think they knew that their behavior was not going to have good results but they couldn't change.  Repeatedly, I've had students in my office telling me that they wished that *I* had done something to force them to stay on track with the assigned work.  When I asked them whether they thought that it was a fair of me to expect a college student at UT to be able to follow a syllabus and manage their time, they conceded that it was; but then repeated their desire to have me provide an external structure that would take the choice away from them.  The scores on the second midterm were right in line with what they were for cohorts who took the class as lecture-based--surprising given all the extra learning opportunities and study help that the current cohort has.  Once again, I repeated my advice to avoid cramming.  Many of them would have seen the negative results of their efforts to cram.

I expected that at least some of them would see the second midterm as a wake-up call and mend their ways.  Nope.  If anything, their behaviors worsened.  They stopped attending class (it was not longer mandatory) and seem to have completely checked out.  They then tried to cram for the third midterm in the 24-48 hours before the exam, during a week when they likely had several other exams and projects due.  The results were abysmal.   Despite being told repeatedly that their strategies would not work; despite being told what strategies WOULD work, they refused to change their ways.  At the same time, many of them wished that I would force them to change. The issue wasn't that they didn't see the problems with their learning strategies; it was that they didn't have the self-discipline to change them when they were feeling pressure from other directions.

One of the major changes in the spring is a result of my realization that I cannot bring about change in student learning strategies simply by reason (or data).  They hear it but they don't believe it, and they don't have the self-discipline to change (not unlike the diabetics who refuse to be compliant despite knowing the short- and long-term consequences for their health).  They need someone else to impose structure and discipline on them.  On the one hand, I feel like this is a skill that college students should have.  On the other, I acknowledge the reality that they don't and recognize that they only way they are possibly going to learn it is through experience.  I have begun to think about my class--one which will be taught in the fall alone in future semesters--as a kind of bridge course between high school/community college and UT (though, to be fair, it enrolls as many upper division as lower division students at present, something I am hoping to change in the fall).

One of the main course objectives going forward will be to teach students good learning strategies.  I will do this, in part, by imposing a lot more structure on the course.  There will be homework modules that must be completed before each class.  There will be weekly quizzes.  Emphasis will be shifted somewhat away from midterm exams and towards the day to day learning.  There will be a comprehensive final, but it is my hope that students will learn experientially that doing assigned work and studying weekly makes it much easier to learn and apply larger chunks of content.  I am also reading a lot about how to motivate and sustain change.  The message seems to be that it happens by reinforcing positive behaviors, so I am brainstorming about ways to do that.

Perhaps the clearest lesson from my current cohort of students is that I can't reason with them to change their learning strategies.  Even the experience of doing poorly on exams does not persuade them to change their behaviors.  Instead, it persuades them that the "flipped" model must be to blame; or my expectations must be too high; or the tests are too hard; or the lectures that they can't learn except from a professor lecturing in a classroom.  Only about 10% of them will even begin to confess that they bear some responsibility for their less than stellar performance.  The new tactic will be to force them to change their learning strategies (or quickly realize that they are going to fail the class) and hope that, somewhere in there, they will figure out how to manage their time and be able to adopt these strategies without the external, grade-driven structure.

Am I going to have to take more classes like this?

In a class of 387 students, I don't have the chance to talk to many of them.  I had planned to invite smaller groups of them to meet with me during the week over cookies and chat about the class, but the semester had some unexpected challenges from other quarters and I didn't have the time to do this.  It is something I am going to be sure to do in the spring, now that I realize just how skeptical so many of them are of any kind of pedagogical change (especially the kind that seems to require more engagement on their part).  But I do have students come by my office hours from time to time, and I always try to get them to talk about their experience in the class.  Some of them really like it, others clearly don't.  One common question I get is something like "are all classes at UT going to be taught like this?"

Faculty are feeling a bit on edge these days as we hear about plans for extensive transformation of introductory level, large enrollment courses.  The UT System is spending a lot of money on this, and UT's recent deal with EdX is clearly intended to allow faculty to develop a range of teaching tools that can be delivered in a range of teaching environments, from a huge MOOC to a 100 person, campus-based group of UT students.  As nervous as faculty are about what all this will mean for their teaching (and the time such transformation will require), students are perhaps even more nervous.  They may complain about large classes.  They are certainly not learning at the rate one would hope to see from college students, and I think most faculty would acknowledge that there is a "student learning" problem on campus.  They seem to be less prepared for college than ever and even very intelligent students lack basic learning habits that most faculty aren't prepared to teach (e.g. how to read a textbook).  But, if my own sample of students is any indication, students are not going to embrace blended learning approaches without complaint.  They may complain about their boring lecture class, but they complain more about the large class where they are expected to pay attention and stay on task and engage.  From their perspective (and I've heard this from more than one student), they are paying for their college experience and should get whatever they want.  All to say, a significant part of transforming large enrollment courses, at least for the first few, transitional years, will be to also transform the students who are taking them.  In particular, it will require these students to understand that they are going to have to work--and work consistently--in all of their classes; and that they may not always get the A or B that they feel they deserve.

If I am being honest, I expected that the students in my Rome class would be so excited to have the chance to talk and engage that it would motivate them to continue to prepare.  In my head, as I was taping the lectures during the dog days of summer, I imagined them eagerly engaging with the course material.  Instead, they behaved exactly like previous cohorts of students (x2): about 25-30% actually took the class, did the work, and did well.  The rest of the students crammed, looked for shortcuts, and probably retained very little.  We won't know whether that is actually true because there's no comprehensive final, but I'd bet some good money on it.  It was a huge disappointment to spend so much time reworking the class and then have the majority of the students make it clear that they only cared about their grade, not about learning the material.  I am optimistic that I can shift these numbers, so that more like 65-70% are truly engaged.  I accept that there are 30% that probably can't be reached, no matter what I do.  They are the ones who are in college to check of boxes, get a degree, and make a lot of money.  My class is just a box to them and it isn't going to make them a lot of money to know Roman history.  But there's that middle group that I think can be reached, who can be persuaded that they are lucky to be taking a class "like this one" and who will find themselves getting genuinely interested in the subject.  We'll see...

Sunday, December 2, 2012

A late drop

I hate the last few weeks of the semester when I am teaching a large enrollment class.  The third midterm has finished up and, finally, students who are doing poorly are forced out of their state of denial.  Mad grade calculations ensue as they work out exactly what grade they need to make on the final exam to get their desired grade in the course.  They finally come to my office to express their concerns about their performance and to ask how they might do better (they have been encouraged to do this since the first midterm).  A few of them realize that they either can't get the grade they want; or don't want to put in the work to get the grade they want, and so do late drops.

I had one of these students in my office on Friday.  I didn't recognize him, but that just means that he doesn't sit near the front and probably doesn't attend class all that regularly.  He had done fairly well on the first exam--a B--but then had earned Ds on the next two exams.  He could still get a C and possibly even a low B if he did particularly well on the final, but it was clear that he just didn't want to invest the time.  I signed his form, but also started a conversation with him about his experience in the class.  In particular, I was interested to hear why he felt that he didn't do as well as wanted.  Part of what stimulated my questions was his own: "Is this going to be the way all lecture classes are taught?"

I hear this a lot from my students.  I have to confess that it comes as a shock to me that so many students have reacted so strongly to the decision to use class time to review and discuss content delivered outside of class.  They insist that they "learn better" when being lectured to in class (something I find difficult to accept when I sit in on other large enrollment classes, focusing entirely on the behavior of the students, and observe that almost nobody is engaged in the lecture; many are texting or playing games on their phones; and they write down only what appears on the Powerpoint).  They say that they can't learn from lectures they watch at home--lectures that are about 20 minutes long and based closely on the textbook.  I suppose what I find so bizarre about this is that, for generations, students have been asked to read textbooks outside of class.  I fail to understand how it is particularly different to read a textbook or view a textbook-based lecture.

As I probe more deeply into these complaints, the truth emerges: it's just too much work and effort to do assigned homework and then come to class and think about and apply the knowledge gained.  As I realize this, I feel incredibly frustrated.  Certainly, I will be making some major design changes to the class for the spring to address the most significant issues that came up this fall.  But I can't force students to work.  One of the "complaints" of the student who was late-dropping the class was that I had so many different ways to try to get students engaged and he didn't really want to engage.  He just wanted to come to class, be told what to learn in a clear and uncomplicated way (he mentioned another lecture class where the professor put up an outline and key terms so that students knew what "mattered" in the lecture), and the regurgitate it on the exam.  This was what he told me.  I wasn't surprised, but I was pretty sad.  He was clearly an intelligent person; but he was also a transfer student who didn't have great study skills and struggled to learn more than 2 weeks of material at a time.  The new course design will help some with this issue (this student isn't the only one who struggles with it), but I also believe that one of the missions of a college education is to teach students to take in and digest large amounts of content.

Students are going to be doing a number of different surveys about their experience in the class.  But it's clear that the majority of them would have preferred a standard lecture class.  By this, they mean that they would have preferred a class that couldn't deliver as much content or hold them responsible for knowing that content.  When I designed the course, I assumed that most students would care about learning.  What I have learned is that, in this particular class (a core requirement), most don't.  They just want a grade (that is, an A or B).  The challenge going forward will be to see if there are ways to get them to care about learning at least as much as they care about their grade.  I think I can do it, but this semester has definitely provided a wake-up call of sorts.  In the past, I wasn't as tuned in to the students and their learning habits.  What I've seen this semester has highlighted just how great a challenge it will be to transform larger enrollment classes at UT.  It's not just a matter of getting faculty on board.  Teaching the students how to take such classes, and designing courses that force them to do the assigned work, will be key.  I suspect that there will be a lot of kicking and screaming during the transition period and faculty are going to have to learn to tolerate that (it sucks, by the way).  Those who evaluate us are going to have to understand that flipped and blended courses are not going to please students who expect a large enrollment class to require little effort or engagement.

Friday, November 9, 2012

Loopholes, Rational Behavior, and Learning

Tuesday was the last day that students could either late drop a course or change their grading option to Pass/Fail.  As expected, a handful of students wanted to late drop.  Less expected were their various reasons for doing so.  In about half of the cases, they were in danger of failing and had realized that they didn't have the time to raise their grade in the Rome class while still doing well in their other classes.  This happens.  But then there was the woman who had not taken either midterm nor had she attended a single class.  I didn't know this when she appeared in my office and so, making small chat as I signed the form, I said that I was sorry that the class didn't work out for her.  "I'm just so busy this semester and it wasn't very convenient for me."  This comment was odd enough that I looked up her grade record and discovered that apparently she was too busy to ever show up or even drop the class early enough in the semester that she might have received some of her tuition money back.  Of course, some students do this so that they can claim to be full-time students for various reasons--I suspect that was what was going on with her.

The other 50% of the late drop forms were for students who were earning Bs or high Cs--that is, for students who would clearly pass the class and, in most cases, would have a good shot at earning at least a B.  In some cases, they might even have ended up with an A (but it would have taken a focused effort and commitment to the class).  Several of these students were either Business Honors students or students hoping to transfer into the Business program.  It concerns me that UT enables this sort of gaming of the system.  Certainly, it makes sense for a program to have GPA requirements.  But it is absurd for students to be unwilling to take a B in a class because they feel that it might jeopardize their future as a business student.  This is a terrible way to educate students; and goes a long way to explaining why students don't graduate in four years.  But, as with all things, the students have learned how to game a system that allows them to do that.

I also was made aware of an unfortunate loophole in my current course.  When I redesigned the class, I took out the comprehensive final, in part because I was working on the assumption that most students would be engaging weekly in discussions and it would be enough to test them over each section.  This was a terrible mistake for many reasons and I compounded it by allowing them to count their lowest of the first three midterms 5% (this was a way to avoid make-up exams but also to help students who did poorly on one exam but fine on the others).  These same students who won't do assigned work for a class apparently spend hours figuring out how to get out of work.  Just about everyone who has an A average on the first two exams is completely blowing off a third of the course.  They will pay for this somewhat on the fourth midterm and final writing assignment, but not as much as they should.  As well,  Because I did not directly say in the syllabus that all students had to complete the final writing assignment with a passing grade to earn a pass in the class, I have two students who figured out that they could pass the class without doing any work after the second midterm (they are both earning low As). 

All of these loopholes will disappear in the spring version; and I suppose I should not be surprised to see some students making these choices when the course design permits it.  In a class about ethics, though, it somehow feels like a double-whammy: first, I was a fool for assuming that college students would care about learning; second, I was a fool for creating this loophole.  When I think about the time I spent on the course re-design, and decisions I made that changed the syllabus from my traditional lecture class, I realize that I was operating under the assumption that the students would actually want to learn.  To be fair, many of them do.  And most that don't aren't in a position to simply stop working.  Still, it makes me sick that someone can pass the class without doing crucial assignments and demonstrating the course objectives.  That's entirely my fault and thanks to Texas laws about syllabuses as contracts, there's nothing I can do except fix it in the spring and, in the future, not assume that my students share my values around learning and education.

Monday, November 5, 2012

Quo vadis?

One of my favorite observations about education was made by the cultural historian and media studies scholar (and UT Austin grad) Siva Vaidhyanthan.  He notes, "real education happens only by failing, changing, challenging, and adjusting.  All of those gerunds apply to teachers as well as students.  No person is an "educator", because education is not something that one person does to another.  Education is an imprecise process, a dance, and a collaborative experience."   I love this quote in part because Dr. Vaidhyanthan knows what a gerund is; but also because it captures what I think is the essence of what I want to be doing when I interact with my students.  At the same time, it describes an experience that strikes my students as entirely foreign.  They expect me to educate them, and for many of them, they want this process to be as painless as possible.  Many of them don't want to take part in a collaboration, and they definitely don't want to be made uncomfortable by being challenged or asked to adjust.  In class on Wednesday, the last day that I required attendance, I put this quotation up and spoke about it a bit.  I explained that their learning depended on their willingness to engage.  I described the different kinds of learning tools that I had made available to them--a textbook, recorded lectures, practice questions, a discussion board, office hours, and in class exercises--and then told them it was up to them to decide how they wanted to use these tools to engage with the course content.

On Friday, about 25% of the class showed up.  Fridays have been optional all semester and attendance has generally run at about 175-200 students, depending on what else is going on.  Yesterday, about 100 students were in the room.  I hope that this was a reflection of the fact that it is the start of a new unit rather than a new normal.  I also realize that, yet again, I'd need to make some adjustments to what we were doing in class, to make it clear what the benefit of being in the room was.  I will post some extra practice questions, but in class will be almost exclusively peer interaction.  As I walked back to my office after, in essence, delivering a lecture on content that they should have already watched, I realized that I was still working on the assumption that most of them were unprepared and unable/unwilling to discuss the material.

What also strikes me, though, is the extent to which I feel like I am in a tug-o-war with my students.  The course design is such that it works on the assumption that many of them can participate in a class discussion.  Yet they have figured out that if they simply refuse to prepare--and that if enough of them do this--it clogs up the gears and leaves me in a kind of no-man's land where I am stuck back in some kind of imperfect lecture mode.  I am unhappy and feel like I am just saying things that I've already said (on tape) elsewhere.  It looks to some of the students like they aren't learning anything new or really doing application work (and they aren't entirely wrong about this).  Most of all, I feel like I, too, am lapsing back into what is the more familiar and comfortable mode of large enrollment class teaching: the lecture.  Sure, these lectures are more interactive.  But, really, they are lectures and not discussions.

This weekend, as I prepped class for Monday and Wednesday, I made a concerted effort to change things up.  We will start with a few i>clicker questions that let them test their mastery of material covered in the previous class.  These questions are purely for their own purposes (and to give us a record of who was in the room) and they will be instructed to use them to highlight areas that need additional review.  Otherwise, everything is focused on group discussion and is asking them to think about material they should have learned, asking them to turn it over, examine it, retrieve it, and otherwise work with it.  THIS is what will be most beneficial to their learning and THIS is what makes it worth showing up to the classroom at a specific time.  I am going to be ruthless about not lecturing, about not using class time to simply repeat factual material that is covered in the lectures or textbook.

Teaching really is a dance.  This semester has taught me, as well, that it requires constant adjustments but also mindfulness.  I have to watch what I am doing, and examine my reasons for doing it.  I realized that a big source of my own frustration was that, in effect, I was enabling their lack of preparation by lapsing into lecture mode instead of standing firm and working extra hard to think about interesting ways for them to apply their learning.  I've also realized that I need to work harder on the in-class portion of the class, to make it interesting and clearly focused on application rather than regurgitation.  That way, the clear message is sent that, at least when it comes to the structure of the class, I will not hold their hands no matter how much they kick and scream and refuse to prepare.  I will be better prepared for the resistance in the spring, but also better prepared to stand firm and know that I am well-armed with engaging in-class activities.  Of course, this also means that a big part of the time between semester will be devoted to thinking carefully about how to re-work the in class part of the course.

Friday, November 2, 2012

A Tale of Two Classes

The University of Texas System recently created an Institute for Transformational Learning and hired a historian from Columbia, Dr. Steven Mintz, as director.  This roughly coincided with the public announcement that UT had joined EdX and would be getting into the business of creating MOOCs and other types of online learning experiences.  But it isn't all about designing online courses here at UT.  There's also a real interest in using the tools and platform of EdX (as well as home grown tools and platforms) to design and teach innovative hybrid courses.  I sat in a meeting today hearing about some of the plans that Dr. Mintz has for the UT System.  It was hard not to get excited about the future and to feel lucky to be an academic at such an interesting and challenging time.  Certainly, when I arrived in Austin a decade ago I never imagined that I'd one day be standing on the frontiers of pedagogy, trying to figure out how best to teach UT students using the tools that are now available to us.

One thing that I keep coming back to during conversations about innovative teaching, though, is the fact that they always focus on the design side, on the instructor.  Little thought is given to the consumer and the behavior of that consumer.  There seems to be a general sense that students will eagerly embrace the opportunity to study in the learning environments we are being encouraged to create.  I do think that will eventually be true.  But I also think it is going to take some serious effort--and the passage of time--to persuade our students that a blended or flipped class is doing a better job of helping them learn.  In part, this is because some of them (perhaps more than some of them) aren't that interested in learning.  They care about grades, and would prefer to get an A with as little effort as possible (especially in a core course that isn't part of their major and isn't in a field in which they plan to take additional courses).  When I was designing my class, I think I had this idealistic sense that they would share my excitement about learning and just be swept away.

For the first 5 weeks of the course, they seemed to be excited and were preparing regularly for class.  Class sessions were vigorous, engaged, and I left feeling energized--something that rarely happens with a large class.  After the first midterm, and after midterm season started, they reverted to form: perhaps 30% were preparing ahead of class and able to really benefit from the class activities.  The rest took three weeks off and then crammed for the exam.  I could watch this happening with something like a sense of helplessness because I was seeing data from them.  In the end, the exam scores ended up right about where I expected: almost identical in terms of average, median, and grade distribution to the previous two, lecture-based versions of the course.  Their learning behaviors mimicked exactly what I imagine were the learning behaviors of the students in my lecture-based courses.  There were the "devoted" students who came to class, did the readings, and studied as they went.  But most came to class infrequently and didn't bother with the reading until the week or so before the exam.  They then spent a week cramming.  It's not a surprise that the outcome was nearly identical.  One of the notable differences, however, is that I had a lot more students with very high grades.  So, in other words, those taking advantage of all the tools of the flipped class are thriving.  In some ways, that makes it worth it.

One concern I have is a much larger number of Ds and Fs than usual.  These are kids who apparently just totally tuned out.  Didn't really do the reading or watch lectures in a way that allowed them to learn the material; tuned out in class because, well, they were unprepared and didn't feel like exerting the mental energy to try to follow the discussion.  I'm not quite sure what to make of this or how to respond.  It may be that most of these are students who figured they would count this exam for their "5% exam" and so didn't really try.  But, even if that is true, it suggests to me that 20% of my students are totally tuned out for one reason or another.  I wonder if at least some of these are the students for whom the discombobulation of the flip is just too much and they respond by doing nothing?  Or they are students who think they are doing the work but don't really understand how to watch a lecture in a way that will allow them to process the contents and don't have me there to discipline them.

Up to this point, I have a tale of two classes: for the first third of the class, the students flipped.  For the second third of the class, the majority tried to behave as if they were in a traditional lecture course even without the in class lecture (this is a behavior that I will pre-empt with a change in the assessment plan for the spring).  I am curious to see what the final third of the course will bring.  I am announcing today that they no longer are required to attend class.  I expect that many of them will consequently treat this like an online class, rarely if ever coming to class.  I am most curious to see how many will access the recordings of class even if they don't come.  But then 175-200 will come on a regular basis.  And those are the A-B students, on the whole.  I am beginning to wonder if this is just the nature of this kind of course: either I "dumb it down" so that everyone gets at least a C; or, if I keep it challenging, I end up teaching basically half of the enrolled students while the other half refuse to engage.  It seems to me that this refusal to engage--something that outsiders might be quick to see as a failure of the instructor but is in fact endemic to our student population at UT--is something that needs to be taken very seriously as we think about transforming courses and making them more active and student-centered.  Not all students want to be at the center of their courses.  Indeed, a good number prefer to remain on the periphery, well out of sight.

Thursday, November 1, 2012

Redefining the Role of the Professor

Yesterday, as I prepared to talk to my Rome class about my job and their job in this thing we call education, it occurred to me that part of the problem is my job title.  I am an Associate Professor of Classics.  I am paid a good salary to profess my knowledge of Classics in a range of environments, including the classroom.  Or, at least, I used to be.  Now, though, I'm not so sure about anything.  In a world where higher education is changing, particularly at budget-challenged state institutions, it's pretty clear that the job description of professors will change no matter how much we kick and scream (students aren't the only ones who resist the changes in learning strategies that blended and flipped classes demand).  We might still do a fair amount of professing, but it will be in recordings for our students to watch on their own time.  Rather, our job will be to create a range of learning tools, including more interactive activities; to teach our students how to use those tools wisely and well; to provide structure and incentives for their learning (i.e. formative assessments); and to give them regular feedback on their learning and learning strategies.  This is an entirely different job description than the one I had when I arrived at UT in 2002, but it's the reality of higher education in 2012 and beyond.

These days I shrink away from calling myself a professor in the context of teaching.  I prefer to call myself a facilitator of learning.  When I'm the professor, students expect me to profess and they expect to passively consume what I am professing, perhaps to write down my professions, and then to  regurgitate them to me as accurately as possible on an exam.  A big reason I started to explore other models of teaching a large enrollment course was simply that I was tired of giving students As for being able to do nothing more than repeat back to me what I said in lecture--but also feeling that I couldn't expect them to apply that knowledge if I wasn't providing opportunities for them to do so in the classroom.  Certainly, I want my students to have a firm grounding in the facts of Roman history and culture.  But, to my mind, my job has to go beyond professing and extend to prodding my students to play with that knowledge, to use it to do higher-order thinking.  It doesn't take someone with a PhD in Classics and a decade of teaching experience to do a decent lecture on the basic facts of Roman history (though, admittedly, delivering an organized, clear, and carefully plotted lecture aimed at an introductory level audience is also much harder than it looks).  It's also the fact that, in the not distant future, someone will make a set of lectures based on our course textbook (Boatwright et al.'s A Brief History of the Romans) available for free--it's just a matter of time.  If all I can do is profess, then I'm going to be out of a job.  Or, at least, my graduate students will be.

Fortunately, I can do a lot more than profess.  I can facilitate learning.  I can create learning tools besides textbook-based lectures.  Really, even though my students don't quite realize this right now, THAT is what they are paying for.  Of course, these radical changes in teaching mean that universities like UT are going to have to invest serious money in retraining their work force.  In re-designing my class, I have worked with a team of learning specialists and IT folks, but have also done an enormous amount of work on my own.  All in all, it will be 6 months of intensely focused work and more time fine-tuning and sustaining the class.  I do believe that the spring version will address the most significant issues that came up this fall.   Of course, to address those issues means spending the break between semesters working intensively to create additional resources for the students to help them manage the demands of the course.  .

For me, it's a really exciting time to be an academic.  I like solving problems, I like teaching, and I am excited about the conversations and changes going on in education right now.  I think there's every reason to be optimistic about the future of higher education in Texas.  At the same time, it's important for everyone to grasp that these changes will also be challenging, for faculty and for students; there will be resistance from both expected and unexpected quarters.  We are entering a period of experimentation and that's a great thing.   It should be encouraged and supported.  We won't get it exactly right the first time, but as more people start experimenting, we will learn more quickly what works and doesn't work in general; and specifically here at UT with our student body and our particular types of courses.

Most importantly, though, we faculty are no longer just going to be professors.  We are going to be designers, problem-solvers, coaches, and facilitators as we encourage our students to become "successful users" of our courses.  What will a successful user look like?  That will be the student whose learning improves and who is able to demonstrate a mastery of the course objectives.

Tuesday, October 30, 2012

Student Resistance and Comfort Zones in the Flipped Classroom

Once the shine of the new academic year wore off for my students--and the reality that they would have to work hard for their grades in my course set in--I've been dealing with some pockets of students resistance to the flipped class.  It's not clear how widespread it is.  Certainly, there are some vocal resisters who have contributed to a class FB page set up by the students and for the students (I'm not part of the group and so only hear reports secondhand).  The model has its vocal defenders, but I imagine that many who like it don't feel particularly inclined to announce that on FB or to try to persuade their classmates to change their minds.  Likewise, I imagine that there is a group of students who aren't particularly fond of the flipped class but also don't absolutely hate it.

I've been trying to get a grip on the underlying reasons for this resistance and seem to have landed on two general sources: first, there are students who simply don't want to do the work.  They prefer the lecture model because they can come to class, be spoon-fed, and then not be expected to know very much or be able to do any higher-order thinking with the information they've been spoon-fed.  In the flipped class, expectations are higher and there is more of an onus on them to organize and participate in the learning process.  I have science and business majors who think they shouldn't have to spend so much time on a stupid humanities course that is just a core requirement.  I have students who are perfectly capable of doing well in the course but resent having to spend so much time learning the material so that their GPA isn't adversely affected.  For this group, I don't have a whole lot of sympathy.  Mostly, I will do a better job of letting my future classes know what they are in for so that students who are shopping around for an effortless A can go to another store.

Another significant source of student resistance seems to come from their sense that I have pushed them out of their comfort zone.  I hadn't realized how much this was true until recently, and I hadn't realized how much it was also true for me as an instructor.  Indeed, what the flipped classroom does is force all of the participants--instructors and students--to learn new roles in the teaching process.  I know that I've felt like I was operating out of my comfort zone during our in class sessions.  I have felt good about the job I was doing, but also very aware of how much less scripted class is when I am running a more discussion-based class.  We don't always get through what I planned--in fact, we rarely do.  Every single class, something unexpected seems to happen and I have to think on my feet.  It's such a different experience from walking in and doing my lecture performance and walking out.

I realize now that my students must be having a similar experience.  In lecture courses, they know what to do, what is expected of them.  They think they know how to learn in that environment.  They've come into my classroom and suddenly been told that they need to learn in a completely new way.  The pieces are the same, just in a different order; and there's what I considered the added bonus of getting to practice that learning in class.  From their perspective, however, it isn't at all clear how those pieces fit together.  Yes, I explained all this several times at the start of class and several more times afterward.  I will explain it again this week.  But they are so accustomed to the "lecture in the classroom" model that it's not at all clear to them that the recorded lecture is the same as that, just in a different space.  And it's not clear to them that what they are doing in class is practicing concepts that will appear on the exam.  I know this isn't clear because I see how few of them access the recordings of our classes.  I am going to "source" the exam questions to show them very clearly that they are coming largely from i>clicker and review questions.  I have learned, however, that I have to take seriously the fact that the flipped class isn't just an easy transposition of in class/out of class work.  It can be completely disorienting to them, especially to those who have taken several large lecture courses and have figured out how to do well in that environment.

In my spring class, I will spend some time at the start of the term talking directly about this issue of comfort zones and disorientation.  I will acknowledge that it is a real thing and I will provide them with a lot more guidance in figuring out how to "do" the course.  Sometimes I think we GenXers assume that our students are far more able to navigate different forms of technology, far more flexible about space, than they actually are.  To the contrary, this course has taught me that they are very tied to the idea that learning happens in a classroom and it happens best with the instructor telling them what to learn.  Yet these students will be entering a workplace that will be ever more flexible, both in terms of when the 8 hour day happens and where.  Some and perhaps many of them will be telecommuting on a regular basis.  They will be entering a workforce that values adaptability, flexibility, and critical thinking skills.  Part of my job besides teaching them about ancient Rome, is to get them to strengthen these muscles.  Indeed, pushing them out of their comfort zones and getting them to learn how to adapt to and resolve that discomfort is part of what I need to be doing, and part of what they need to be learning how to do.

I can empathize with their resistance, though.  I chose to make myself uncomfortable.  Most of them had no idea what I meant when I explained the flipped class model to them.  Further, most of them probably have no idea why, exactly, they feel uneasy and unsure of what to do to succeed.  The answer, of course, is simple: just do the outside of class work thoughtfully and come to class and engage.  But first, this requires them to know HOW to do that outside of class work and to grasp in a deep way how it is helping them learn.  In a class that isn't problem-based, it's very difficult to get them to see their learning (or lack thereof) until they take an exam.  For this reason, I think it nearly impossible to flip a large enrollment humanities class (like Ancient Rome) without also having frequent, low-stakes assessments.  They have to be able to see how well their learning strategies are (or are not) working.  In a math class, this is likely to be easier for them to ascertain based on their ability to do a problem.  In non-problem based course, however, it is too easy for them to confuse recognition with knowledge.

Monday, October 29, 2012

Re-defining Learning spaces

Slowly but surely, I am learning that one of the most significant hurdles in flipping a class is the issue of learning space.  When we flip our classes--that is, shift the instruction of content outside the classroom to make room for practicing application inside the classroom--we don't really take account of the fact that we also need to work closely with our students to create appropriate learning spaces.  It's not a surprise that this point isn't highlighted in the literature about the flipped class.  After all, the concept originated in the high schools where parents do a lot to shape and control the learning space of their children.  As we start to implement the principles of blended learning at the post-secondary level, however, we need to spend some serious time thinking about strategies for teaching our students how to learn outside of class.  This is all the more urgent these days, when high schools are moving away from assigning outside of class work.  In many cases, our students are coming to college/university with little experience in learning outside the class and no idea of how to do it (or, even, why they should have to do it).

In the coming years, teaching our students how to learn outside of class is going to be our job.  We can no longer assume that they come to us with good "homework habits".  Even if they did do some outside of class learning in high school, we can't assume that they know how to create a good learning environment for themselves.  My generation (GenX) went to the library or we studied in our rooms.  Sometimes we met in empty classrooms or in the halls of buildings for group study.  We were used to doing homework and we weren't surprised when we had to do homework in college.  We also lived in a world without the internet, facebook, smartphones, and the hundreds of other distractions our students (and we ourselves) deal with on a daily basis.  I took for granted that, if I asked them to watch videos of lectures (20 minutes each) and gave them a short video about how to watch the lectures (i.e. take notes, write down questions), that this would be enough.  I didn't realize that I really needed to spend time talking about HOW to do their homework, how to create a quiet environment that would let them focus, etc.  Too late, I realized that they are probably listening to these recorded lectures while playing on FB, talking on the phone, watching football, etc.

I was floored when I learned that some of them were actively demanding that I lecture to them in class.  This made no sense to me at all.  After all, when I had my in class lectures "captured" using Echo360, students were quite happy to skip class and watch the recordings.  Nobody was demanding that I require them to come to class.  So I tried to analyze what was underlying this demand.  I realized that the answer probably had two parts: on the one hand, they didn't think they should HAVE to come to class AND do outside of work.  It should be one or the other.  Second, many of them aren't learning nearly enough from the recorded lectures because, well, pressing play isn't going to get the information into their brains.  They have the (false) idea that they would somehow learn better if they were in the class, listening to me talk at them (while, of course, they were tuned out, on FB, napping, and so on).  I am coming to think that this strange longing for the in class lecture has more to do with the fact that they feel comfortable with that space--it's a space someone else has constructed for them.  On the other hand, they don't feel at all comfortable with the idea of constructing their own learning space.  I have learned that part of what I need to do with this flipped class--but probably with every class I ever teach again--is talk in a direct and focused way about how to construct learning spaces outside of class.

Sunday, October 28, 2012

Teaching, Learning, and the Space of the Classroom

This week I have been thinking about the space of the classroom and what it means for me as an instructor as well as for my students.  My students have spent 13+ years traveling to a space that is generally populated with desks, fellow students, and some expert who usually stands at the front of the class (the infamous "sage on the stage").  They know that, when they are in this space, they will be expected to do that thing they call "learning"; and they expect that the instructor will teach them.  From their 13+ years of formal education, they have developed very specific expectations about what this teaching will look like and how they will experience it: that expert, probably standing near the front of the class (especially if it is a large class in a UT auditorium), will pontificate on some set topic.  S/he will likely have assigned the students some outside of class reading but few of them will have done it since, well, nobody else is doing it and they aren't going to be expected to participate because, well, the instructor is teaching.

The instructor might pause from time to time to take questions or even to ask a few questions.  The usual suspects near the front of the class will be paying attention, will have done the reading before class, and will know the answers.  The majority of students will slump in their seats, take a furtive glance at the text messages appearing on their phone, and tune out.  Some of them will be recording the lecture for later transcription.  Some of them will be doing homework for other classes; a few will be taking notes on the lecture.  And the lecture?  Well, it will be the instructor digesting the assigned readings, repeating a lot of the information, and perhaps adding a few insights and some context or making some connections that weren't in the textbook.  Early in the semester, students will realize that they can get an A just by having access to the lecture notes.  These days, that doesn't require regular class attendance.  Groups of students will take a class, rotate attendance, and share notes with one another.  As well,  a week or so before the exam, desperate emails will circulate from students who missed most of the class meetings but need the lecture notes (I know about these emails because, often, they forget to delete the email addresses of the instructor and teaching assistants).  Some kind soul, in an act of misguided altruism, will post their notes on google docs and share with the entire class.  They will do the same with any study guides that are handed out.

THIS is the classroom that my students know; it is a space that is comfortable and familiar to them.  They know what their role is and they think they know what my role is.  I am learning this semester that a major source of student dissatisfaction with the flipped class is owed to discomfort.  I am taking their familiar space and redefining it (without their permission, mind you).  I am forcing them to take on entirely new roles in this space, roles that require them to DO something, roles that require them to BE THERE.  In other words, I have not only pushed them out of their comfort zone; I have completely removed their comfort zone (as they see it).  Suddenly, everything they thought they knew about their role and my role has been declared null and void.  It doesn't apply.  Some of them embraced their new roles as active students in an active classroom with an instructor who does something other than lecture at them during class time.  But others of them are angry.  They had mastered the old, familiar system and were perfectly content with it.  They want it back, darn it.  And they are going to kick and scream (figuratively) until it is restored and they are back to feeling comfortable.

When Technology doesn't Keep up with Pedagogy

I spent my Saturday hand-grading six scantron questions from my recent midterm exam.  These were questions that asked the students to mark all correct answers.  In five of the six questions, there were multiple correct answers.  During my reading about assessment and also from my own experiences with a Coursera myth course, I had learned that such questions were highly effective at creating an accurate representation of what students DID know.  They largely eliminate the advantage of being a "good guesser" and having a partial knowledge of course material.  They reward deep knowledge.  Pedagogically, they are a great question type.  For a class of 400 with limited TA support, however, they are a huge burden to grade.  Each question has to be graded, with partial credit awarded for each bubble; and then the points have to be totaled.  And then those points have to be added to whatever they scored on the regular scantron.  In our case, a new Excel document has to be created with the total scantron score that can then be distributed to the 2 grad TAs and 2 student TAs who are grading the exams.  In other words, hours and hours of extra work, all because our scantron machine (apparently) can't be programmed to read multiple correct answers (and because the person running it had no idea if there was a work-around).

I am beyond frustrated.  I plan to continue to include them, but will include a few of them on the short answer part of the exam.  It will add some extra grading for the TAs, but at least will minimize that.  Of course, I now realize yet again why it is that students looking for an easy class flock to the very large courses--they figure it's impossible for us to require them to keep up with the course or to hold them to test depth of knowledge.  They know we don't have the TA support and they count on the fact that many of the faculty who are teaching these large courses are underpaid adjuncts and lecturers who have made their peace with the limitation imposed on their teaching.  It is mind-boggling to me that, in this day and age, a scantron machine can't perform such a simple task that allows instructors to ask machine-gradeable questions that get much closer to measuring a student's real knowledge.

I'd love to hear any solutions people have found; and I'd love to hear about other types of MC questions one can ask.  I do ask the traditional "EXCEPT" questions, which are related to these.  But of course, it's all or nothing.  What I liked about the "mark all of the above" is that it meant they got partial credit.  I suppose I could spread the answers over 5 blanks and they have to mark them in order?  And one of the options is "none"?  But it seems like that would be really confusing, even with careful explanation of how to answer the question.

Tuesday, October 23, 2012

Classroom Co-dependence and Course Evaluations

I've been thinking a lot about the issue of classroom co-dependence: the unhealthy relationship that can sometimes develop between an instructor and a class, facilitated by our needs as instructors for validation of our hard work, good course evaluations for tenure and promotion, and (sometimes) the love and adoration of our students that make up for all those sacrifices we made to get our degrees.  It can be very difficult to remember that we are there to facilitate learning, by whatever reasonable means, and that sometimes that conflicts with being liked by our students or having them give us a high five for our hard work in creating a learning experience for them.  Sometimes it means raising the bar high enough, pushing the students enough out of their comfort zones that they are not too happy with us.  Now, I generally receive high course evaluations from undergraduates and my students generally respond well to me.  At the same time, if I am being honest, I know this is because I've found that sweet spot of pushing them a little but not too much out of their comfort zones; and making sure that, though I ask a lot of them, the grades are still reasonably high (that is, they are in keeping with the average grades at UT).  I know in my heart of hearts that if I pushed them as hard as I probably should, most of them would resent me and give me low course evaluations--not because I am a bad teacher or because they didn't learn a heck of a lot but because they did not like to be made uncomfortable.

Indeed, as I meditate on teaching and students and classroom strategies this fall, I find myself returning again and again to the course evaluation.  At the risk of blaming course evaluations for all that is wrong with higher education, I do think they have done far more harm than good. The information they provide is not especially reliable, especially for demanding courses that award lower grades (aka not an A).  Very few students do more than fill in the bubbles.  We are lucky if a statistically valid sample of our students (in a large class) show up to do them and in small classes the sample is too small to be valid.  Comments are either extremely positive or extremely negative and most don't take the time to write comments at all.  It doesn't help matters that we hand out the evaluations while they are stressing about exams and finals and going home for the holidays or the end of the year.  And, finally, any instructor with a bit of 7th grade math figures out pretty quickly that the key to high course evaluations is a. making sure that students think they are getting As or Bs at the time of doing them (I know faculty who write easy midterms and killer finals to elevate their course evals while still giving them a reasonable grade distribution at the end); and b. making sure that, even if they don't all love the class, that nobody hates the class.  Nothing screws up an average, especially in a small class, like a few malcontents.  Never hand them out on a day when the students who never showed up all semester suddenly come out of the woodwork.  And so on.

Over the years, I've heard hundreds of "tips".  One of my favorites is the instructor who addresses the questions on the course evaluations point by point during the semester, without drawing attention to what he is doing, to tell his students how he is doing an outstanding job of X (returning exams quickly, conveying information clearly, being accessible).  His students, like students well-trained to parrot back what we tell them, dutifully do so on their evaluations of his course.  I don't know whether to be horrified at the manipulation or impressed by the savvy of this.  Mostly, I have taken the position that I don't pay much attention to them. Truth be told, I don't even read them until about a year after the class is over and I rarely find a comment that inclines me to rethink an element of the course (I am taking about undergrad courses, not grad seminars here: different story entirely with grad students).  These days I note that I have just become skilled at avoiding the zinger comments--the comments section is usually blank or says something sweet like "Nice job!"

More recently, however, I have started to wonder if course evaluations were more than a useless annoyance and, actually, something more insidious.  I have wondered if their very existence, if knowing that we faculty are going to be evaluated by our customers based not on the product we have delivered but on some vague sense of their comfort level and happiness, has been hugely detrimental to higher education.  Don't get me wrong: we faculty need to be held accountable for our teaching.  But surely there is some other way to do that?  Annual teaching portfolios?  Demonstrating that our students achieved some agreed-upon outcome? Peer evaluations?  Student evaluations can and should still be part of the package, but they can't be the sole source of evaluating instructor success.  Despite the discourse, students aren't customers and the university isn't a supermarket selling a customer experience.  In an era when data collection is so much easier; when we can now track student behavior, it makes much more sense to focus on what the students are learning and retaining and much less on whether the learning experience makes them as happy as a visit to Disneyland.

I am NOT advocating a joyless classroom--quite the opposite.  But what I AM advocating is a move away from this pernicious co-dependency that too many instructors have with the students, a co-dependency that is nearly forced on us by the situation in which we teach and have our teaching judged.  One of the reasons that students kick and scream at being asked to do reasonable amounts of work is simply that they have learned that it works--that we faculty, like a bad parent, back off and give them their way all too often.  It's no accident that, when they are upset at being asked to do reasonable things or move a bit out of their comfort zone (as with the flipped class), that their first reaction is to talk about what they are going to say on course evaluations.  That is their weapon, or so they believe.

Has the time come to re-think not just the role of the course evaluation but even the ways that we evaluate teaching?  Of course, to do so rigorously and well would require a much greater investment of time on the part of faculty and administration than handing over the task to a room full of freshmen.

Messing with the Boundaries between Play and Work

A commentator on a previous post made a very astute observation about the students in our high school and university classrooms these days: they are more wired than any previous generation; they frequently have multiple devices with internet connectivity; they download all manner of things and watch all manner of things from a variety of internet sources.  And yet it is this same generation of students who struggle to use course management systems effectively (and, often, simply avoid engagement whenever possible).  They will discuss things, but only on Facebook and only if it is an "unofficial" page (i.e. a group that does not include the instructor).  Official course discussion boards (in my class it is Piazza) become a Q&A site.  This is still a huge time-saver in that I am not answering the same question over and over again via email, but it's disappointing that I can't entice them to want to discuss course material unless I attach a grade to their participation.  Finally, some of them are furious that they are expected to watch pre-recorded, short (15-20 min.) lectures outside of class.  The same students who will watch hours of Netflix TV on their computers rebel when asked to watch a few hours/week of recorded lectures.

Many of us educators think we are doing such a cool thing by reaching our students in "their" language, technology.  But I wonder if, in fact, what we are doing is violating their boundaries.  These may well be boundaries that they themselves are not even aware they have.  I wonder if, in making videos and adapting various social media to have educational functions, we are basically taking their sources of play away.  In some basic sense, we are getting into their space and taking it over, draining it of fun and relaxation and re-investing it with purpose and grade anxiety and all the rest.  Now, I don't think we should stop using technology and social media for education; but it might help us understand their reluctance to engage with the education-version of things that they use every day in their lives if we can figure out what it is about, say, you-tube videos for class that causes them to demand a traditional, sage of the stage classroom. 

Monday, October 22, 2012

Greek and Roman Mythology MOOC: Week 4

I was a typical undergraduate student this week: totally swamped and preoccupied with no spare time to watch the videos or do a writing assignment.  I did end up watching the videos on Sunday afternoon and taking the quiz.  I got a perfect score but it felt pretty hollow since I hadn't done the reading and really didn't have the spare brain cells or time to write a very short essay.  Alas.  I am learning a lot about how to do a MOOC by being on the other side of the production.  I am enjoying this one very much.  One thing is pretty clear: short lectures, about 10 minutes long; and not too many of them.  I think there were about 90 minutes of lectures.  That is about right.  I take good notes and of course have read the Odyssey many times in the past.  Still, I am enjoying the fact that I am learning a lot about how to teach the text if I ever had to.  It's been invigorating to be on this side of things--and also a good reminder of how difficult it is for my students to juggle their own multiple obligations.  Like me, they prioritize and let slide what they can let slide. 

Learning outside of the classroom

As I've written about in some recent posts, I am confronting the expected student resistance to the flipped class.  I'm somewhat frustrated that nobody has come to talk to me about in person; or taken me up on the chance to post their complaints anonymously on Piazza.  This silence makes it difficult to know how seriously to take it and also suggests to me that, on some level, they know that they are complaining about the structure of the course because they don't like doing more work--and they don't quite want to take responsibility for that facet of their unhappiness.  It's also not clear how many people are all that unhappy.  On the other side of things, I've had several people email me or come to office hours to tell me how much they are enjoying the flipped class.  All of this makes it really tough for me to find a way to address the dissatisfaction that I know is out there.  I am thinking that I will address it in theoretical terms: "it is common for some students in flipped classes to feel..." and use that to try to open a discussion.  I will also let them opt out of class time in they prefer.  They still have to take the exams and are responsible for the material at the same level, but can opt out of coming to class.

One of the sources of whinging, though, is about the fact that I am expecting them to learn outside of class.  Honestly, I had no idea that this would be a source of complaint.  I assumed that they knew that it was traditional for students to learn outside of class.  Homework?  I am realizing that, in fact, many of them have found ways to minimize their out of class learning.  They don't do assigned readings, at least not until just before the exam when they might skim them.  Maybe they come to class or, more likely these days, they get someone else's notes via google docs.  What is upsetting some of these kids is that I've made it much more difficult for them to get away with not learning, and learning deeply.  It absolutely blows my mind that they are so overwhelmed by a workload of about 3-4 hours/week of outside of class assigned work (lectures + textbook readings) and 90 minutes inside of class (45 minutes x 2 with one other day of optional in class review).  The in-class part is intended to practice and apply the out of class reading and viewing.  It is absolutely shocking to me that this is considered a heavy workload.

I also wonder just how broken every part of our educational system is if we have a generation of students who have somehow been able to get college degrees without doing much out of class work.  Certainly, I am beginning to understand why everyone is talking about demonstrable learning outcomes.  It is clear that student satisfaction absolutely cannot be taken seriously.  It is also clear that we, as a society, need to get serious about how we educate out kids--and, even more, that we get them to grasp the importance of learning how to learn and think.  I worry about what kind of work ethic we are instilling in our kids if they think that doing 3-4 hours of work outside of class (+ extra work during exam weeks) is excessive.  Sure, they will someday be getting paid to work; but it is no wonder that employers are more and more unhappy with the workforce that is being produced by colleges and universities.  Still, if my own experience is any indication, it's clear that blaming teachers and professors is not the solution.  Sure, some of us could and must do a better job of helping students to learn.  But, ultimately, the students themselves have to put in the work; and the system needs to recognize that instructors who are pushing students to learn are, in effect, disturbing a wasps' nest.  There will be stings and buzzing and other signs of unhappiness.  We need to change our evaluations of courses from student satisfaction (though that can still be one element) to learning outcomes.  That is, did a course improve student knowledge?  Did the professor successfully motivate the students to master the course objectives outlined at the start of the term?

I'm Teaching Myself!

Among the criticisms of the flipped course model that are slowly making their way back to me is the complaint that they are teaching themselves because I'm not using class time to lecture.  This particular comment is the one that probably infuriates me the most.  It would have been so much easier--and so much less time and energy on my part--if I had stuck with my traditional lecture-based course.  Instead, I spent hours of time designing and delivering the filmed lectures.  It is unclear to me how the act of uploading a video and pressing play and listening to it (perhaps taking notes on it) is teaching oneself.  Really?  When you read the textbook do you also believe that you are teaching yourself?  Now, if you want to re-frame that comment to say that "I am working harder to construct knowledge", ok.  But, ironically, the place you are working harder to do that is not at home in front of the recorded lecture, where I carefully explain various concepts to you; it is in class, when I am asking you to do some application work.

I grasp that I can't expect undergraduate students to be reflective enough as a group to understand how the parts of the course are working together.  But it infuriates me when they complain that it is just an online class and that isn't what they paid for; and simultaneously complain about being required to come to class and participate in discussions about course material--because of course they don't want to learn the material on a reasonable schedule.  I recognize that these are issues that have to be addressed at the start of a semester with a new cohort.  And that I need to do a better job of explaining this new format to them and also thoroughly explaining the ways in which they aren't teaching themselves; this isn't an online class; etc.  Still, I am flabbergasted.  At the same time, I do understand that, if you haven't done your homework, you probably can't even begin to understand how carefully the in-class part of the course has been designed to identify flaws in understanding; and emphasize via application important concepts.

I just absolutely can't fathom how a student could possibly claim with a straight face that they learned more from lecture in class than outside of class.  I have done in class lectures for years.  When I started recorded them last fall, attendance plummeted.  Clearly those students were more than happy to learn outside of class. So, really, the issue is that they don't want to have to do any heavy lifting with what they learn.  They are more than happy to learn outside of class, but only if that is all they have to do.  I know that I shouldn't be surprised by this behavior, but it makes me sad.  It also shows that, in conversations about why higher ed isn't working very well, it's not enough to blame the faculty for not doing their job.  In many ways, the far more significant problem is that, for most of their lives, students have been taught to exams.  They haven't learned how to learn.  And when asked to do something other than sit in a room, write down what I say, and puke it up on an exam, some of them freak out.  *That* is the biggest problem with higher education right now.  I don't know how we are going to change student attitudes and learning habits except slowly and through a concerted effort to design and implement rigorous courses; and by ignoring student course evaluations until the worst of the growing pains of implementation are over.

Sunday, October 21, 2012

Staying the Course, Sort of

My Rome students will be taking their second midterm on Wednesday.  I suspect that they will do fine on the exam as a class.  The exam isn't easy, but it's testing them over a lot of material that I reviewed in class (in addition to presenting it in the recorded lectures and assigning textbook readings).  Still, it's clear that there is a vocal faction of students who are unhappy with the design of the class.  Specifically, with the fact that it is a "flipped class".  They were fine with it through the first 5 weeks of the semester, but as soon as midterm season started and their time management skills came under fire, they a. stopped watching the videos before class; and b. become much more resistant to the flipped model.  They have a Facebook page where they vent.  I am not part of the group but hear reports, and apparently there is a very active thread titled "I hate the flipped class".  Not "I hate CC 302" or "I hate Dr. Ebbeler" but "I hate the flipped class."

Not a single student has said a word to me (despite being given several opportunities to do so anonymously); but I have good teacher radar and have picked up on the discontent during class, particularly when I ask them to do peer instruction and some number of them take this as a cue to nod off or text their friends.  They don't seem to hate me (or at least that isn't what is dominating the FB discussions).  But some of them clearly can't stand the flipped class model.  One comment was particularly interesting: it was someone who said something about how I was doing so much "hand-holding" (by which I assume "teaching" was meant) that many of them would get As; but that this shouldn't stop them from using course evaluations as an opportunity to complain about the flipped class model.  Otherwise, it might take hold and other classes would adopt it.  In some ways, I feel sorry for this student who is clearly so unaware of what is happening at her own university as to not realize that it's a little late and no matter how much students protest being made to, gasp, learn, the model is here to stay (though, of course, not all instructors will move most content delivery to pre-recorded videos).  I do feel some empathy for these students.  The loudest voices seem to belong to upperclassmen who have been brought up on the huge class=nap time (I mean, lecture) model.  I am taking them completely out of their comfort zone and, in doing so, asking them to be actively involved in the production of knowledge.  That is how learning happens, but they don't like it.  Not one bit.

I confess: I am flummoxed by this.  I mean, I knew that it happened in theory.  I read the research, I gathered that student resistance was a significant issue and that it seemed disconnected to the quality of instruction.  I knew that, with a class of 400 and only 3 TAs, I was going to struggle more with student buy-in and was more at risk of negativity spreading like wildfire.   Certainly, there are design flaws in the course that will be addressed in the next iteration.  But I also worry that some of this is just resistance to change of any kind.  As I noted above, the loudest voices are juniors and seniors who are filling in core requirements for graduation and just want an easy class that doesn't require much effort.  Or they are science major who think that a humanities class should be easy (but of course that they should also get an A).  So they resent that they actually have to show up; and, even worse, that they are expected to have done some work.  One student noted that it was ridiculous for me to expect a class of 375 students to prepare for class (!).  Really?  I do realize now that it was ridiculous for me to expect that they could take responsibility for their education and stay up on their work without some form of external motivation.  But in my Latin classes, where I assign work and then we review it in class, nobody seems to think it's ridiculous of me to expect preparation. 

From what I can discern, the real issue is that the unhappy students don't want to be told how to learn.  And many of them seem to believe that they learn better by sitting in a lecture hall and being talked at for 50 minutes x 3 weekly, not doing the assigned reading, and then cramming a few times/semester for midterm exams.  I suspect that, in a way, this delusion is perpetuated by us professors who have done this because, in doing it, we write tests that are overly easy, simple regurgitation of what we say, and students think that is learning.  I also understand the resistance to, in essence, being asked to work harder and think harder than they are used to doing, especially for a non-major, introductory level class. 

So now I find myself in a pickle.  Many of these comments have spurred me to thinking about how I can tinker with the course design for the spring and I think I have a lot of good ideas.  I realize now that some part of me thought I was going to get this right the first time, that students were going to love it and love Ancient Rome and love me.  I was completely delusional, of course.  My students are reacting the way that research predicts they will.  I am confident that UT understands well that this resistance isn't a reflection of my teaching skills but rather, of asking students to suddenly change the way they are learning and interacting with course material.  I know I will do a better job of letting the students know what they've signed up for in future semesters.  I will also change the assessment structure to incentivize keeping up.  And I will be sure that everything we do in class is application, not review.  This semester, I find myself lapsing more and more towards review of content because I know that so few have prepared.

I am thinking about some major changes for the last third of the semester.  First, I will let anyone who wants to opt out of coming to class.  They can always watch the recordings of class, but don't need to come.  Second, I will make more of a point of using class time for discussion rather than reviewing factual information.  I will prepare three or so questions for discussion and leave it at that.  I think I've been doing far too much hand-holding because I know that so many of them haven't prepared.  So I've been assuming (rightly, of course), that class is their first introduction to the material and have been more reluctant to just give them application questions to discuss.  After this second midterm, though, that is what I am going to do.  I suspect it will be much more pleasant for everyone when the students who don't want to be there but want an A without doing any real work are given permission to stay home.