Wednesday, December 5, 2012

Extra Credit

In general, I'm not a fan of extra credit.  Students, of course, love it.  For them, it's a kind of second chance.  The problem is, they also tend to think of it as "free" points.  They don't view it as a second chance to learn something; or an opportunity to learn content beyond the course syllabus.  Rather, in my experience, they often treat it as something they will sort of take a stab at and then expect to get the maximum number of offered points.  Over the years, I've started offering some extra credit on exams and also to final grades in large enrollment classes.  Partly, it's my way of making up for a question on the exam that may have been unexpectedly difficult for most students; and it's a way of telling them that I won't negotiate on final grades.  It gives them the chance to cover those small gaps between letter grades and leaves me giving them whatever grade they earned without feeling bad for the student.

This semester, I have been amazed to observe the way my students behave at the first mention of extra credit.  They remind me of my kitten, a food hound, who runs as fast as he can to his food bowl when he hears me pouring in his kibble.  Part of what amazes me is the fact that these are the same students who, as a group, won't follow any of my advice about how to prepare for exams.  I can tell them that preparing for class regularly, attending class, and participating in the class activities will raise their grade, on average, 10 points and they don't want to hear it.  But they will expend a tremendous amount of effort for two points that are guaranteed.  Among other things, my observations have reassured me that the way to approach this group is to reallocate all those midterm points to quizzes, homeworks, and other "small stakes" assessments.  They have to see the immediate payoff (woot, I got 2 points for doing my  homework today) in order to behave rationally.  If I simply tell them that doing their homework daily five classes in a row will add ten points to their grade, they wouldn't do it.

This seems to be a generation whose pleasure and reward circuit is wired entirely different from my own.  I find it difficult to relate to an audience of students who require constant, tangible incentives to behave rationally.  At the same time, it's my hope that, in a way, I can train them in how to do this and then, as they progress through college, they will not require quite so many immediate rewards (though half of my students are already upper division students).  I also suspect that I am encountering a large number of students from the sciences who are accustomed to daily homework sets.  They have never been taught how to learn in a humanities class and, for various reasons, aren't inclined to listen to the advice of the instructor.

Failure, mine and theirs

It is that dreaded time of the semester when students in danger of failing the course (but for various reasons unable to do a late drop) come to my office to plead for mercy.  They ask for opportunities to do extra credit (there have already been several during the semester) or for a change in how their grade is calculated (utterly oblivious to the fact that I can't give them special treatment without offering the same treatment to every other student).  Rarely do they take personal responsibility for their failures unless I ask probing questions that force them to confess their lack of study, class attendance, etc.  Often, they will assert that they can only learn in a "normal" lecture format and that the flipped classroom harmed their ability to learn the material.  This might be more persuasive if I didn't also have a lot of data about these students and their learning habits.  After they leave my office, I look at their attendance; I look at the specifics of their i>clicker answers.  What I see confirms my suspicions: they didn't do well because they missed a lot of classed and, when they were in class, they clearly had not prepared and missed most of the i>clicker questions (and surely were not able to make heads or tails of the discussion questions).

It also frustrates me that they wait to come see me until the point in the semester when, in fact, I can do nothing for them.  It is too late to change their learning habits (at least for my course). It is too late for me to suggest that they work closely with me or a TA in the weeks leading up to the exam.  Indeed, I suspect this is precisely the point.  They come to ask for special treatment and favors, but they don't want to be asked to do anything in return.  The same lack of effort that got them in the position of risking a poor grade is precisely what drives them to take action only when it is too late for me to do anything to help them.  They leave my office feeling dissatisfied and I am left feeling disheartened and frustrated (and telling myself that I need to stop taking their failure to engage so personally). 

As classes wind down and the final exam looms ever closer, I feel nervous for them.  The final is worth a significant amount (25%) of their final grade.  I am sure that many of them are counting on doing substantially better on the final than they did on earlier exams.  If history tells me anything, it's that they will do worse.  Students almost always end up doing worse on final exams because they get distracted with end of semester festivities and blow off studying--despite the best intentions.  I've seen this repeatedly over 10 years of teaching at UT and it's one reason I only reluctantly give final exams during finals week to large enrollment classes.  The final exam isn't difficult.  It will be an abbreviated midterm (albeit covering some pretty complex content) and then a set of ethics-related activities, including an ethical analysis.  If they put in the work, they should do extremely well. If they wait until the night before the exam, they will not do very well.  I am letting them talk about the ethics part with one another and I expect to get many exams that are ok but largely students trying to repeat what they read from a Facebook group but didn't really understand.

I worry that I am going to end up failing more students than usual in a class that is supposed to improve student learning.  The reasons for this are clear--they didn't hold up their end of the deal.  They didn't learn despite an abundance of learning tools at the disposal, largely on demand.  The exam questions are handpicked from in class discussions, which are based on the recorded lectures and textbook readings. There are no curveballs.  Still, I can't help but feel like I also have failed as an instructor and motivator every time I fill in that F bubble on the grade sheet.  Rationally, I know the argument about leading horses to water; but it is difficult not to feel that I should have been able to find some way to make them want to drink that water.  Hopefully, once we do all the math, it will turn out that most of them were able to squeak by with Cs or Ds.  I also suspect that it's generally true that, the larger the class, the higher the failure rate--another reason I don't really like teaching 400 students.

As I worry about their grades, I have also been thinking about how I'd grade my own performance.  Perhaps I am being hard on myself, but I'd say an A+ for effort but about a C- for execution.  I made several rookie mistakes in the class design, but the most significant was designing in class activities around the expectation that the students had done their assigned homework but with no grade attached to that homework (and no way of checking, even with a quick i>clicker quiz at the start of class). I assumed an ideal, self-motivated student as my audience (aka me when I was in college).  Of course, that vast majority of my students didn't behave that way.  This meant that in class review and discussion was virtually impossible and led me to eventually remove the required attendance policy.  It was a disaster to have 400 students, 300 of whom were unprepared and totally lost, sitting in a room.  They chatted with their neighbor, left to get a snack at the student store across from the classroom (and then returned, sometimes bearing ice-cream cones), and were generally disruptive.  I felt like I was teaching 9th graders.

In retrospect, I was totally unprepared for the audience I was teaching.  In earlier, lecture-based versions of this course, I could envelop myself in a bubble of denial.  I could focus on the students who had genuine interest in the course and ignore the fact that most of the class was there to get an A or B and check off a core curriculum requirement.  I persuaded myself that I could get these students to engage simply by providing opportunities for engagement.  I didn't grasp that I would have to force that engagement, at least at first and probably for much longer.  I did not understand that many of them would take pride and pleasure not in learning but in bragging on Facebook about how little effort they are putting into the class.  I was utterly unprepared for their strong reaction against change (never have I heard so many praises of the class lecture; typically I hear students say that they CAN'T learn that way because they can't follow the lecturer, etc.).  I was  caught off guard by the way they used the flipped model as an excuse for not working.  It seemed to me that they decided that they couldn't learn from watching pre-recorded lectures and so therefore didn't need to watch them.  And when they subsequently did poorly on the exams, well, that was the fault of the course model and not a consequence of their study habits and lack of preparation. Over and over again, they are given opportunities to improve, but many of them would rather complain that my expectations are ridiculous (I mean, who can expect a lecture class to include discussion?); that the grading is to harsh (because we expect the actual, correct answer and not some vague approximation of it); and that they should be doing better--even when some of them are also proud of how little effort they are expending. The logic baffles.

I couldn't make the kinds of mid-semester adjustments that I wanted to make (and that the class needed) thanks to the Texas legislature, which demands that syllabuses be published on the first class day and remain unchanged.  This is largely a good thing; but it does mean that, when instructors are testing out new course designs, we are stuck with a flawed design for the entire semester.  This is tough on instructors as well as the students, especially the students who wonder why I am not making changes to a model that is clearly not working for a lot of their classmates.  I was motivated to change the structure of the course in the first place because I hated feeling like I wasn't really teaching them much of anything; and they certainly weren't learning to do more than memorize a bunch of facts and then dutifully spew them back to me on exams.  I totally failed to understand that I couldn't force students to learn anymore than a coach can play the game for her team.  All I can do is create a learning environment that motives student learning.  On the whole, I didn't do that as effectively as I had hoped, in large part because I failed to understand how much structure and grade-incentivized assessments would be required to get them to abandon their "cram for the exam" mode.  On the bright side, I have a very good sense of what steps need to be taken to shift the focus to daily preparation and away from exams.  It remains to be seen whether this shift will lead to increased student participation in and enjoyment of class discussion.  It also remains to be seen what it will take to persuade students that, to do well, they will have to actually put in some work and perform at a reasonably high level.  To my mind, these last two issues are probably much more deeply rooted in pre-college preparation and are going to be much more difficult to change.

Tuesday, December 4, 2012

Midterm #3: A New Record

My Rome students made a significant contribution to a new record for number of views of Echo recordings on a single day.  Unsurprisingly, the day they set this record was the day before the third midterm.  I wasn't surprised.  After the second midterm and in part in response to a lot of grumbling from students that they didn't find class a good use of their time (unsurprising since 75% of them were unprepared and so unable to really participate in the various polls and discussions), I decided to make attendance optional.  I expected a big drop off: November is a very busy month for students (and faculty) and they knew that everything for the Rome class was available on demand.  In fact, most days saw about 25% of the enrolled students in class.  It was largely the same students who came for every class (not coincidentally, several of these same students have test averages over 100).

I wasn't that surprised that students decided not to come to class.  When I recorded my lectures last fall, attendance dropped to about 30% of the class most days.  I wouldn't say that there was a significant difference between the two cohorts in this respect.  If given the opportunity, a lot of students will opt not to come to class, regardless of what is happening in the classroom (lecture or review/discussion).  This semester I saw very clearly just how poorly most of my students manage their time; and the depths of their denial about how much work they have to do to catch up.  When I offered the attendance "opt out", I expected that 70-75% would take it.  But I also expected that most of them would view the recordings of class at some point, perhaps even regularly.  In this respect, I was completely deluded.  While I don't have data from last fall showing me WHEN the students viewed the recordings of class lectures, I do have it for this semester.  What I saw was disheartening.  Despite repeated warnings not to save everything for the last few days; and despite being told repeatedly that 90% of the exam questions were taken from the material covered in class (about 50% are taken word for word), they apparently did no work whatsoever for 2.5 weeks and then attempted to cram everything in at the end.

I have a handful of altruistic students who have made a number of different study aids intended to help students review the lectures.  They also collect all the iclicker questions from class.  I am sure that many students skimmed the posted PPTs of lectures and then went to these study aids.  But, in various ways, these study aids were insufficient (notably, they didn't include images of architecture, which played an important role in the last third of the class).  The exam was on the Monday before Thanksgiving.  Starting on Wednesday, I received daily updates of the viewing stats; and over the weekend I received them every 12 hours.  As expected, they skyrocketed in the 24 hours before the exam.  Also as expected, many of the students did not make it through the entire list of assigned lectures.  This ended up causing them a lot of problems since the short answer portion of the exam put more emphasis on the later lectures (I wrote the exam before seeing these stats and had done this in part because I assumed that these would be easier questions for everyone, and especially for those who had been in class).

The stats for this exam aren't really comparable to previous cohorts because I also had a reasonably large group of students who were earning As or Bs blow it off (I allow them to count their lowest midterm for 5%--something I won't do again).  But, overall, the performance on the exam was dismal.  Typically, students score about 10 points higher on this exam than on the second midterm.  This group actually scored lower on this third midterm.  The reasons for this poor performance were apparent from their learning habits.  As several of them have subsequently explained to me, they had exams for their "important" classes (calculus, chemistry, computer science, etc.) during the same period and so put the Rome class on the "back burner".  Interestingly, though, they think that the problem is that the class is too hard, not that perhaps they should accept the consequences of not keeping up in a reasonably demanding class.

If I ever had any doubts about my plan to overhaul the assessment structure for future cohorts, they were laid to rest when I watched the train wreck that was the third midterm.  I also realized that I will never be successful in changing student learning habits by talking to them and trying to reason with them.  I have to simply put a structure in place that gives them constant feedback and forces them to engage or drop the class.  If I don't want them to cram for midterm exams, I need to put less weight on midterm exams and start attaching grades to their daily assignments.  Many of my students come from the natural sciences, engineering, vel sim.  They are used to doing daily homework.  It will be a change for them to do that in a large enrollment, humanities class, but I think they will settle into the habit relatively fast.  One of the biggest lessons of this semester has been that students will repeatedly make bad decisions if the negative consequences of those decisions are not immediately apparent.  In some sense, they WERE apparent when they came to class and couldn't engage, but their response was to devalue the classroom experience rather than to change their behavior. 

Monday, December 3, 2012

Changing Student Learning Strategies

In a recent op-ed in the New York Times, David Brooks took on the challenging subject of how people change (http://www.nytimes.com/2012/11/27/opinion/brooks-how-people-change.html?smid=fb-share).  It has generated a large number of comments from readers (in part because he chose to begin the column with the controversial story of a largely deadbeat dad who got fed up with his children's failures and so sent them a letter in which he disowned them).  One particular paragraph jumped out at me, as I wrap up Phase I of transforming my Introduction to Rome class from lecture-based to a student-centered class.  Wrote Brooks, "People don’t behave badly because they lack information about their shortcomings. They behave badly because they’ve fallen into patterns of destructive behavior from which they’re unable to escape."  This describes perfectly what I've observed about my Rome students, much to my frustration.

At the very start of the semester, I had the students watch a short recorded video titled "How to Avoid your Own Decline and Fall"--basically, tips for success in the class.  I discussed in detail how to watch the videos (take notes!  watch them in a quiet space that allows you to focus!) and gave advice on a range of issues like attendance, keeping up with the assigned work, etc.  We then duly discussed this video in class.  Yet, when push came to shove (i.e. when midterm season hit in about week 6 of the semester and, suddenly, the students' time management skills came under the gun), they ignored every single thing I had told them about how to succeed in the class.  As soon as we handed back the first midterm (on which scores had been well above the usual median and average, in large part because of the extraordinary effort put in by the students to prepare in addition to the extra practice with the course content during class), I let them know that the second exam was going to be significantly more challenging.  I told them the historical stats on the exam, in part to challenge them.  I reminded them that it would not be possible to cram and do well on the exam for all but a few of them.

75% of them ignored me.  They convinced themselves that what I was saying wouldn't happen to them.  Perhaps they thought they were the exception.  Mostly, though, I think they knew that their behavior was not going to have good results but they couldn't change.  Repeatedly, I've had students in my office telling me that they wished that *I* had done something to force them to stay on track with the assigned work.  When I asked them whether they thought that it was a fair of me to expect a college student at UT to be able to follow a syllabus and manage their time, they conceded that it was; but then repeated their desire to have me provide an external structure that would take the choice away from them.  The scores on the second midterm were right in line with what they were for cohorts who took the class as lecture-based--surprising given all the extra learning opportunities and study help that the current cohort has.  Once again, I repeated my advice to avoid cramming.  Many of them would have seen the negative results of their efforts to cram.

I expected that at least some of them would see the second midterm as a wake-up call and mend their ways.  Nope.  If anything, their behaviors worsened.  They stopped attending class (it was not longer mandatory) and seem to have completely checked out.  They then tried to cram for the third midterm in the 24-48 hours before the exam, during a week when they likely had several other exams and projects due.  The results were abysmal.   Despite being told repeatedly that their strategies would not work; despite being told what strategies WOULD work, they refused to change their ways.  At the same time, many of them wished that I would force them to change. The issue wasn't that they didn't see the problems with their learning strategies; it was that they didn't have the self-discipline to change them when they were feeling pressure from other directions.

One of the major changes in the spring is a result of my realization that I cannot bring about change in student learning strategies simply by reason (or data).  They hear it but they don't believe it, and they don't have the self-discipline to change (not unlike the diabetics who refuse to be compliant despite knowing the short- and long-term consequences for their health).  They need someone else to impose structure and discipline on them.  On the one hand, I feel like this is a skill that college students should have.  On the other, I acknowledge the reality that they don't and recognize that they only way they are possibly going to learn it is through experience.  I have begun to think about my class--one which will be taught in the fall alone in future semesters--as a kind of bridge course between high school/community college and UT (though, to be fair, it enrolls as many upper division as lower division students at present, something I am hoping to change in the fall).

One of the main course objectives going forward will be to teach students good learning strategies.  I will do this, in part, by imposing a lot more structure on the course.  There will be homework modules that must be completed before each class.  There will be weekly quizzes.  Emphasis will be shifted somewhat away from midterm exams and towards the day to day learning.  There will be a comprehensive final, but it is my hope that students will learn experientially that doing assigned work and studying weekly makes it much easier to learn and apply larger chunks of content.  I am also reading a lot about how to motivate and sustain change.  The message seems to be that it happens by reinforcing positive behaviors, so I am brainstorming about ways to do that.

Perhaps the clearest lesson from my current cohort of students is that I can't reason with them to change their learning strategies.  Even the experience of doing poorly on exams does not persuade them to change their behaviors.  Instead, it persuades them that the "flipped" model must be to blame; or my expectations must be too high; or the tests are too hard; or the lectures that they can't learn except from a professor lecturing in a classroom.  Only about 10% of them will even begin to confess that they bear some responsibility for their less than stellar performance.  The new tactic will be to force them to change their learning strategies (or quickly realize that they are going to fail the class) and hope that, somewhere in there, they will figure out how to manage their time and be able to adopt these strategies without the external, grade-driven structure.

Am I going to have to take more classes like this?

In a class of 387 students, I don't have the chance to talk to many of them.  I had planned to invite smaller groups of them to meet with me during the week over cookies and chat about the class, but the semester had some unexpected challenges from other quarters and I didn't have the time to do this.  It is something I am going to be sure to do in the spring, now that I realize just how skeptical so many of them are of any kind of pedagogical change (especially the kind that seems to require more engagement on their part).  But I do have students come by my office hours from time to time, and I always try to get them to talk about their experience in the class.  Some of them really like it, others clearly don't.  One common question I get is something like "are all classes at UT going to be taught like this?"

Faculty are feeling a bit on edge these days as we hear about plans for extensive transformation of introductory level, large enrollment courses.  The UT System is spending a lot of money on this, and UT's recent deal with EdX is clearly intended to allow faculty to develop a range of teaching tools that can be delivered in a range of teaching environments, from a huge MOOC to a 100 person, campus-based group of UT students.  As nervous as faculty are about what all this will mean for their teaching (and the time such transformation will require), students are perhaps even more nervous.  They may complain about large classes.  They are certainly not learning at the rate one would hope to see from college students, and I think most faculty would acknowledge that there is a "student learning" problem on campus.  They seem to be less prepared for college than ever and even very intelligent students lack basic learning habits that most faculty aren't prepared to teach (e.g. how to read a textbook).  But, if my own sample of students is any indication, students are not going to embrace blended learning approaches without complaint.  They may complain about their boring lecture class, but they complain more about the large class where they are expected to pay attention and stay on task and engage.  From their perspective (and I've heard this from more than one student), they are paying for their college experience and should get whatever they want.  All to say, a significant part of transforming large enrollment courses, at least for the first few, transitional years, will be to also transform the students who are taking them.  In particular, it will require these students to understand that they are going to have to work--and work consistently--in all of their classes; and that they may not always get the A or B that they feel they deserve.

If I am being honest, I expected that the students in my Rome class would be so excited to have the chance to talk and engage that it would motivate them to continue to prepare.  In my head, as I was taping the lectures during the dog days of summer, I imagined them eagerly engaging with the course material.  Instead, they behaved exactly like previous cohorts of students (x2): about 25-30% actually took the class, did the work, and did well.  The rest of the students crammed, looked for shortcuts, and probably retained very little.  We won't know whether that is actually true because there's no comprehensive final, but I'd bet some good money on it.  It was a huge disappointment to spend so much time reworking the class and then have the majority of the students make it clear that they only cared about their grade, not about learning the material.  I am optimistic that I can shift these numbers, so that more like 65-70% are truly engaged.  I accept that there are 30% that probably can't be reached, no matter what I do.  They are the ones who are in college to check of boxes, get a degree, and make a lot of money.  My class is just a box to them and it isn't going to make them a lot of money to know Roman history.  But there's that middle group that I think can be reached, who can be persuaded that they are lucky to be taking a class "like this one" and who will find themselves getting genuinely interested in the subject.  We'll see...

Sunday, December 2, 2012

A late drop

I hate the last few weeks of the semester when I am teaching a large enrollment class.  The third midterm has finished up and, finally, students who are doing poorly are forced out of their state of denial.  Mad grade calculations ensue as they work out exactly what grade they need to make on the final exam to get their desired grade in the course.  They finally come to my office to express their concerns about their performance and to ask how they might do better (they have been encouraged to do this since the first midterm).  A few of them realize that they either can't get the grade they want; or don't want to put in the work to get the grade they want, and so do late drops.

I had one of these students in my office on Friday.  I didn't recognize him, but that just means that he doesn't sit near the front and probably doesn't attend class all that regularly.  He had done fairly well on the first exam--a B--but then had earned Ds on the next two exams.  He could still get a C and possibly even a low B if he did particularly well on the final, but it was clear that he just didn't want to invest the time.  I signed his form, but also started a conversation with him about his experience in the class.  In particular, I was interested to hear why he felt that he didn't do as well as wanted.  Part of what stimulated my questions was his own: "Is this going to be the way all lecture classes are taught?"

I hear this a lot from my students.  I have to confess that it comes as a shock to me that so many students have reacted so strongly to the decision to use class time to review and discuss content delivered outside of class.  They insist that they "learn better" when being lectured to in class (something I find difficult to accept when I sit in on other large enrollment classes, focusing entirely on the behavior of the students, and observe that almost nobody is engaged in the lecture; many are texting or playing games on their phones; and they write down only what appears on the Powerpoint).  They say that they can't learn from lectures they watch at home--lectures that are about 20 minutes long and based closely on the textbook.  I suppose what I find so bizarre about this is that, for generations, students have been asked to read textbooks outside of class.  I fail to understand how it is particularly different to read a textbook or view a textbook-based lecture.

As I probe more deeply into these complaints, the truth emerges: it's just too much work and effort to do assigned homework and then come to class and think about and apply the knowledge gained.  As I realize this, I feel incredibly frustrated.  Certainly, I will be making some major design changes to the class for the spring to address the most significant issues that came up this fall.  But I can't force students to work.  One of the "complaints" of the student who was late-dropping the class was that I had so many different ways to try to get students engaged and he didn't really want to engage.  He just wanted to come to class, be told what to learn in a clear and uncomplicated way (he mentioned another lecture class where the professor put up an outline and key terms so that students knew what "mattered" in the lecture), and the regurgitate it on the exam.  This was what he told me.  I wasn't surprised, but I was pretty sad.  He was clearly an intelligent person; but he was also a transfer student who didn't have great study skills and struggled to learn more than 2 weeks of material at a time.  The new course design will help some with this issue (this student isn't the only one who struggles with it), but I also believe that one of the missions of a college education is to teach students to take in and digest large amounts of content.

Students are going to be doing a number of different surveys about their experience in the class.  But it's clear that the majority of them would have preferred a standard lecture class.  By this, they mean that they would have preferred a class that couldn't deliver as much content or hold them responsible for knowing that content.  When I designed the course, I assumed that most students would care about learning.  What I have learned is that, in this particular class (a core requirement), most don't.  They just want a grade (that is, an A or B).  The challenge going forward will be to see if there are ways to get them to care about learning at least as much as they care about their grade.  I think I can do it, but this semester has definitely provided a wake-up call of sorts.  In the past, I wasn't as tuned in to the students and their learning habits.  What I've seen this semester has highlighted just how great a challenge it will be to transform larger enrollment classes at UT.  It's not just a matter of getting faculty on board.  Teaching the students how to take such classes, and designing courses that force them to do the assigned work, will be key.  I suspect that there will be a lot of kicking and screaming during the transition period and faculty are going to have to learn to tolerate that (it sucks, by the way).  Those who evaluate us are going to have to understand that flipped and blended courses are not going to please students who expect a large enrollment class to require little effort or engagement.