Once the shine of the new academic year wore off for my students--and the reality that they would have to work hard for their grades in my course set in--I've been dealing with some pockets of students resistance to the flipped class. It's not clear how widespread it is. Certainly, there are some vocal resisters who have contributed to a class FB page set up by the students and for the students (I'm not part of the group and so only hear reports secondhand). The model has its vocal defenders, but I imagine that many who like it don't feel particularly inclined to announce that on FB or to try to persuade their classmates to change their minds. Likewise, I imagine that there is a group of students who aren't particularly fond of the flipped class but also don't absolutely hate it.
I've been trying to get a grip on the underlying reasons for this resistance and seem to have landed on two general sources: first, there are students who simply don't want to do the work. They prefer the lecture model because they can come to class, be spoon-fed, and then not be expected to know very much or be able to do any higher-order thinking with the information they've been spoon-fed. In the flipped class, expectations are higher and there is more of an onus on them to organize and participate in the learning process. I have science and business majors who think they shouldn't have to spend so much time on a stupid humanities course that is just a core requirement. I have students who are perfectly capable of doing well in the course but resent having to spend so much time learning the material so that their GPA isn't adversely affected. For this group, I don't have a whole lot of sympathy. Mostly, I will do a better job of letting my future classes know what they are in for so that students who are shopping around for an effortless A can go to another store.
Another significant source of student resistance seems to come from their sense that I have pushed them out of their comfort zone. I hadn't realized how much this was true until recently, and I hadn't realized how much it was also true for me as an instructor. Indeed, what the flipped classroom does is force all of the participants--instructors and students--to learn new roles in the teaching process. I know that I've felt like I was operating out of my comfort zone during our in class sessions. I have felt good about the job I was doing, but also very aware of how much less scripted class is when I am running a more discussion-based class. We don't always get through what I planned--in fact, we rarely do. Every single class, something unexpected seems to happen and I have to think on my feet. It's such a different experience from walking in and doing my lecture performance and walking out.
I realize now that my students must be having a similar experience. In lecture courses, they know what to do, what is expected of them. They think they know how to learn in that environment. They've come into my classroom and suddenly been told that they need to learn in a completely new way. The pieces are the same, just in a different order; and there's what I considered the added bonus of getting to practice that learning in class. From their perspective, however, it isn't at all clear how those pieces fit together. Yes, I explained all this several times at the start of class and several more times afterward. I will explain it again this week. But they are so accustomed to the "lecture in the classroom" model that it's not at all clear to them that the recorded lecture is the same as that, just in a different space. And it's not clear to them that what they are doing in class is practicing concepts that will appear on the exam. I know this isn't clear because I see how few of them access the recordings of our classes. I am going to "source" the exam questions to show them very clearly that they are coming largely from i>clicker and review questions. I have learned, however, that I have to take seriously the fact that the flipped class isn't just an easy transposition of in class/out of class work. It can be completely disorienting to them, especially to those who have taken several large lecture courses and have figured out how to do well in that environment.
In my spring class, I will spend some time at the start of the term talking directly about this issue of comfort zones and disorientation. I will acknowledge that it is a real thing and I will provide them with a lot more guidance in figuring out how to "do" the course. Sometimes I think we GenXers assume that our students are far more able to navigate different forms of technology, far more flexible about space, than they actually are. To the contrary, this course has taught me that they are very tied to the idea that learning happens in a classroom and it happens best with the instructor telling them what to learn. Yet these students will be entering a workplace that will be ever more flexible, both in terms of when the 8 hour day happens and where. Some and perhaps many of them will be telecommuting on a regular basis. They will be entering a workforce that values adaptability, flexibility, and critical thinking skills. Part of my job besides teaching them about ancient Rome, is to get them to strengthen these muscles. Indeed, pushing them out of their comfort zones and getting them to learn how to adapt to and resolve that discomfort is part of what I need to be doing, and part of what they need to be learning how to do.
I can empathize with their resistance, though. I chose to make myself uncomfortable. Most of them had no idea what I meant when I explained the flipped class model to them. Further, most of them probably have no idea why, exactly, they feel uneasy and unsure of what to do to succeed. The answer, of course, is simple: just do the outside of class work thoughtfully and come to class and engage. But first, this requires them to know HOW to do that outside of class work and to grasp in a deep way how it is helping them learn. In a class that isn't problem-based, it's very difficult to get them to see their learning (or lack thereof) until they take an exam. For this reason, I think it nearly impossible to flip a large enrollment humanities class (like Ancient Rome) without also having frequent, low-stakes assessments. They have to be able to see how well their learning strategies are (or are not) working. In a math class, this is likely to be easier for them to ascertain based on their ability to do a problem. In non-problem based course, however, it is too easy for them to confuse recognition with knowledge.
Tuesday, October 30, 2012
Monday, October 29, 2012
Re-defining Learning spaces
Slowly but surely, I am learning that one of the most significant hurdles in flipping a class is the issue of learning space. When we flip our classes--that is, shift the instruction of content outside the classroom to make room for practicing application inside the classroom--we don't really take account of the fact that we also need to work closely with our students to create appropriate learning spaces. It's not a surprise that this point isn't highlighted in the literature about the flipped class. After all, the concept originated in the high schools where parents do a lot to shape and control the learning space of their children. As we start to implement the principles of blended learning at the post-secondary level, however, we need to spend some serious time thinking about strategies for teaching our students how to learn outside of class. This is all the more urgent these days, when high schools are moving away from assigning outside of class work. In many cases, our students are coming to college/university with little experience in learning outside the class and no idea of how to do it (or, even, why they should have to do it).
In the coming years, teaching our students how to learn outside of class is going to be our job. We can no longer assume that they come to us with good "homework habits". Even if they did do some outside of class learning in high school, we can't assume that they know how to create a good learning environment for themselves. My generation (GenX) went to the library or we studied in our rooms. Sometimes we met in empty classrooms or in the halls of buildings for group study. We were used to doing homework and we weren't surprised when we had to do homework in college. We also lived in a world without the internet, facebook, smartphones, and the hundreds of other distractions our students (and we ourselves) deal with on a daily basis. I took for granted that, if I asked them to watch videos of lectures (20 minutes each) and gave them a short video about how to watch the lectures (i.e. take notes, write down questions), that this would be enough. I didn't realize that I really needed to spend time talking about HOW to do their homework, how to create a quiet environment that would let them focus, etc. Too late, I realized that they are probably listening to these recorded lectures while playing on FB, talking on the phone, watching football, etc.
I was floored when I learned that some of them were actively demanding that I lecture to them in class. This made no sense to me at all. After all, when I had my in class lectures "captured" using Echo360, students were quite happy to skip class and watch the recordings. Nobody was demanding that I require them to come to class. So I tried to analyze what was underlying this demand. I realized that the answer probably had two parts: on the one hand, they didn't think they should HAVE to come to class AND do outside of work. It should be one or the other. Second, many of them aren't learning nearly enough from the recorded lectures because, well, pressing play isn't going to get the information into their brains. They have the (false) idea that they would somehow learn better if they were in the class, listening to me talk at them (while, of course, they were tuned out, on FB, napping, and so on). I am coming to think that this strange longing for the in class lecture has more to do with the fact that they feel comfortable with that space--it's a space someone else has constructed for them. On the other hand, they don't feel at all comfortable with the idea of constructing their own learning space. I have learned that part of what I need to do with this flipped class--but probably with every class I ever teach again--is talk in a direct and focused way about how to construct learning spaces outside of class.
In the coming years, teaching our students how to learn outside of class is going to be our job. We can no longer assume that they come to us with good "homework habits". Even if they did do some outside of class learning in high school, we can't assume that they know how to create a good learning environment for themselves. My generation (GenX) went to the library or we studied in our rooms. Sometimes we met in empty classrooms or in the halls of buildings for group study. We were used to doing homework and we weren't surprised when we had to do homework in college. We also lived in a world without the internet, facebook, smartphones, and the hundreds of other distractions our students (and we ourselves) deal with on a daily basis. I took for granted that, if I asked them to watch videos of lectures (20 minutes each) and gave them a short video about how to watch the lectures (i.e. take notes, write down questions), that this would be enough. I didn't realize that I really needed to spend time talking about HOW to do their homework, how to create a quiet environment that would let them focus, etc. Too late, I realized that they are probably listening to these recorded lectures while playing on FB, talking on the phone, watching football, etc.
I was floored when I learned that some of them were actively demanding that I lecture to them in class. This made no sense to me at all. After all, when I had my in class lectures "captured" using Echo360, students were quite happy to skip class and watch the recordings. Nobody was demanding that I require them to come to class. So I tried to analyze what was underlying this demand. I realized that the answer probably had two parts: on the one hand, they didn't think they should HAVE to come to class AND do outside of work. It should be one or the other. Second, many of them aren't learning nearly enough from the recorded lectures because, well, pressing play isn't going to get the information into their brains. They have the (false) idea that they would somehow learn better if they were in the class, listening to me talk at them (while, of course, they were tuned out, on FB, napping, and so on). I am coming to think that this strange longing for the in class lecture has more to do with the fact that they feel comfortable with that space--it's a space someone else has constructed for them. On the other hand, they don't feel at all comfortable with the idea of constructing their own learning space. I have learned that part of what I need to do with this flipped class--but probably with every class I ever teach again--is talk in a direct and focused way about how to construct learning spaces outside of class.
Sunday, October 28, 2012
Teaching, Learning, and the Space of the Classroom
This week I have been thinking about the space of the classroom and what it means for me as an instructor as well as for my students. My students have spent 13+ years traveling to a space that is generally populated with desks, fellow students, and some expert who usually stands at the front of the class (the infamous "sage on the stage"). They know that, when they are in this space, they will be expected to do that thing they call "learning"; and they expect that the instructor will teach them. From their 13+ years of formal education, they have developed very specific expectations about what this teaching will look like and how they will experience it: that expert, probably standing near the front of the class (especially if it is a large class in a UT auditorium), will pontificate on some set topic. S/he will likely have assigned the students some outside of class reading but few of them will have done it since, well, nobody else is doing it and they aren't going to be expected to participate because, well, the instructor is teaching.
The instructor might pause from time to time to take questions or even to ask a few questions. The usual suspects near the front of the class will be paying attention, will have done the reading before class, and will know the answers. The majority of students will slump in their seats, take a furtive glance at the text messages appearing on their phone, and tune out. Some of them will be recording the lecture for later transcription. Some of them will be doing homework for other classes; a few will be taking notes on the lecture. And the lecture? Well, it will be the instructor digesting the assigned readings, repeating a lot of the information, and perhaps adding a few insights and some context or making some connections that weren't in the textbook. Early in the semester, students will realize that they can get an A just by having access to the lecture notes. These days, that doesn't require regular class attendance. Groups of students will take a class, rotate attendance, and share notes with one another. As well, a week or so before the exam, desperate emails will circulate from students who missed most of the class meetings but need the lecture notes (I know about these emails because, often, they forget to delete the email addresses of the instructor and teaching assistants). Some kind soul, in an act of misguided altruism, will post their notes on google docs and share with the entire class. They will do the same with any study guides that are handed out.
THIS is the classroom that my students know; it is a space that is comfortable and familiar to them. They know what their role is and they think they know what my role is. I am learning this semester that a major source of student dissatisfaction with the flipped class is owed to discomfort. I am taking their familiar space and redefining it (without their permission, mind you). I am forcing them to take on entirely new roles in this space, roles that require them to DO something, roles that require them to BE THERE. In other words, I have not only pushed them out of their comfort zone; I have completely removed their comfort zone (as they see it). Suddenly, everything they thought they knew about their role and my role has been declared null and void. It doesn't apply. Some of them embraced their new roles as active students in an active classroom with an instructor who does something other than lecture at them during class time. But others of them are angry. They had mastered the old, familiar system and were perfectly content with it. They want it back, darn it. And they are going to kick and scream (figuratively) until it is restored and they are back to feeling comfortable.
The instructor might pause from time to time to take questions or even to ask a few questions. The usual suspects near the front of the class will be paying attention, will have done the reading before class, and will know the answers. The majority of students will slump in their seats, take a furtive glance at the text messages appearing on their phone, and tune out. Some of them will be recording the lecture for later transcription. Some of them will be doing homework for other classes; a few will be taking notes on the lecture. And the lecture? Well, it will be the instructor digesting the assigned readings, repeating a lot of the information, and perhaps adding a few insights and some context or making some connections that weren't in the textbook. Early in the semester, students will realize that they can get an A just by having access to the lecture notes. These days, that doesn't require regular class attendance. Groups of students will take a class, rotate attendance, and share notes with one another. As well, a week or so before the exam, desperate emails will circulate from students who missed most of the class meetings but need the lecture notes (I know about these emails because, often, they forget to delete the email addresses of the instructor and teaching assistants). Some kind soul, in an act of misguided altruism, will post their notes on google docs and share with the entire class. They will do the same with any study guides that are handed out.
THIS is the classroom that my students know; it is a space that is comfortable and familiar to them. They know what their role is and they think they know what my role is. I am learning this semester that a major source of student dissatisfaction with the flipped class is owed to discomfort. I am taking their familiar space and redefining it (without their permission, mind you). I am forcing them to take on entirely new roles in this space, roles that require them to DO something, roles that require them to BE THERE. In other words, I have not only pushed them out of their comfort zone; I have completely removed their comfort zone (as they see it). Suddenly, everything they thought they knew about their role and my role has been declared null and void. It doesn't apply. Some of them embraced their new roles as active students in an active classroom with an instructor who does something other than lecture at them during class time. But others of them are angry. They had mastered the old, familiar system and were perfectly content with it. They want it back, darn it. And they are going to kick and scream (figuratively) until it is restored and they are back to feeling comfortable.
When Technology doesn't Keep up with Pedagogy
I spent my Saturday hand-grading six scantron questions from my recent midterm exam. These were questions that asked the students to mark all correct answers. In five of the six questions, there were multiple correct answers. During my reading about assessment and also from my own experiences with a Coursera myth course, I had learned that such questions were highly effective at creating an accurate representation of what students DID know. They largely eliminate the advantage of being a "good guesser" and having a partial knowledge of course material. They reward deep knowledge. Pedagogically, they are a great question type. For a class of 400 with limited TA support, however, they are a huge burden to grade. Each question has to be graded, with partial credit awarded for each bubble; and then the points have to be totaled. And then those points have to be added to whatever they scored on the regular scantron. In our case, a new Excel document has to be created with the total scantron score that can then be distributed to the 2 grad TAs and 2 student TAs who are grading the exams. In other words, hours and hours of extra work, all because our scantron machine (apparently) can't be programmed to read multiple correct answers (and because the person running it had no idea if there was a work-around).
I am beyond frustrated. I plan to continue to include them, but will include a few of them on the short answer part of the exam. It will add some extra grading for the TAs, but at least will minimize that. Of course, I now realize yet again why it is that students looking for an easy class flock to the very large courses--they figure it's impossible for us to require them to keep up with the course or to hold them to test depth of knowledge. They know we don't have the TA support and they count on the fact that many of the faculty who are teaching these large courses are underpaid adjuncts and lecturers who have made their peace with the limitation imposed on their teaching. It is mind-boggling to me that, in this day and age, a scantron machine can't perform such a simple task that allows instructors to ask machine-gradeable questions that get much closer to measuring a student's real knowledge.
I'd love to hear any solutions people have found; and I'd love to hear about other types of MC questions one can ask. I do ask the traditional "EXCEPT" questions, which are related to these. But of course, it's all or nothing. What I liked about the "mark all of the above" is that it meant they got partial credit. I suppose I could spread the answers over 5 blanks and they have to mark them in order? And one of the options is "none"? But it seems like that would be really confusing, even with careful explanation of how to answer the question.
I am beyond frustrated. I plan to continue to include them, but will include a few of them on the short answer part of the exam. It will add some extra grading for the TAs, but at least will minimize that. Of course, I now realize yet again why it is that students looking for an easy class flock to the very large courses--they figure it's impossible for us to require them to keep up with the course or to hold them to test depth of knowledge. They know we don't have the TA support and they count on the fact that many of the faculty who are teaching these large courses are underpaid adjuncts and lecturers who have made their peace with the limitation imposed on their teaching. It is mind-boggling to me that, in this day and age, a scantron machine can't perform such a simple task that allows instructors to ask machine-gradeable questions that get much closer to measuring a student's real knowledge.
I'd love to hear any solutions people have found; and I'd love to hear about other types of MC questions one can ask. I do ask the traditional "EXCEPT" questions, which are related to these. But of course, it's all or nothing. What I liked about the "mark all of the above" is that it meant they got partial credit. I suppose I could spread the answers over 5 blanks and they have to mark them in order? And one of the options is "none"? But it seems like that would be really confusing, even with careful explanation of how to answer the question.
Tuesday, October 23, 2012
Classroom Co-dependence and Course Evaluations
I've been thinking a lot about the issue of classroom co-dependence: the unhealthy relationship that can sometimes develop between an instructor and a class, facilitated by our needs as instructors for validation of our hard work, good course evaluations for tenure and promotion, and (sometimes) the love and adoration of our students that make up for all those sacrifices we made to get our degrees. It can be very difficult to remember that we are there to facilitate learning, by whatever reasonable means, and that sometimes that conflicts with being liked by our students or having them give us a high five for our hard work in creating a learning experience for them. Sometimes it means raising the bar high enough, pushing the students enough out of their comfort zones that they are not too happy with us. Now, I generally receive high course evaluations from undergraduates and my students generally respond well to me. At the same time, if I am being honest, I know this is because I've found that sweet spot of pushing them a little but not too much out of their comfort zones; and making sure that, though I ask a lot of them, the grades are still reasonably high (that is, they are in keeping with the average grades at UT). I know in my heart of hearts that if I pushed them as hard as I probably should, most of them would resent me and give me low course evaluations--not because I am a bad teacher or because they didn't learn a heck of a lot but because they did not like to be made uncomfortable.
Indeed, as I meditate on teaching and students and classroom strategies this fall, I find myself returning again and again to the course evaluation. At the risk of blaming course evaluations for all that is wrong with higher education, I do think they have done far more harm than good. The information they provide is not especially reliable, especially for demanding courses that award lower grades (aka not an A). Very few students do more than fill in the bubbles. We are lucky if a statistically valid sample of our students (in a large class) show up to do them and in small classes the sample is too small to be valid. Comments are either extremely positive or extremely negative and most don't take the time to write comments at all. It doesn't help matters that we hand out the evaluations while they are stressing about exams and finals and going home for the holidays or the end of the year. And, finally, any instructor with a bit of 7th grade math figures out pretty quickly that the key to high course evaluations is a. making sure that students think they are getting As or Bs at the time of doing them (I know faculty who write easy midterms and killer finals to elevate their course evals while still giving them a reasonable grade distribution at the end); and b. making sure that, even if they don't all love the class, that nobody hates the class. Nothing screws up an average, especially in a small class, like a few malcontents. Never hand them out on a day when the students who never showed up all semester suddenly come out of the woodwork. And so on.
Over the years, I've heard hundreds of "tips". One of my favorites is the instructor who addresses the questions on the course evaluations point by point during the semester, without drawing attention to what he is doing, to tell his students how he is doing an outstanding job of X (returning exams quickly, conveying information clearly, being accessible). His students, like students well-trained to parrot back what we tell them, dutifully do so on their evaluations of his course. I don't know whether to be horrified at the manipulation or impressed by the savvy of this. Mostly, I have taken the position that I don't pay much attention to them. Truth be told, I don't even read them until about a year after the class is over and I rarely find a comment that inclines me to rethink an element of the course (I am taking about undergrad courses, not grad seminars here: different story entirely with grad students). These days I note that I have just become skilled at avoiding the zinger comments--the comments section is usually blank or says something sweet like "Nice job!"
More recently, however, I have started to wonder if course evaluations were more than a useless annoyance and, actually, something more insidious. I have wondered if their very existence, if knowing that we faculty are going to be evaluated by our customers based not on the product we have delivered but on some vague sense of their comfort level and happiness, has been hugely detrimental to higher education. Don't get me wrong: we faculty need to be held accountable for our teaching. But surely there is some other way to do that? Annual teaching portfolios? Demonstrating that our students achieved some agreed-upon outcome? Peer evaluations? Student evaluations can and should still be part of the package, but they can't be the sole source of evaluating instructor success. Despite the discourse, students aren't customers and the university isn't a supermarket selling a customer experience. In an era when data collection is so much easier; when we can now track student behavior, it makes much more sense to focus on what the students are learning and retaining and much less on whether the learning experience makes them as happy as a visit to Disneyland.
I am NOT advocating a joyless classroom--quite the opposite. But what I AM advocating is a move away from this pernicious co-dependency that too many instructors have with the students, a co-dependency that is nearly forced on us by the situation in which we teach and have our teaching judged. One of the reasons that students kick and scream at being asked to do reasonable amounts of work is simply that they have learned that it works--that we faculty, like a bad parent, back off and give them their way all too often. It's no accident that, when they are upset at being asked to do reasonable things or move a bit out of their comfort zone (as with the flipped class), that their first reaction is to talk about what they are going to say on course evaluations. That is their weapon, or so they believe.
Has the time come to re-think not just the role of the course evaluation but even the ways that we evaluate teaching? Of course, to do so rigorously and well would require a much greater investment of time on the part of faculty and administration than handing over the task to a room full of freshmen.
Indeed, as I meditate on teaching and students and classroom strategies this fall, I find myself returning again and again to the course evaluation. At the risk of blaming course evaluations for all that is wrong with higher education, I do think they have done far more harm than good. The information they provide is not especially reliable, especially for demanding courses that award lower grades (aka not an A). Very few students do more than fill in the bubbles. We are lucky if a statistically valid sample of our students (in a large class) show up to do them and in small classes the sample is too small to be valid. Comments are either extremely positive or extremely negative and most don't take the time to write comments at all. It doesn't help matters that we hand out the evaluations while they are stressing about exams and finals and going home for the holidays or the end of the year. And, finally, any instructor with a bit of 7th grade math figures out pretty quickly that the key to high course evaluations is a. making sure that students think they are getting As or Bs at the time of doing them (I know faculty who write easy midterms and killer finals to elevate their course evals while still giving them a reasonable grade distribution at the end); and b. making sure that, even if they don't all love the class, that nobody hates the class. Nothing screws up an average, especially in a small class, like a few malcontents. Never hand them out on a day when the students who never showed up all semester suddenly come out of the woodwork. And so on.
Over the years, I've heard hundreds of "tips". One of my favorites is the instructor who addresses the questions on the course evaluations point by point during the semester, without drawing attention to what he is doing, to tell his students how he is doing an outstanding job of X (returning exams quickly, conveying information clearly, being accessible). His students, like students well-trained to parrot back what we tell them, dutifully do so on their evaluations of his course. I don't know whether to be horrified at the manipulation or impressed by the savvy of this. Mostly, I have taken the position that I don't pay much attention to them. Truth be told, I don't even read them until about a year after the class is over and I rarely find a comment that inclines me to rethink an element of the course (I am taking about undergrad courses, not grad seminars here: different story entirely with grad students). These days I note that I have just become skilled at avoiding the zinger comments--the comments section is usually blank or says something sweet like "Nice job!"
More recently, however, I have started to wonder if course evaluations were more than a useless annoyance and, actually, something more insidious. I have wondered if their very existence, if knowing that we faculty are going to be evaluated by our customers based not on the product we have delivered but on some vague sense of their comfort level and happiness, has been hugely detrimental to higher education. Don't get me wrong: we faculty need to be held accountable for our teaching. But surely there is some other way to do that? Annual teaching portfolios? Demonstrating that our students achieved some agreed-upon outcome? Peer evaluations? Student evaluations can and should still be part of the package, but they can't be the sole source of evaluating instructor success. Despite the discourse, students aren't customers and the university isn't a supermarket selling a customer experience. In an era when data collection is so much easier; when we can now track student behavior, it makes much more sense to focus on what the students are learning and retaining and much less on whether the learning experience makes them as happy as a visit to Disneyland.
I am NOT advocating a joyless classroom--quite the opposite. But what I AM advocating is a move away from this pernicious co-dependency that too many instructors have with the students, a co-dependency that is nearly forced on us by the situation in which we teach and have our teaching judged. One of the reasons that students kick and scream at being asked to do reasonable amounts of work is simply that they have learned that it works--that we faculty, like a bad parent, back off and give them their way all too often. It's no accident that, when they are upset at being asked to do reasonable things or move a bit out of their comfort zone (as with the flipped class), that their first reaction is to talk about what they are going to say on course evaluations. That is their weapon, or so they believe.
Has the time come to re-think not just the role of the course evaluation but even the ways that we evaluate teaching? Of course, to do so rigorously and well would require a much greater investment of time on the part of faculty and administration than handing over the task to a room full of freshmen.
Messing with the Boundaries between Play and Work
A commentator on a previous post made a very astute observation about the students in our high school and university classrooms these days: they are more wired than any previous generation; they frequently have multiple devices with internet connectivity; they download all manner of things and watch all manner of things from a variety of internet sources. And yet it is this same generation of students who struggle to use course management systems effectively (and, often, simply avoid engagement whenever possible). They will discuss things, but only on Facebook and only if it is an "unofficial" page (i.e. a group that does not include the instructor). Official course discussion boards (in my class it is Piazza) become a Q&A site. This is still a huge time-saver in that I am not answering the same question over and over again via email, but it's disappointing that I can't entice them to want to discuss course material unless I attach a grade to their participation. Finally, some of them are furious that they are expected to watch pre-recorded, short (15-20 min.) lectures outside of class. The same students who will watch hours of Netflix TV on their computers rebel when asked to watch a few hours/week of recorded lectures.
Many of us educators think we are doing such a cool thing by reaching our students in "their" language, technology. But I wonder if, in fact, what we are doing is violating their boundaries. These may well be boundaries that they themselves are not even aware they have. I wonder if, in making videos and adapting various social media to have educational functions, we are basically taking their sources of play away. In some basic sense, we are getting into their space and taking it over, draining it of fun and relaxation and re-investing it with purpose and grade anxiety and all the rest. Now, I don't think we should stop using technology and social media for education; but it might help us understand their reluctance to engage with the education-version of things that they use every day in their lives if we can figure out what it is about, say, you-tube videos for class that causes them to demand a traditional, sage of the stage classroom.
Many of us educators think we are doing such a cool thing by reaching our students in "their" language, technology. But I wonder if, in fact, what we are doing is violating their boundaries. These may well be boundaries that they themselves are not even aware they have. I wonder if, in making videos and adapting various social media to have educational functions, we are basically taking their sources of play away. In some basic sense, we are getting into their space and taking it over, draining it of fun and relaxation and re-investing it with purpose and grade anxiety and all the rest. Now, I don't think we should stop using technology and social media for education; but it might help us understand their reluctance to engage with the education-version of things that they use every day in their lives if we can figure out what it is about, say, you-tube videos for class that causes them to demand a traditional, sage of the stage classroom.
Monday, October 22, 2012
Greek and Roman Mythology MOOC: Week 4
I was a typical undergraduate student this week: totally swamped and preoccupied with no spare time to watch the videos or do a writing assignment. I did end up watching the videos on Sunday afternoon and taking the quiz. I got a perfect score but it felt pretty hollow since I hadn't done the reading and really didn't have the spare brain cells or time to write a very short essay. Alas. I am learning a lot about how to do a MOOC by being on the other side of the production. I am enjoying this one very much. One thing is pretty clear: short lectures, about 10 minutes long; and not too many of them. I think there were about 90 minutes of lectures. That is about right. I take good notes and of course have read the Odyssey many times in the past. Still, I am enjoying the fact that I am learning a lot about how to teach the text if I ever had to. It's been invigorating to be on this side of things--and also a good reminder of how difficult it is for my students to juggle their own multiple obligations. Like me, they prioritize and let slide what they can let slide.
Learning outside of the classroom
As I've written about in some recent posts, I am confronting the expected student resistance to the flipped class. I'm somewhat frustrated that nobody has come to talk to me about in person; or taken me up on the chance to post their complaints anonymously on Piazza. This silence makes it difficult to know how seriously to take it and also suggests to me that, on some level, they know that they are complaining about the structure of the course because they don't like doing more work--and they don't quite want to take responsibility for that facet of their unhappiness. It's also not clear how many people are all that unhappy. On the other side of things, I've had several people email me or come to office hours to tell me how much they are enjoying the flipped class. All of this makes it really tough for me to find a way to address the dissatisfaction that I know is out there. I am thinking that I will address it in theoretical terms: "it is common for some students in flipped classes to feel..." and use that to try to open a discussion. I will also let them opt out of class time in they prefer. They still have to take the exams and are responsible for the material at the same level, but can opt out of coming to class.
One of the sources of whinging, though, is about the fact that I am expecting them to learn outside of class. Honestly, I had no idea that this would be a source of complaint. I assumed that they knew that it was traditional for students to learn outside of class. Homework? I am realizing that, in fact, many of them have found ways to minimize their out of class learning. They don't do assigned readings, at least not until just before the exam when they might skim them. Maybe they come to class or, more likely these days, they get someone else's notes via google docs. What is upsetting some of these kids is that I've made it much more difficult for them to get away with not learning, and learning deeply. It absolutely blows my mind that they are so overwhelmed by a workload of about 3-4 hours/week of outside of class assigned work (lectures + textbook readings) and 90 minutes inside of class (45 minutes x 2 with one other day of optional in class review). The in-class part is intended to practice and apply the out of class reading and viewing. It is absolutely shocking to me that this is considered a heavy workload.
I also wonder just how broken every part of our educational system is if we have a generation of students who have somehow been able to get college degrees without doing much out of class work. Certainly, I am beginning to understand why everyone is talking about demonstrable learning outcomes. It is clear that student satisfaction absolutely cannot be taken seriously. It is also clear that we, as a society, need to get serious about how we educate out kids--and, even more, that we get them to grasp the importance of learning how to learn and think. I worry about what kind of work ethic we are instilling in our kids if they think that doing 3-4 hours of work outside of class (+ extra work during exam weeks) is excessive. Sure, they will someday be getting paid to work; but it is no wonder that employers are more and more unhappy with the workforce that is being produced by colleges and universities. Still, if my own experience is any indication, it's clear that blaming teachers and professors is not the solution. Sure, some of us could and must do a better job of helping students to learn. But, ultimately, the students themselves have to put in the work; and the system needs to recognize that instructors who are pushing students to learn are, in effect, disturbing a wasps' nest. There will be stings and buzzing and other signs of unhappiness. We need to change our evaluations of courses from student satisfaction (though that can still be one element) to learning outcomes. That is, did a course improve student knowledge? Did the professor successfully motivate the students to master the course objectives outlined at the start of the term?
One of the sources of whinging, though, is about the fact that I am expecting them to learn outside of class. Honestly, I had no idea that this would be a source of complaint. I assumed that they knew that it was traditional for students to learn outside of class. Homework? I am realizing that, in fact, many of them have found ways to minimize their out of class learning. They don't do assigned readings, at least not until just before the exam when they might skim them. Maybe they come to class or, more likely these days, they get someone else's notes via google docs. What is upsetting some of these kids is that I've made it much more difficult for them to get away with not learning, and learning deeply. It absolutely blows my mind that they are so overwhelmed by a workload of about 3-4 hours/week of outside of class assigned work (lectures + textbook readings) and 90 minutes inside of class (45 minutes x 2 with one other day of optional in class review). The in-class part is intended to practice and apply the out of class reading and viewing. It is absolutely shocking to me that this is considered a heavy workload.
I also wonder just how broken every part of our educational system is if we have a generation of students who have somehow been able to get college degrees without doing much out of class work. Certainly, I am beginning to understand why everyone is talking about demonstrable learning outcomes. It is clear that student satisfaction absolutely cannot be taken seriously. It is also clear that we, as a society, need to get serious about how we educate out kids--and, even more, that we get them to grasp the importance of learning how to learn and think. I worry about what kind of work ethic we are instilling in our kids if they think that doing 3-4 hours of work outside of class (+ extra work during exam weeks) is excessive. Sure, they will someday be getting paid to work; but it is no wonder that employers are more and more unhappy with the workforce that is being produced by colleges and universities. Still, if my own experience is any indication, it's clear that blaming teachers and professors is not the solution. Sure, some of us could and must do a better job of helping students to learn. But, ultimately, the students themselves have to put in the work; and the system needs to recognize that instructors who are pushing students to learn are, in effect, disturbing a wasps' nest. There will be stings and buzzing and other signs of unhappiness. We need to change our evaluations of courses from student satisfaction (though that can still be one element) to learning outcomes. That is, did a course improve student knowledge? Did the professor successfully motivate the students to master the course objectives outlined at the start of the term?
I'm Teaching Myself!
Among the criticisms of the flipped course model that are slowly making their way back to me is the complaint that they are teaching themselves because I'm not using class time to lecture. This particular comment is the one that probably infuriates me the most. It would have been so much easier--and so much less time and energy on my part--if I had stuck with my traditional lecture-based course. Instead, I spent hours of time designing and delivering the filmed lectures. It is unclear to me how the act of uploading a video and pressing play and listening to it (perhaps taking notes on it) is teaching oneself. Really? When you read the textbook do you also believe that you are teaching yourself? Now, if you want to re-frame that comment to say that "I am working harder to construct knowledge", ok. But, ironically, the place you are working harder to do that is not at home in front of the recorded lecture, where I carefully explain various concepts to you; it is in class, when I am asking you to do some application work.
I grasp that I can't expect undergraduate students to be reflective enough as a group to understand how the parts of the course are working together. But it infuriates me when they complain that it is just an online class and that isn't what they paid for; and simultaneously complain about being required to come to class and participate in discussions about course material--because of course they don't want to learn the material on a reasonable schedule. I recognize that these are issues that have to be addressed at the start of a semester with a new cohort. And that I need to do a better job of explaining this new format to them and also thoroughly explaining the ways in which they aren't teaching themselves; this isn't an online class; etc. Still, I am flabbergasted. At the same time, I do understand that, if you haven't done your homework, you probably can't even begin to understand how carefully the in-class part of the course has been designed to identify flaws in understanding; and emphasize via application important concepts.
I just absolutely can't fathom how a student could possibly claim with a straight face that they learned more from lecture in class than outside of class. I have done in class lectures for years. When I started recorded them last fall, attendance plummeted. Clearly those students were more than happy to learn outside of class. So, really, the issue is that they don't want to have to do any heavy lifting with what they learn. They are more than happy to learn outside of class, but only if that is all they have to do. I know that I shouldn't be surprised by this behavior, but it makes me sad. It also shows that, in conversations about why higher ed isn't working very well, it's not enough to blame the faculty for not doing their job. In many ways, the far more significant problem is that, for most of their lives, students have been taught to exams. They haven't learned how to learn. And when asked to do something other than sit in a room, write down what I say, and puke it up on an exam, some of them freak out. *That* is the biggest problem with higher education right now. I don't know how we are going to change student attitudes and learning habits except slowly and through a concerted effort to design and implement rigorous courses; and by ignoring student course evaluations until the worst of the growing pains of implementation are over.
I grasp that I can't expect undergraduate students to be reflective enough as a group to understand how the parts of the course are working together. But it infuriates me when they complain that it is just an online class and that isn't what they paid for; and simultaneously complain about being required to come to class and participate in discussions about course material--because of course they don't want to learn the material on a reasonable schedule. I recognize that these are issues that have to be addressed at the start of a semester with a new cohort. And that I need to do a better job of explaining this new format to them and also thoroughly explaining the ways in which they aren't teaching themselves; this isn't an online class; etc. Still, I am flabbergasted. At the same time, I do understand that, if you haven't done your homework, you probably can't even begin to understand how carefully the in-class part of the course has been designed to identify flaws in understanding; and emphasize via application important concepts.
I just absolutely can't fathom how a student could possibly claim with a straight face that they learned more from lecture in class than outside of class. I have done in class lectures for years. When I started recorded them last fall, attendance plummeted. Clearly those students were more than happy to learn outside of class. So, really, the issue is that they don't want to have to do any heavy lifting with what they learn. They are more than happy to learn outside of class, but only if that is all they have to do. I know that I shouldn't be surprised by this behavior, but it makes me sad. It also shows that, in conversations about why higher ed isn't working very well, it's not enough to blame the faculty for not doing their job. In many ways, the far more significant problem is that, for most of their lives, students have been taught to exams. They haven't learned how to learn. And when asked to do something other than sit in a room, write down what I say, and puke it up on an exam, some of them freak out. *That* is the biggest problem with higher education right now. I don't know how we are going to change student attitudes and learning habits except slowly and through a concerted effort to design and implement rigorous courses; and by ignoring student course evaluations until the worst of the growing pains of implementation are over.
Sunday, October 21, 2012
Staying the Course, Sort of
My Rome students will be taking their second midterm on Wednesday. I suspect that they will do fine on the exam as a class. The exam isn't easy, but it's testing them over a lot of material that I reviewed in class (in addition to presenting it in the recorded lectures and assigning textbook readings). Still, it's clear that there is a vocal faction of students who are unhappy with the design of the class. Specifically, with the fact that it is a "flipped class". They were fine with it through the first 5 weeks of the semester, but as soon as midterm season started and their time management skills came under fire, they a. stopped watching the videos before class; and b. become much more resistant to the flipped model. They have a Facebook page where they vent. I am not part of the group but hear reports, and apparently there is a very active thread titled "I hate the flipped class". Not "I hate CC 302" or "I hate Dr. Ebbeler" but "I hate the flipped class."
Not a single student has said a word to me (despite being given several opportunities to do so anonymously); but I have good teacher radar and have picked up on the discontent during class, particularly when I ask them to do peer instruction and some number of them take this as a cue to nod off or text their friends. They don't seem to hate me (or at least that isn't what is dominating the FB discussions). But some of them clearly can't stand the flipped class model. One comment was particularly interesting: it was someone who said something about how I was doing so much "hand-holding" (by which I assume "teaching" was meant) that many of them would get As; but that this shouldn't stop them from using course evaluations as an opportunity to complain about the flipped class model. Otherwise, it might take hold and other classes would adopt it. In some ways, I feel sorry for this student who is clearly so unaware of what is happening at her own university as to not realize that it's a little late and no matter how much students protest being made to, gasp, learn, the model is here to stay (though, of course, not all instructors will move most content delivery to pre-recorded videos). I do feel some empathy for these students. The loudest voices seem to belong to upperclassmen who have been brought up on the huge class=nap time (I mean, lecture) model. I am taking them completely out of their comfort zone and, in doing so, asking them to be actively involved in the production of knowledge. That is how learning happens, but they don't like it. Not one bit.
I confess: I am flummoxed by this. I mean, I knew that it happened in theory. I read the research, I gathered that student resistance was a significant issue and that it seemed disconnected to the quality of instruction. I knew that, with a class of 400 and only 3 TAs, I was going to struggle more with student buy-in and was more at risk of negativity spreading like wildfire. Certainly, there are design flaws in the course that will be addressed in the next iteration. But I also worry that some of this is just resistance to change of any kind. As I noted above, the loudest voices are juniors and seniors who are filling in core requirements for graduation and just want an easy class that doesn't require much effort. Or they are science major who think that a humanities class should be easy (but of course that they should also get an A). So they resent that they actually have to show up; and, even worse, that they are expected to have done some work. One student noted that it was ridiculous for me to expect a class of 375 students to prepare for class (!). Really? I do realize now that it was ridiculous for me to expect that they could take responsibility for their education and stay up on their work without some form of external motivation. But in my Latin classes, where I assign work and then we review it in class, nobody seems to think it's ridiculous of me to expect preparation.
From what I can discern, the real issue is that the unhappy students don't want to be told how to learn. And many of them seem to believe that they learn better by sitting in a lecture hall and being talked at for 50 minutes x 3 weekly, not doing the assigned reading, and then cramming a few times/semester for midterm exams. I suspect that, in a way, this delusion is perpetuated by us professors who have done this because, in doing it, we write tests that are overly easy, simple regurgitation of what we say, and students think that is learning. I also understand the resistance to, in essence, being asked to work harder and think harder than they are used to doing, especially for a non-major, introductory level class.
So now I find myself in a pickle. Many of these comments have spurred me to thinking about how I can tinker with the course design for the spring and I think I have a lot of good ideas. I realize now that some part of me thought I was going to get this right the first time, that students were going to love it and love Ancient Rome and love me. I was completely delusional, of course. My students are reacting the way that research predicts they will. I am confident that UT understands well that this resistance isn't a reflection of my teaching skills but rather, of asking students to suddenly change the way they are learning and interacting with course material. I know I will do a better job of letting the students know what they've signed up for in future semesters. I will also change the assessment structure to incentivize keeping up. And I will be sure that everything we do in class is application, not review. This semester, I find myself lapsing more and more towards review of content because I know that so few have prepared.
I am thinking about some major changes for the last third of the semester. First, I will let anyone who wants to opt out of coming to class. They can always watch the recordings of class, but don't need to come. Second, I will make more of a point of using class time for discussion rather than reviewing factual information. I will prepare three or so questions for discussion and leave it at that. I think I've been doing far too much hand-holding because I know that so many of them haven't prepared. So I've been assuming (rightly, of course), that class is their first introduction to the material and have been more reluctant to just give them application questions to discuss. After this second midterm, though, that is what I am going to do. I suspect it will be much more pleasant for everyone when the students who don't want to be there but want an A without doing any real work are given permission to stay home.
Not a single student has said a word to me (despite being given several opportunities to do so anonymously); but I have good teacher radar and have picked up on the discontent during class, particularly when I ask them to do peer instruction and some number of them take this as a cue to nod off or text their friends. They don't seem to hate me (or at least that isn't what is dominating the FB discussions). But some of them clearly can't stand the flipped class model. One comment was particularly interesting: it was someone who said something about how I was doing so much "hand-holding" (by which I assume "teaching" was meant) that many of them would get As; but that this shouldn't stop them from using course evaluations as an opportunity to complain about the flipped class model. Otherwise, it might take hold and other classes would adopt it. In some ways, I feel sorry for this student who is clearly so unaware of what is happening at her own university as to not realize that it's a little late and no matter how much students protest being made to, gasp, learn, the model is here to stay (though, of course, not all instructors will move most content delivery to pre-recorded videos). I do feel some empathy for these students. The loudest voices seem to belong to upperclassmen who have been brought up on the huge class=nap time (I mean, lecture) model. I am taking them completely out of their comfort zone and, in doing so, asking them to be actively involved in the production of knowledge. That is how learning happens, but they don't like it. Not one bit.
I confess: I am flummoxed by this. I mean, I knew that it happened in theory. I read the research, I gathered that student resistance was a significant issue and that it seemed disconnected to the quality of instruction. I knew that, with a class of 400 and only 3 TAs, I was going to struggle more with student buy-in and was more at risk of negativity spreading like wildfire. Certainly, there are design flaws in the course that will be addressed in the next iteration. But I also worry that some of this is just resistance to change of any kind. As I noted above, the loudest voices are juniors and seniors who are filling in core requirements for graduation and just want an easy class that doesn't require much effort. Or they are science major who think that a humanities class should be easy (but of course that they should also get an A). So they resent that they actually have to show up; and, even worse, that they are expected to have done some work. One student noted that it was ridiculous for me to expect a class of 375 students to prepare for class (!). Really? I do realize now that it was ridiculous for me to expect that they could take responsibility for their education and stay up on their work without some form of external motivation. But in my Latin classes, where I assign work and then we review it in class, nobody seems to think it's ridiculous of me to expect preparation.
From what I can discern, the real issue is that the unhappy students don't want to be told how to learn. And many of them seem to believe that they learn better by sitting in a lecture hall and being talked at for 50 minutes x 3 weekly, not doing the assigned reading, and then cramming a few times/semester for midterm exams. I suspect that, in a way, this delusion is perpetuated by us professors who have done this because, in doing it, we write tests that are overly easy, simple regurgitation of what we say, and students think that is learning. I also understand the resistance to, in essence, being asked to work harder and think harder than they are used to doing, especially for a non-major, introductory level class.
So now I find myself in a pickle. Many of these comments have spurred me to thinking about how I can tinker with the course design for the spring and I think I have a lot of good ideas. I realize now that some part of me thought I was going to get this right the first time, that students were going to love it and love Ancient Rome and love me. I was completely delusional, of course. My students are reacting the way that research predicts they will. I am confident that UT understands well that this resistance isn't a reflection of my teaching skills but rather, of asking students to suddenly change the way they are learning and interacting with course material. I know I will do a better job of letting the students know what they've signed up for in future semesters. I will also change the assessment structure to incentivize keeping up. And I will be sure that everything we do in class is application, not review. This semester, I find myself lapsing more and more towards review of content because I know that so few have prepared.
I am thinking about some major changes for the last third of the semester. First, I will let anyone who wants to opt out of coming to class. They can always watch the recordings of class, but don't need to come. Second, I will make more of a point of using class time for discussion rather than reviewing factual information. I will prepare three or so questions for discussion and leave it at that. I think I've been doing far too much hand-holding because I know that so many of them haven't prepared. So I've been assuming (rightly, of course), that class is their first introduction to the material and have been more reluctant to just give them application questions to discuss. After this second midterm, though, that is what I am going to do. I suspect it will be much more pleasant for everyone when the students who don't want to be there but want an A without doing any real work are given permission to stay home.
Friday, October 19, 2012
Student Resistance and "Cram for the Exam"
The students in my Intro to Ancient Rome class had their first midterm on the 26th of September. Overall, the performance was very strong. The median grade was about 87. About 2/3 of the class got As and Bs. Interestingly, though, there was some not insignificant amount of grumbling after we returned the exams. The grumbling focused on the short answer portion of the exam, specifically on the fact that we deducted points on the short answer portion of the exam because they didn't include important details in their answers. The reasons for the grumbling seem to be multiple: first, (my fault) the graders deducted points from the number possible rather than giving them the number they earned. We spent a lot of time justifying deductions because they were working from the assumption that they started out with 5 points, not 0. Second, (again my fault) I didn't practice short answer questions with them so they could see what it meant for them to be graded by a rubric and could see the level of precision that we were expecting. Third, they worked really hard for the first part of the course, staying on top of the video viewing and being able to take advantage of the in class practice. They felt that all this hard work should give them an A and were discouraged when it really earned them an A- or a B+ (or, in some cases, a 96 instead of a 99). I also think that part of the issue was the number of points that each question was worth. For various reasons, I reduced the total number of questions but made them worth more points. This meant that even small details ended up being worth 1.5-2 points. That struck them as a lot to lose for such a small detail. But it was really an artifact of the shorter exam.
More surprising to me, however, was the complete change in learning behavior following the first midterm. Up to the first midterm they had been energetic, engaged, clearly staying on top of the material, and sharp. After the first midterm, they hit a major slump. At first, I thought that the issue was feeling upset about their grades on the exam (although they were very high, perhaps these were students who all thought that they would get an A just by studying?). After several more weeks of observation, however, I realize that the problem is more complex and more banal: these are students who have survived and thrived by "cramming for the exam." Given the different model of learning before the first exam, they were unsure what to expect and so did what was asked of them in terms of keeping up week to week. But once they realized that the exams were normal and even not very hard (because of how they had prepared...), they lapsed. Like addicts, they fell back into the familiar behavior once they felt more comfortable and once the pressures of their other classes and commitments hit them. Since the first midterm, only about 25-30% of them come to class prepared; the i>clicker response accuracy dropped by about 25%. And, overall, I find myself doing more of the talking because so few of them know anything. Peer instruction is more difficult because there aren't enough of them who are prepared and can teach their peers (and the class is so large that I can't know who is prepared and who isn't so that I might pair them up more effectively). All in all, I'm disappointed and frustrated.
I will write in more detail about student resistance to the flipped model in another post. Clearly, that is part of what is happening here. More precisely, it's student resistance to working hard; and resistance to not being in total control of the pace of their learning. It's not actually that they find the in class activities useless (though they surely aren't that enjoyable if you are utterly unprepared and three weeks behind). Rather, it's that they want to learn on their schedule, not mine. And, more to the point, they want to stick with what is familiar and comfortable: cramming. So, they still access the videos--it's just that now they are accessing the prerecorded videos as well as the recordings of class sessions during this week before the exam. Instead of seeing a steady pace of access, there's a big spike. Before the first midterm, we saw a big spike, but it was students who were reviewing. This time, it's going to be students watching for the first time. Some will likely try to get by using the elaborate study guides constructed by some of their more ambitious classmates.
I've learned a lot in these three weeks: first, my spring class will have weekly "small stakes" assessments and then probably two midterms, both of the cumulative (so that the students can see the relationship between the weekly work and the cumulative exams). Second, I won't "require" attendance. I will let them decide what they think is best for them. Something I am learning is that some/many undergraduate students fully believe that they know what is best for them and will not listen to facts and reason and data. They dig in their heels and moan and vent, usually on Facebook where they pollute the environment for the rest of the class. I am there to teach anyone who want to learn. But if a student truly believes that s/he can learn better on his/her own, I am happy to give them that opportunity.
These past few weeks have given me a lot to think about on the subject of student learning and my role in that process. I have struggled to understand why they reverted back to old habits after using the flipped model to do so well on the first exam. I have struggled to figure out how to manage the emerging student resistance to the flipped class model and the extra work it brings. Most of all, I have struggled to accept that some students just don't care about learning--and that nothing I say or do is going to get them to care. They just want to check off a box on their way to a diploma, and want a grade that will help their GPA. I have thought long and hard about what to do next. First, of course, I will see how this second exam goes (I'm concerned). Then I will take a day of class to revisit the flipped model, my impressions thus far, and ask them for comments on their experiences. I will validate their discomfort and acknowledge that I am asking them to do something difficult. I will also offer them the opportunity to "opt out" for the last month of the semester. I am curious to see how many will do so, and I'm curious to see how that will correlate with their performance in the course.
I still believe in the flipped classroom and I know that it can drastically improve learning. I know that many of my students are having an excellent experience and appreciate how deeply they are able to engage with the course material. Still, it's discomforting to confront the reality that there are others who hate the flipped class, who don't want to be taken out of their comfort zones. Even more, it's discomforting to realize that, until the flipped class becomes the norm (or at least more pervasive) on my campus, I will always be fighting against this inclination of students to cram for exams. I can't blame them, either: it's exactly what I did as an undergraduate.
More surprising to me, however, was the complete change in learning behavior following the first midterm. Up to the first midterm they had been energetic, engaged, clearly staying on top of the material, and sharp. After the first midterm, they hit a major slump. At first, I thought that the issue was feeling upset about their grades on the exam (although they were very high, perhaps these were students who all thought that they would get an A just by studying?). After several more weeks of observation, however, I realize that the problem is more complex and more banal: these are students who have survived and thrived by "cramming for the exam." Given the different model of learning before the first exam, they were unsure what to expect and so did what was asked of them in terms of keeping up week to week. But once they realized that the exams were normal and even not very hard (because of how they had prepared...), they lapsed. Like addicts, they fell back into the familiar behavior once they felt more comfortable and once the pressures of their other classes and commitments hit them. Since the first midterm, only about 25-30% of them come to class prepared; the i>clicker response accuracy dropped by about 25%. And, overall, I find myself doing more of the talking because so few of them know anything. Peer instruction is more difficult because there aren't enough of them who are prepared and can teach their peers (and the class is so large that I can't know who is prepared and who isn't so that I might pair them up more effectively). All in all, I'm disappointed and frustrated.
I will write in more detail about student resistance to the flipped model in another post. Clearly, that is part of what is happening here. More precisely, it's student resistance to working hard; and resistance to not being in total control of the pace of their learning. It's not actually that they find the in class activities useless (though they surely aren't that enjoyable if you are utterly unprepared and three weeks behind). Rather, it's that they want to learn on their schedule, not mine. And, more to the point, they want to stick with what is familiar and comfortable: cramming. So, they still access the videos--it's just that now they are accessing the prerecorded videos as well as the recordings of class sessions during this week before the exam. Instead of seeing a steady pace of access, there's a big spike. Before the first midterm, we saw a big spike, but it was students who were reviewing. This time, it's going to be students watching for the first time. Some will likely try to get by using the elaborate study guides constructed by some of their more ambitious classmates.
I've learned a lot in these three weeks: first, my spring class will have weekly "small stakes" assessments and then probably two midterms, both of the cumulative (so that the students can see the relationship between the weekly work and the cumulative exams). Second, I won't "require" attendance. I will let them decide what they think is best for them. Something I am learning is that some/many undergraduate students fully believe that they know what is best for them and will not listen to facts and reason and data. They dig in their heels and moan and vent, usually on Facebook where they pollute the environment for the rest of the class. I am there to teach anyone who want to learn. But if a student truly believes that s/he can learn better on his/her own, I am happy to give them that opportunity.
These past few weeks have given me a lot to think about on the subject of student learning and my role in that process. I have struggled to understand why they reverted back to old habits after using the flipped model to do so well on the first exam. I have struggled to figure out how to manage the emerging student resistance to the flipped class model and the extra work it brings. Most of all, I have struggled to accept that some students just don't care about learning--and that nothing I say or do is going to get them to care. They just want to check off a box on their way to a diploma, and want a grade that will help their GPA. I have thought long and hard about what to do next. First, of course, I will see how this second exam goes (I'm concerned). Then I will take a day of class to revisit the flipped model, my impressions thus far, and ask them for comments on their experiences. I will validate their discomfort and acknowledge that I am asking them to do something difficult. I will also offer them the opportunity to "opt out" for the last month of the semester. I am curious to see how many will do so, and I'm curious to see how that will correlate with their performance in the course.
I still believe in the flipped classroom and I know that it can drastically improve learning. I know that many of my students are having an excellent experience and appreciate how deeply they are able to engage with the course material. Still, it's discomforting to confront the reality that there are others who hate the flipped class, who don't want to be taken out of their comfort zones. Even more, it's discomforting to realize that, until the flipped class becomes the norm (or at least more pervasive) on my campus, I will always be fighting against this inclination of students to cram for exams. I can't blame them, either: it's exactly what I did as an undergraduate.
Wednesday, October 10, 2012
BlendKit 2012 Week 3: Assessments
This week, the readings and conversation in the BlendKit course focused on the issue of assessment, that is, evaluation of student performance/mastery of content. My hybrid course is a very large one, without discussion sections, so the majority of the grade is made up of midterm exams. I give 3 midterm exams, about 4 weeks apart; and then an exam during the finals period that covers the content since the third midterm + a written analysis of an ethical case study. These exams and writing assignment comprise 90% of the final grade. The additional 10% is attendance taken via i>clicker on days when we are doing ethical analysis in class.
One thing in the reading jumped out at me, and I wanted to respond to it in detail. I realize that my response might me somewhat controversial and I welcome comments that take issue with my stance. One of the suggestions in the reading was that we instructors design courses that "avoid a 'high stakes' environment that sets students up for cheating/failure." Here's the thing: I think a very big part of my job is precisely to create this sort of environment for them, and teach them how to navigate it (including how to avoid succumbing to the temptation to cheat and how to manage failure). If my students don't learn these skills in college, they are going to go out into the world and be confronted with high stakes environments, where a lot is on the line, and have no coping mechanisms, no skills for managing that level of stress. Employers will then turn around and say that colleges and universities aren't preparing students for the workplace.
Now, I am a big advocate of doing everything in my power to set students up for success: give them as many tools as possible to master the course content and practice analysis/application; make my expectations for the exam as clear as possible; hold review sessions and otherwise be available for meetings. But my students are adults and they need to learn some basic life skills, among which are how to manage high stakes situations without breaking down or cheating. Cheating, in particular, is rampant. But the solution is not to remove the incentive so long as we can control the environment enough to catch them when they do cheat. I understand why it makes sense to offer lots of low stakes assessments in large online classes--it may reduce the incentive to cheat on any particular assessment. But my students are taking exams in a classroom with proctors. We also talk about why the ethics of cheating, why cheating is a moral wrong and who gets harmed by it. If at all possible--and it *is* possible in a hybrid class--I want them to confront the temptation to cheat and be able to rationalize for themselves why it is wrong. I know that they are going to face the same temptations in their real, post-college lives and I want them to be ready to cope.
Learning how to fail is another key life skill (see this essay from Inside Higher Ed on the importance of teaching failure). It's also one that most of my students don't have when they arrive at UT, thanks to our infamous "Top 10%" rule. Most of them got here because they clawed their way to the very top of their high school classes. They avoided doing anything that would risk lowering their GPA, they fretted about grades. Failure meant that they might not get into the college of their choice, the family alma mater. Failure had seemingly huge stakes. They fear it, as if it were a monster looming under their bed waiting to devour them. They arrive at UT and, for the first time, they might fail an exam. Or get a B. They freak out, they blame the TAs, they claim that the grading was too nit-picky (aka we wanted precise answers to precise questions). These are kids who have no idea how to handle not getting an A (or, in some cases, an A+). Part of my job is to absorb their shame, their fear, their attempt to lay responsibility at some other doorstep and to force them to dig in, work harder, and realize that failing (however they define that) is not the end of the world. In fact, what matters is persistence. When they fail (by their standards), it gives me a chance to teach them about the value of persistence. And we adults all know that it is persistence that is so closely related to professional success, not perfectionism.
I have not tried to integrate online assessments into my class. I can imagine, in some future iteration, doing something like a weekly quiz instead of midterms. I am hoping to be able to create a purely online version of my class, which would certainly use weekly, randomized quizzes in place of midterms. I would also try to make use of auto-scoring software (though, for humanities, there are still a lot of kinks to be worked out on that front). I am teaching this class again in spring 2013 and fall 2013. I am thinking that, in the fall, I might add low stakes weekly quizzes in addition to the midterms. I also like the idea of student-generated questions. I might even start accepting those for my course this semester.
One thing in the reading jumped out at me, and I wanted to respond to it in detail. I realize that my response might me somewhat controversial and I welcome comments that take issue with my stance. One of the suggestions in the reading was that we instructors design courses that "avoid a 'high stakes' environment that sets students up for cheating/failure." Here's the thing: I think a very big part of my job is precisely to create this sort of environment for them, and teach them how to navigate it (including how to avoid succumbing to the temptation to cheat and how to manage failure). If my students don't learn these skills in college, they are going to go out into the world and be confronted with high stakes environments, where a lot is on the line, and have no coping mechanisms, no skills for managing that level of stress. Employers will then turn around and say that colleges and universities aren't preparing students for the workplace.
Now, I am a big advocate of doing everything in my power to set students up for success: give them as many tools as possible to master the course content and practice analysis/application; make my expectations for the exam as clear as possible; hold review sessions and otherwise be available for meetings. But my students are adults and they need to learn some basic life skills, among which are how to manage high stakes situations without breaking down or cheating. Cheating, in particular, is rampant. But the solution is not to remove the incentive so long as we can control the environment enough to catch them when they do cheat. I understand why it makes sense to offer lots of low stakes assessments in large online classes--it may reduce the incentive to cheat on any particular assessment. But my students are taking exams in a classroom with proctors. We also talk about why the ethics of cheating, why cheating is a moral wrong and who gets harmed by it. If at all possible--and it *is* possible in a hybrid class--I want them to confront the temptation to cheat and be able to rationalize for themselves why it is wrong. I know that they are going to face the same temptations in their real, post-college lives and I want them to be ready to cope.
Learning how to fail is another key life skill (see this essay from Inside Higher Ed on the importance of teaching failure). It's also one that most of my students don't have when they arrive at UT, thanks to our infamous "Top 10%" rule. Most of them got here because they clawed their way to the very top of their high school classes. They avoided doing anything that would risk lowering their GPA, they fretted about grades. Failure meant that they might not get into the college of their choice, the family alma mater. Failure had seemingly huge stakes. They fear it, as if it were a monster looming under their bed waiting to devour them. They arrive at UT and, for the first time, they might fail an exam. Or get a B. They freak out, they blame the TAs, they claim that the grading was too nit-picky (aka we wanted precise answers to precise questions). These are kids who have no idea how to handle not getting an A (or, in some cases, an A+). Part of my job is to absorb their shame, their fear, their attempt to lay responsibility at some other doorstep and to force them to dig in, work harder, and realize that failing (however they define that) is not the end of the world. In fact, what matters is persistence. When they fail (by their standards), it gives me a chance to teach them about the value of persistence. And we adults all know that it is persistence that is so closely related to professional success, not perfectionism.
I have not tried to integrate online assessments into my class. I can imagine, in some future iteration, doing something like a weekly quiz instead of midterms. I am hoping to be able to create a purely online version of my class, which would certainly use weekly, randomized quizzes in place of midterms. I would also try to make use of auto-scoring software (though, for humanities, there are still a lot of kinks to be worked out on that front). I am teaching this class again in spring 2013 and fall 2013. I am thinking that, in the fall, I might add low stakes weekly quizzes in addition to the midterms. I also like the idea of student-generated questions. I might even start accepting those for my course this semester.
Monday, October 8, 2012
The High Costs of Course Re-design: Two Challenges
When I first began to think about flipping my Intro to Rome lecture class, I had no idea what a flipped class was. I simply wanted to find a way to free up time in class for talking about and applying ethics. The obvious solution seemed to be to shift some of the content delivery/analysis (of which I do a fair amount) out of the classroom via pre-recorded videos. After much consultation with other instructors as well as IT folks, the project became more ambitious: I would, in fact, shift ALL pure content delivery outside of class. This required that I spend 5 weeks this past summer, working very long days, preparing lectures and being filmed delivering them. I've discussed this process in detail in an earlier post. After much begging, pleading, and asking around, I was able to secure funding for this summer work via the College of Liberal Arts ITS Department and the Provost's Office. I was paid 1/6 of my 9 month salary, so for about 6 weeks of work. In fact, starting on 1 June, I worked nearly every day for most of the day on this project: preparing lectures (which had to be completely reconfigured for the new medium); filming and refilming; researching studies on the flipped class and blended learning; learning how to use and integrate iclickers into my class; and designing my in class case studies, including how to present each case. Even with the start of the semester, the work has continued apace.
I have thoroughly enjoyed the intellectual challenge of redesigning a large lecture-based class to make it student-centered. My class is far better for my efforts and my students are being given the opportunity to learn better and more deeply than any previous cohort of students I have taught this course to. I find it stimulating to teach this course in a completely different way--but also exhausting. I am very grateful that my second course this semester is one that requires little management: a graduate seminar in which the primary aim is to read a significant amount of Latin prose in order to improve speed and accuracy of reading. If I were trying to teach this redesigned Rome class with a more demanding second course, I don't think I could manage. I devote hours a day to course preparation, exam writing, and solving course logistical issues. As well, I am trying to carve out some time for reflecting on what is working and what can/should be revised for the next version in Spring 2013.
I was very grateful for the summer support I received--about $4K more than if I had been teaching a 5 week class. That said, it didn't come close to compensating me for the actual amount of work I ended up doing to make sure that the course design was sound and that it would run smoothly this fall (I didn't want to do a mediocre job and have a total disaster on my hands). I received no extra support, financial or otherwise, this fall. My chair, thankfully, gave me a relatively easy-to-manage second class; but this assignment had more to do with the fact that I also supervise 10 grad student Latin teachers. I have appreciated the support from the CTP project when I have specific questions. The CTL team working for the CTP were indispensable in the summer design phase. Still, it is clear to me that UT needs to find a way to offer significantly more support--financial and otherwise--to faculty who are willing to put their research agenda on temporary hold and invest in redesigning their courses to reflect current research on student learning. Currently, whatever the official story may be, it is research production and not teaching excellence which earns the most gold stars for faculty.
I have two challenges to put forward, one to the UT administration and the other to critics of the University of Texas (and other R1 institutions) who suggest that, as Jeff Sandefer did last year and as some of our regents and members of the Texas Public Policy Foundation continue to do, current faculty are lazy, overpaid, inefficient drains on the state's economic resources. (Ok, my first challenge to the latter group would be to collect some real data about what labor faculty *actually* expend to keep the university running; this extends far, far beyond the time we spend in a classroom).
First, to the UT administration: create a program similar to CTP, but aiming at non-gateway courses; and focused primarily on designing an active learning environment in large classes (150+ students) that enroll at least 50% freshmen and sophomores. Fund it generously. Offer faculty who are chosen to participate support from CTL staff, extra TA/grading support, as well as course reduction for the first semester of teaching their new course. This will be an enormous investment in the future of the university, not least because it will change the learning experience and expectations for new students. They will take those lessons learned, those sharpened critical thinking skills, to their other classes. More than likely, they will also graduate in 4 years at higher rates (assuming they can get into the classes they need for their major) because they will have better study skills, they will have a better idea of what interests them. By investing in the design of such courses, it will be easier to add seats to already large classes without sacrificing student learning. This ability is going to be crucial in the years to come, as we continue to be asked to do more, for more students, with fewer dollars to offer additional sections of courses.
Second, to the critics of UT faculty (and post-secondary education in general at R1 institutions): put your money where your mouth is. Instead of funding studies that use bad and incomplete data to draw false and demoralizing conclusions about what is happening in classrooms around campus, spend money to help faculty devote the necessary time and energy to redesigning their courses. Spend money to hire additional staff at the Center for Teaching and Learning to support us in our attempts to improve the classroom experience for our students. Keep in mind that most of us arrive at UT with no training in how to teach a large lecture class. NONE. We learn as we go, and the vast majority of us work very hard to give our students a great learning experience even though we know those efforts will be largely unrecognized and unrewarded by anyone except our students. Spend money to support professional development for faculty, especially those of us teaching large lecture courses. Finally, more than anything, recognize that you are far more likely to achieve good results with honey than with a stick. The stick is particularly ineffective when many of us are working our $#%#es off to keep our departments running, to keep our students advised, to teach our classes, to place our graduate students in jobs, to serve on ever larger numbers of committees--all while being portrayed as hiding out in the ivory tower eating cake and growing fat off the labor of hardworking Texans.
A thoughtfully conceived and executed course that teaches large numbers of underclassmen each semester has the potential to have a very swift impact on the intellectual lives and futures of tens of thousands of UT students. The challenge is to find a way to support the time, mental and physical energy, blood, sweat, and tears that such a redesign requires. This is going to take money as well as the support of staff trained in instructional design, IT, and assessment.
I have thoroughly enjoyed the intellectual challenge of redesigning a large lecture-based class to make it student-centered. My class is far better for my efforts and my students are being given the opportunity to learn better and more deeply than any previous cohort of students I have taught this course to. I find it stimulating to teach this course in a completely different way--but also exhausting. I am very grateful that my second course this semester is one that requires little management: a graduate seminar in which the primary aim is to read a significant amount of Latin prose in order to improve speed and accuracy of reading. If I were trying to teach this redesigned Rome class with a more demanding second course, I don't think I could manage. I devote hours a day to course preparation, exam writing, and solving course logistical issues. As well, I am trying to carve out some time for reflecting on what is working and what can/should be revised for the next version in Spring 2013.
I was very grateful for the summer support I received--about $4K more than if I had been teaching a 5 week class. That said, it didn't come close to compensating me for the actual amount of work I ended up doing to make sure that the course design was sound and that it would run smoothly this fall (I didn't want to do a mediocre job and have a total disaster on my hands). I received no extra support, financial or otherwise, this fall. My chair, thankfully, gave me a relatively easy-to-manage second class; but this assignment had more to do with the fact that I also supervise 10 grad student Latin teachers. I have appreciated the support from the CTP project when I have specific questions. The CTL team working for the CTP were indispensable in the summer design phase. Still, it is clear to me that UT needs to find a way to offer significantly more support--financial and otherwise--to faculty who are willing to put their research agenda on temporary hold and invest in redesigning their courses to reflect current research on student learning. Currently, whatever the official story may be, it is research production and not teaching excellence which earns the most gold stars for faculty.
I have two challenges to put forward, one to the UT administration and the other to critics of the University of Texas (and other R1 institutions) who suggest that, as Jeff Sandefer did last year and as some of our regents and members of the Texas Public Policy Foundation continue to do, current faculty are lazy, overpaid, inefficient drains on the state's economic resources. (Ok, my first challenge to the latter group would be to collect some real data about what labor faculty *actually* expend to keep the university running; this extends far, far beyond the time we spend in a classroom).
First, to the UT administration: create a program similar to CTP, but aiming at non-gateway courses; and focused primarily on designing an active learning environment in large classes (150+ students) that enroll at least 50% freshmen and sophomores. Fund it generously. Offer faculty who are chosen to participate support from CTL staff, extra TA/grading support, as well as course reduction for the first semester of teaching their new course. This will be an enormous investment in the future of the university, not least because it will change the learning experience and expectations for new students. They will take those lessons learned, those sharpened critical thinking skills, to their other classes. More than likely, they will also graduate in 4 years at higher rates (assuming they can get into the classes they need for their major) because they will have better study skills, they will have a better idea of what interests them. By investing in the design of such courses, it will be easier to add seats to already large classes without sacrificing student learning. This ability is going to be crucial in the years to come, as we continue to be asked to do more, for more students, with fewer dollars to offer additional sections of courses.
Second, to the critics of UT faculty (and post-secondary education in general at R1 institutions): put your money where your mouth is. Instead of funding studies that use bad and incomplete data to draw false and demoralizing conclusions about what is happening in classrooms around campus, spend money to help faculty devote the necessary time and energy to redesigning their courses. Spend money to hire additional staff at the Center for Teaching and Learning to support us in our attempts to improve the classroom experience for our students. Keep in mind that most of us arrive at UT with no training in how to teach a large lecture class. NONE. We learn as we go, and the vast majority of us work very hard to give our students a great learning experience even though we know those efforts will be largely unrecognized and unrewarded by anyone except our students. Spend money to support professional development for faculty, especially those of us teaching large lecture courses. Finally, more than anything, recognize that you are far more likely to achieve good results with honey than with a stick. The stick is particularly ineffective when many of us are working our $#%#es off to keep our departments running, to keep our students advised, to teach our classes, to place our graduate students in jobs, to serve on ever larger numbers of committees--all while being portrayed as hiding out in the ivory tower eating cake and growing fat off the labor of hardworking Texans.
A thoughtfully conceived and executed course that teaches large numbers of underclassmen each semester has the potential to have a very swift impact on the intellectual lives and futures of tens of thousands of UT students. The challenge is to find a way to support the time, mental and physical energy, blood, sweat, and tears that such a redesign requires. This is going to take money as well as the support of staff trained in instructional design, IT, and assessment.
Sunday, October 7, 2012
BlendKit 2012 Week 2: Interaction
This week's reading poses the question, "is there value in interaction in every sort of class." My gut tells me that the answer to this question is yes, even if the medium of the class means that such interaction is limited to discussion boards, wikis, asynchronous webinars, etc. Frankly, I still think that face to face interaction is the standard for a majority of students. That said, it comes to mean very little when it is reduced to meaning that a student sits in the back of an auditorium in a class of 500 students and who never visits either the professor's or TAs' office hours. Personally, I have tried to reframe the question of interaction to be one of meaningful interaction. That is, how can I use the various tools available to me as well as my time in class with the students to create opportunities for meaningful interaction. I absolutely agree with the observation that learning involves risk and vulnerability. For this reason, a feeling of connection and support is crucial for a high-functioning classroom. It is my job to make sure that every student is experiencing connection and feeling supported in his/her efforts to learn--even if there are 400 students in the classroom.
In my flipped class, there are a range of different possibilities for interaction: the students watch me in videos and I teach 3 classes x 45 min. each week. I have office hours as well. They interact with each other frequently in class via peer instruction and also out of class in study groups, on Piazza, on Twitter, and on the class FB site. The greatest limit, to my mind, is just time. These students have several other classes as well as personal lives. They are constantly making choices about how to allocate their time and energy.
In the reading, various models were proposed for instructors to imagine themselves playing. For my part, I see myself as a party planner. It is my job to somehow provide that mix of choreographed "ice-breakers" and opportunities for spontaneous interaction. Sometime I notice each semester is that, as at the start of a party of people who don't know each other well, I'm doing a lot of work to bring people together. As time goes on, however, I am not needed and the party takes on a life and energy of its own. This is also a difficult role to manage: I need to still be vigilant and make sure the chip bowls are filled; but I also need to let people go around the room making their own friends and talking about subjects of interest to them.
At the same time, I am the instructor because I have a lot of experience with the subject matter. It's not enough to just bring people together and get them talking. I have to make sure that they are having meaningful interactions that advance their learning of the course content. That's no easy thing to do. Students are happy to chatter away, but it's a challenge to tap into that to get them to chatter away while thinking hard about conceptually difficult topics. Something that is proving helpful in my current course is a mix of asynchronous and synchronous conversations. It has helped them feel more comfortable with talking about course content, from my perspective, and has let them see how other students do it (especially Piazza).
I do think that we may be on the cusp of the moment when face to face interactions will less clearly be the privileged form of pedagogical interaction. Instead of being about absent/present (spatial) it will be about asynchronous/synchronous (temporal) kinds of interactions. Students are already very sophisticated in how they manage these types of interactions, and which they prefer. It is my own sense that there will never NOT be a role for the face to face student-teacher interaction when that is feasible. But we are getting very creative about finding ways to mimic some of the benefits of the f2f interaction using ed tech tools; and students themselves seem to be less interested in it than, say, my generation or even my more senior grad students' generation.
In my flipped class, there are a range of different possibilities for interaction: the students watch me in videos and I teach 3 classes x 45 min. each week. I have office hours as well. They interact with each other frequently in class via peer instruction and also out of class in study groups, on Piazza, on Twitter, and on the class FB site. The greatest limit, to my mind, is just time. These students have several other classes as well as personal lives. They are constantly making choices about how to allocate their time and energy.
In the reading, various models were proposed for instructors to imagine themselves playing. For my part, I see myself as a party planner. It is my job to somehow provide that mix of choreographed "ice-breakers" and opportunities for spontaneous interaction. Sometime I notice each semester is that, as at the start of a party of people who don't know each other well, I'm doing a lot of work to bring people together. As time goes on, however, I am not needed and the party takes on a life and energy of its own. This is also a difficult role to manage: I need to still be vigilant and make sure the chip bowls are filled; but I also need to let people go around the room making their own friends and talking about subjects of interest to them.
At the same time, I am the instructor because I have a lot of experience with the subject matter. It's not enough to just bring people together and get them talking. I have to make sure that they are having meaningful interactions that advance their learning of the course content. That's no easy thing to do. Students are happy to chatter away, but it's a challenge to tap into that to get them to chatter away while thinking hard about conceptually difficult topics. Something that is proving helpful in my current course is a mix of asynchronous and synchronous conversations. It has helped them feel more comfortable with talking about course content, from my perspective, and has let them see how other students do it (especially Piazza).
I do think that we may be on the cusp of the moment when face to face interactions will less clearly be the privileged form of pedagogical interaction. Instead of being about absent/present (spatial) it will be about asynchronous/synchronous (temporal) kinds of interactions. Students are already very sophisticated in how they manage these types of interactions, and which they prefer. It is my own sense that there will never NOT be a role for the face to face student-teacher interaction when that is feasible. But we are getting very creative about finding ways to mimic some of the benefits of the f2f interaction using ed tech tools; and students themselves seem to be less interested in it than, say, my generation or even my more senior grad students' generation.
Greek and Roman Mythology MOOC: Week 2
This week, the Coursera Greek and Roman Mythology class focused on Homer's Odyssey, Book 1-8. It was an insanely busy and stressful week for me and I had no time to re-read Homer (to my dismay). I did put aside a few hours on Saturday to watch the lectures for the week. These were chunked mostly in about 10 minute bits. I liked that. Since I was watching them one after the other, I think I'd have been ok with 20 minutes and a place to pause in the middle. But it's clear to me that 10 minutes really is a good guideline for "chunking" content delivery, even for a more "storytelling" kind of class. I have a very fast broadband connection and so didn't mind having to load new videos. It might also be that 10 minute videos are easier for students who have slower connections.
Peter Struck does an excellent job of delivering content in a memorable, clear, and interesting way. I learned a lot about Homer's Odyssey from him, and I could imagine using some of the bits I took away in classes of my own or, certainly, in interactions with our classics majors. This is a very high quality class and Dr. Struck is not "dumbing down" much of anything. I do wonder how non-English speakers are doing, particularly with the pace of the course. There is a lot to take in with each lecture: details, larger cultural history, and larger points about the narrative as a whole. As someone coming at this with a lot of previous knowledge, I find it pretty easy to keep up. I can imagine, though, that my own students would struggle a lot with the level of the class. That said, one post on the discussion board was complaining that the course didn't engage in enough substantive analysis. This criticism seemed to me entirely off the mark--it engages with analysis in a very sophisticated way, in fact; and several other students immediately responded that it was all they could do to keep up.
The quiz was based on the lecture and, apart from one question that popped up on my third quiz, it didn't ask about details not covered in the lecture. This seems like a good idea, though it does mean that students must be entirely self-motivated to do the reading. In my experience, this will mean that they mostly won't (as I didn't). This isn't the worst thing in the world--after all, a MOOC should introduce students to cool bits about ancient Greek and Roman myth. Dr. Struck also does a lot of close reading during his lectures for those students who are interested in reading. But if we assume, as many of us do, that part of what we are doing in university courses is getting students to read complex literary texts, well, that's probably not going to happen. Still, I was amazed at home much information can be taught in this format: sophisticated readings of Homer; important details about Greek culture (e.g. xenia); and an introduction to basic schools of interpretation of mythology (functionalism this week).
For this second week, I knew that I should take careful notes on the lectures as the quizzes would focus on details a bit more than on "significance" kinds of questions. One thing that I think would greatly improve the student experience of this course is embedded quiz questions. Not only would this encourage focused and active viewing of the videos, but it would help the student know what sorts of details to pay attention to. Especially in a MOOC, it seems to me very important to have frequent opportunities for students to test their understanding of the material and to receive feedback on any misunderstandings. Having student take a 20 question quiz after 90+ minutes of lecture is perhaps not the most effective way to encourage and improve learning. Specifically, it doesn't let students correct misapprehensions until after they have been "punished" by getting an answer wrong on the quiz. I am becoming a big fan of low stakes quizzing embedded in video lectures.
The quizzing system needs a bit of refinement. One question was missing a key bit of information needed to answer the question. Another question was testing material not covered in the lecture or reading. A third question was too vague and none of the answers really fit (I had taken very detailed notes and even when I consulted those, I couldn't figure out what was supposed to be the correct answer). If real grades were involved, the teaching team would have been inundated with emails and protests after both Quiz 1 and Quiz 2. As it is, I'm a little annoyed that I can't get a 20/20 because of these problems with the quiz set-up!
Peter Struck does an excellent job of delivering content in a memorable, clear, and interesting way. I learned a lot about Homer's Odyssey from him, and I could imagine using some of the bits I took away in classes of my own or, certainly, in interactions with our classics majors. This is a very high quality class and Dr. Struck is not "dumbing down" much of anything. I do wonder how non-English speakers are doing, particularly with the pace of the course. There is a lot to take in with each lecture: details, larger cultural history, and larger points about the narrative as a whole. As someone coming at this with a lot of previous knowledge, I find it pretty easy to keep up. I can imagine, though, that my own students would struggle a lot with the level of the class. That said, one post on the discussion board was complaining that the course didn't engage in enough substantive analysis. This criticism seemed to me entirely off the mark--it engages with analysis in a very sophisticated way, in fact; and several other students immediately responded that it was all they could do to keep up.
The quiz was based on the lecture and, apart from one question that popped up on my third quiz, it didn't ask about details not covered in the lecture. This seems like a good idea, though it does mean that students must be entirely self-motivated to do the reading. In my experience, this will mean that they mostly won't (as I didn't). This isn't the worst thing in the world--after all, a MOOC should introduce students to cool bits about ancient Greek and Roman myth. Dr. Struck also does a lot of close reading during his lectures for those students who are interested in reading. But if we assume, as many of us do, that part of what we are doing in university courses is getting students to read complex literary texts, well, that's probably not going to happen. Still, I was amazed at home much information can be taught in this format: sophisticated readings of Homer; important details about Greek culture (e.g. xenia); and an introduction to basic schools of interpretation of mythology (functionalism this week).
For this second week, I knew that I should take careful notes on the lectures as the quizzes would focus on details a bit more than on "significance" kinds of questions. One thing that I think would greatly improve the student experience of this course is embedded quiz questions. Not only would this encourage focused and active viewing of the videos, but it would help the student know what sorts of details to pay attention to. Especially in a MOOC, it seems to me very important to have frequent opportunities for students to test their understanding of the material and to receive feedback on any misunderstandings. Having student take a 20 question quiz after 90+ minutes of lecture is perhaps not the most effective way to encourage and improve learning. Specifically, it doesn't let students correct misapprehensions until after they have been "punished" by getting an answer wrong on the quiz. I am becoming a big fan of low stakes quizzing embedded in video lectures.
The quizzing system needs a bit of refinement. One question was missing a key bit of information needed to answer the question. Another question was testing material not covered in the lecture or reading. A third question was too vague and none of the answers really fit (I had taken very detailed notes and even when I consulted those, I couldn't figure out what was supposed to be the correct answer). If real grades were involved, the teaching team would have been inundated with emails and protests after both Quiz 1 and Quiz 2. As it is, I'm a little annoyed that I can't get a 20/20 because of these problems with the quiz set-up!
Monday, October 1, 2012
Teaching with Data
Over the past six months, as I have moved forward with redesigning and implementing my flipped class, I have adopted the phrase "evidence-based" as my mantra. Especially when it comes to teaching, we are too often forced to rely on our own impressions and uncontextualized student self-report (usually in the form of end-of-semester course evaluations, administered during one of the most stressful weeks of their semester). I have countless conversations about teaching with other faculty members, in the course of which they tell me about what their students are thinking and how their students are behaving. Of course it's possible to observe certain aspects of student behavior in the classroom. Still, I'd maintain that, too often, what we think we see isn't what is really happening. Certainly, speaking for myself, I'm too busy thinking about my own part (delivering the lecture) to really notice how many students are paying attention and taking notes; and how many are on Facebook, texting, or doing other things of that sort. This even became clear to me this past Friday. My TA was leading the review session and I sat out on the middle of the auditorium. As I looked around, I could see phones out on desks and students reading and writing occasional texts--this in a classroom where computers and tablets are banned (but apparently I wasn't specific enough?)!
One of the biggest problems in teaching large classes is getting students to read before class. They know that their chances of getting "called out" aren't very high. Certainly, the risk isn't high enough to motivate preparation. As well, in a traditional lecture format, the rewards are also not high enough to motivate preparation. After all, what's the point of knowing stuff if they can't share it or apply it?
Similarly, arguments against using lecture capture have centered on the claim that it will encourage students to skip class, save all their studying for the night before the exam, and then cram. I am sure this happens, but it is not clear that it is the norm even in a regular lecture class. Our ITS Department does a survey at the end of each semester and, at least in my class, they seem to have used the lectures as the class went along and then to review for the exam.
Thanks to lecture capture (and the fact that we can see number of views of each lecture), I know before I walk into my class how many views each lecture has had. This isn't identical to unique views and it isn't broken down to tell me how long they watched each video (they have that data but I just want a rough idea before each class). What I am seeing is that about 25% have watched the pre-recorded lectures prior to the Friday review (but, since they just finished up working on the previous week's material on Wednesday, that is a lot). By Monday, when we use that material in class, about 75% of them have watched the lectures. That's incredible and I suspect a much higher number than did the reading in previous semesters. I have some data to back this up because they consistently score about 75% accuracy on i>clicker questions quizzing the material. As the semester progresses, I will be gathering more material about when/if they do textbook readings. I do know that they are reading the primary source readings because they are able to answer questions (at about 75% accuracy) and apply it in peer discussion.
What is motivating this level of preparation? I am pretty sure it's simply the chance to demonstrate and apply it, either through i>clicker questions or peer discussion on various topics. Certainly, we saw an uptick in views in the 2 days before the exam, but it wasn't massive and seemed to indicate students reviewing the material, not encountering it for the first time.
Something I love about getting data about views before I walk into class? I can adapt how I integrate peer instruction. If I know that only 25% have viewed the videos, then I know I am going to be doing more teaching than reviewing for most of them. We still do i>clicker questions but I will often follow up a poll with a "turn to your peer" and then retake the poll and then review the question. If I know that 75% have done the viewing, then I might move through polls on basic facts a bit more quickly and focus on complex concepts. My prepared PowerPoint presentation remains the same, but I can improvise based on the numbers I get from ITS.
At the end of the semester, I will be working with ITS and some analysts from our campus Center for Teaching and Learning to engage in some more serious data analysis, in an effort to better understand student learning. We will be looking at unique views as well as patterns in how they watched the videos (including how often they closed the window with my talking head in it!). In this way, we will be able to move away from impressionistic ideas about student learning habits to data-driven, evidence-based arguments. In doing so, hopefully, we will be able to work towards a better understanding of how learning worked in my particular class; and, perhaps, start to get at describing some "best practices" for a large enrollment flipped class in the humanities.
One of the biggest problems in teaching large classes is getting students to read before class. They know that their chances of getting "called out" aren't very high. Certainly, the risk isn't high enough to motivate preparation. As well, in a traditional lecture format, the rewards are also not high enough to motivate preparation. After all, what's the point of knowing stuff if they can't share it or apply it?
Similarly, arguments against using lecture capture have centered on the claim that it will encourage students to skip class, save all their studying for the night before the exam, and then cram. I am sure this happens, but it is not clear that it is the norm even in a regular lecture class. Our ITS Department does a survey at the end of each semester and, at least in my class, they seem to have used the lectures as the class went along and then to review for the exam.
Thanks to lecture capture (and the fact that we can see number of views of each lecture), I know before I walk into my class how many views each lecture has had. This isn't identical to unique views and it isn't broken down to tell me how long they watched each video (they have that data but I just want a rough idea before each class). What I am seeing is that about 25% have watched the pre-recorded lectures prior to the Friday review (but, since they just finished up working on the previous week's material on Wednesday, that is a lot). By Monday, when we use that material in class, about 75% of them have watched the lectures. That's incredible and I suspect a much higher number than did the reading in previous semesters. I have some data to back this up because they consistently score about 75% accuracy on i>clicker questions quizzing the material. As the semester progresses, I will be gathering more material about when/if they do textbook readings. I do know that they are reading the primary source readings because they are able to answer questions (at about 75% accuracy) and apply it in peer discussion.
What is motivating this level of preparation? I am pretty sure it's simply the chance to demonstrate and apply it, either through i>clicker questions or peer discussion on various topics. Certainly, we saw an uptick in views in the 2 days before the exam, but it wasn't massive and seemed to indicate students reviewing the material, not encountering it for the first time.
Something I love about getting data about views before I walk into class? I can adapt how I integrate peer instruction. If I know that only 25% have viewed the videos, then I know I am going to be doing more teaching than reviewing for most of them. We still do i>clicker questions but I will often follow up a poll with a "turn to your peer" and then retake the poll and then review the question. If I know that 75% have done the viewing, then I might move through polls on basic facts a bit more quickly and focus on complex concepts. My prepared PowerPoint presentation remains the same, but I can improvise based on the numbers I get from ITS.
At the end of the semester, I will be working with ITS and some analysts from our campus Center for Teaching and Learning to engage in some more serious data analysis, in an effort to better understand student learning. We will be looking at unique views as well as patterns in how they watched the videos (including how often they closed the window with my talking head in it!). In this way, we will be able to move away from impressionistic ideas about student learning habits to data-driven, evidence-based arguments. In doing so, hopefully, we will be able to work towards a better understanding of how learning worked in my particular class; and, perhaps, start to get at describing some "best practices" for a large enrollment flipped class in the humanities.
Subscribe to:
Posts (Atom)