When I first started teaching large enrollment classes many moons ago, I refused to post extended notes or my PowerPoint slides. I didn't take attendance (I had no efficient way to do so pre-i>clicker) and didn't want to encourage students to skip class and then rely on a skeleton outline of the lecture. About three years ago, I realized that I needed to change my tactics. Students were pooling notes via GoogleDocs; they were using their phones to snap pics of the PPTs in class (!); and some had disabilities accommodations that entitled them to all my notes and slides (which were then magically being distributed to other students in the class who did not have the same accommodation). Around the same time that I realized I was fighting a losing battle, I chanced into teaching in a room equipped with lecture capture technology (Echo360). After some back and forth, I opted to use the lecture capture for my class lectures--and loved it. No more students asking for PPTs, no more pooling of notes. Attendance wasn't fantastic, but it wasn't significantly lower than it would have been without lecture capture. As well, because the lectures were recorded, I felt like I could hold the students to a higher standard of learning on the midterm exams.
Last fall, I used Echo360 extensively, to pre-record lectures and also to record in class lectures. Within the first few weeks of the semester, I had students asking for me to upload the PPT slides that went along with the pre-recorded lectures. In part because I had decided to give them every possible tool and then observe how they used it, I did so. I could appreciate the argument that they wanted to be able to take notes from the lectures directly on print outs of the slides. Unfortunately, it soon became clear that, when pressed for time, the students were skipping the lectures entirely and simply reading through the PPT presentations. These presentations had a fair amount of text on them, in part because I was worried I would forget important details while taping the lectures. Still, they weren't a sufficient replacement for the actual lectures.
Once again this semester, I had students asking for the PPT slides of the Echo recordings. The argument was that it was difficult to scroll through the recording to find specific details. Certainly, depending on the speed of one's connection, this can be true. It is nevertheless interesting to me that, just 18 months ago, students were delighted to have the recordings. Nobody asked for the PPTs--after all, that is part of the point of using lecture capture. I opted not to provide the PPTs this semester. I explained why I wasn't doing so and suggested some techniques for finding individual slides within a lecture. I want the students going back to the recordings; I want them listening to what I am saying and not just reading a phrase on a slide. I hope that, maybe, in going back to the lecture they will listen to longer parts of it. I am thinking that I might excerpt the images and provide those for them to review outside of the Echo recordings. But, honestly, I feel like they are lucky to have the recordings (I'm one of the few professors on campus to use this technology) and am not inclined to do anything to encourage shortcuts. Yes, it would take less time to look at a PPT slide than to scroll through a lecture; but the entire point of recording the lecture is so that they will go back through them and listen to all or parts of them, depending on where they have gaps in knowledge and understanding.
Thursday, February 14, 2013
Wednesday, February 13, 2013
What Lecture Does (and Doesn't Do)
One helpful side effect of the national dialogue about higher education is the realization that it is very difficult to teach a large enrollment class (<100 students). This realization is especially important as budgets are cut and fewer new faculty and lecturers are hired. Consequently, fewer sections of courses are offered and class sizes are in turn rapidly increasing (my own course doubled in size from 220 to 400 because of need for seats). Large enrollment courses are traditionally taught via lecture with PowerPoint, Prezi, vel sim. They are taught in large auditoriums with tiered seating and projection screens at the front of the room. To the students in the audience, the experience of going to lecture is not particularly different from that of going to a film or a live performance of a ballet or play (apart from the fact that few professors deliver their lectures in a pink tutu).
It's no wonder that students sit back in their seats and assume the role of the passive observer. Sure, they may take some notes (usually nothing beyond writing down the phrases on our presentation slides); but, on the whole, they are taking in the show rather than actively engaging in and helping to create their own learning experience on any given day. It's also no wonder that they don't learn very much--a situation not aided by their own expectation that learning happens only in the classroom. As many studies have documented, the lecture--however entertaining an individual lecturer might be--is not particularly conducive to student learning. Certainly, a powerful lecture can inspire students to go off and learn on their own; but qua teaching tool, the lecture relies on the flawed idea that knowledge and conceptual understanding can magically travel from the mind of the instructor to the mind of the student.
If lecture doesn't work very well as a learning tool (and assuming that we care about learning and not just student evaluations that reflect not their learning but their enjoyment of their classroom experience), what are we left with? More to the point, if learning--especially deep learning--requires active engagement, practice, and regular feedback, how can we facilitate that in a large enrollment classroom? The techniques of blended learning provide some of the answers: shift (some) content delivery to outside of class--and set up a system of incentives for your students to do that work; and then use class time to facilitate active learning via i>clicker and other student response systems; peer discussion; and larger class discussion. (Erik Mazur has spoken eloquently about his conversion from lecturer to facilitator of learning.)
Lecture may not be a very effective tool for the transmission of knowledge, especially more conceptual knowledge; but we should be careful not to throw out the proverbial baby with the bath water. In a large enrollment class, a class in which the instructor has very little personal contact with individual students and is often a mysterious figure at the front of the class, lecture *does* transmit a tremendous amount of significant information. First and foremost, it gives the students a sense of connection to the course. It gives them a sense of who the instructor is, and lets us construct a professorial self for the students. Sure, this can be done to some extent via pre-recorded lectures, but, given that we lack acting experience, it's difficult to do more than deliver content. It feels weird to crack jokes and we have no audience to play off of.
I was reminded of this benefit of the in class lecture yesterday when I told my class the story of the Roman republican consul Regulus. Regulus was captured by the Carthaginians and, eventually, killed in rather gruesome fashion. The stories vary: in some versions, he was put in a box and spikes were driven in until he stopped screaming in agony. In others, he was put in a spiked barrel and rolled down the hill. After warning my class that I had a dark sense of humor, I told this story to my class with a certain amount of glee, emphasizing the gruesome aspects. As I told the story, I watched their reactions and adjusted accordingly. After class, one of my TAs found a great medieval illustration of Regulus' death, which we posted on Piazza. This led to additional commentary by one of the students, who noted that I told the story in class with a smile on my face. This exchange was such a reminder for me of why a certain kind of lecture matters. I try to avoid using class time for the rehearsal of facts and figures, focusing instead of stories and interpretation. But, after shifting all content--including the stories--outside of class last semester, I realized that I had thrown out the baby. This semester, I've tried to leave the bathwater outside of class while bringing the baby back into class.
To my mind the truly significant challenge of teaching a large enrollment class these days is figuring out how to make the best use of technology. To do so requires an intimate understanding of how students learn; and of what each learning tool can and cannot do. It requires figuring out what to use the in class lecture for (in the case of my course, story-telling and modeling interpretation); and what to shift to outside of class. It requires having a good sense of students' tolerance for watching lectures out of class. It requires understanding their behavior and working with it to create an assessment structure that rewards them for making good choices. It used to be that large lecture classes were a kind of plum for distinguished professors. They walked into class, awed the students with their learning and, in some cases, their entertainment skills, and then went back to their research. They had a team of teaching assistants to handle student questions (though, of course, instructors also held office hours) and to grade the 3-4 midterm exams. These days, effective teaching in a large enrollment class requires enormous effort and a significant investment of time by the entire teaching team, from the instructor down to undergraduate student teaching assistants. The in class lecture is just a small but significant part of the package of learning tools we instructors provide our students.
It's no wonder that students sit back in their seats and assume the role of the passive observer. Sure, they may take some notes (usually nothing beyond writing down the phrases on our presentation slides); but, on the whole, they are taking in the show rather than actively engaging in and helping to create their own learning experience on any given day. It's also no wonder that they don't learn very much--a situation not aided by their own expectation that learning happens only in the classroom. As many studies have documented, the lecture--however entertaining an individual lecturer might be--is not particularly conducive to student learning. Certainly, a powerful lecture can inspire students to go off and learn on their own; but qua teaching tool, the lecture relies on the flawed idea that knowledge and conceptual understanding can magically travel from the mind of the instructor to the mind of the student.
If lecture doesn't work very well as a learning tool (and assuming that we care about learning and not just student evaluations that reflect not their learning but their enjoyment of their classroom experience), what are we left with? More to the point, if learning--especially deep learning--requires active engagement, practice, and regular feedback, how can we facilitate that in a large enrollment classroom? The techniques of blended learning provide some of the answers: shift (some) content delivery to outside of class--and set up a system of incentives for your students to do that work; and then use class time to facilitate active learning via i>clicker and other student response systems; peer discussion; and larger class discussion. (Erik Mazur has spoken eloquently about his conversion from lecturer to facilitator of learning.)
Lecture may not be a very effective tool for the transmission of knowledge, especially more conceptual knowledge; but we should be careful not to throw out the proverbial baby with the bath water. In a large enrollment class, a class in which the instructor has very little personal contact with individual students and is often a mysterious figure at the front of the class, lecture *does* transmit a tremendous amount of significant information. First and foremost, it gives the students a sense of connection to the course. It gives them a sense of who the instructor is, and lets us construct a professorial self for the students. Sure, this can be done to some extent via pre-recorded lectures, but, given that we lack acting experience, it's difficult to do more than deliver content. It feels weird to crack jokes and we have no audience to play off of.
I was reminded of this benefit of the in class lecture yesterday when I told my class the story of the Roman republican consul Regulus. Regulus was captured by the Carthaginians and, eventually, killed in rather gruesome fashion. The stories vary: in some versions, he was put in a box and spikes were driven in until he stopped screaming in agony. In others, he was put in a spiked barrel and rolled down the hill. After warning my class that I had a dark sense of humor, I told this story to my class with a certain amount of glee, emphasizing the gruesome aspects. As I told the story, I watched their reactions and adjusted accordingly. After class, one of my TAs found a great medieval illustration of Regulus' death, which we posted on Piazza. This led to additional commentary by one of the students, who noted that I told the story in class with a smile on my face. This exchange was such a reminder for me of why a certain kind of lecture matters. I try to avoid using class time for the rehearsal of facts and figures, focusing instead of stories and interpretation. But, after shifting all content--including the stories--outside of class last semester, I realized that I had thrown out the baby. This semester, I've tried to leave the bathwater outside of class while bringing the baby back into class.
To my mind the truly significant challenge of teaching a large enrollment class these days is figuring out how to make the best use of technology. To do so requires an intimate understanding of how students learn; and of what each learning tool can and cannot do. It requires figuring out what to use the in class lecture for (in the case of my course, story-telling and modeling interpretation); and what to shift to outside of class. It requires having a good sense of students' tolerance for watching lectures out of class. It requires understanding their behavior and working with it to create an assessment structure that rewards them for making good choices. It used to be that large lecture classes were a kind of plum for distinguished professors. They walked into class, awed the students with their learning and, in some cases, their entertainment skills, and then went back to their research. They had a team of teaching assistants to handle student questions (though, of course, instructors also held office hours) and to grade the 3-4 midterm exams. These days, effective teaching in a large enrollment class requires enormous effort and a significant investment of time by the entire teaching team, from the instructor down to undergraduate student teaching assistants. The in class lecture is just a small but significant part of the package of learning tools we instructors provide our students.
Tuesday, February 12, 2013
Early Warning System: The Low Tech Version
The integration of weekly quizzes to the assessments in my 400-student Intro to Rome course has had many benefits. First and foremost, it requires the students to stay on top of the course material and learn it cumulatively, week by week, rather than trying to stuff it into their brains over a 48 hour, Red Bull-fueled cram session. This means that, for all intents and purposes, my students are flipping. They are therefore able to engage actively in class and make good use of the opportunities for reviewing and applying content in class, under my guidance. The quizzes, which are machine graded, also provide the students with weekly feedback about their learning strategies. I take care that the quizzes are challenging, so that there is good separation among the students; and so that they do not get lulled into thinking that they can get away with not knowing the week's material deeply. So far, this is working well. We saw many of the students making adjustments from the first quiz to the second quiz, and we also saw many students performing consistently well from quiz to quiz.
The other significant benefit of these weekly quizzes is the information it provides to the teaching team. We are now starting the fifth week of the semester--1/3 of the way into the course. The class has its first midterm on the 19th of February. Those students who are scoring 10-12 on the quizzes are likely in good shape for the midterm and will be able to prepare with relatively little extra work, just a few hours of careful review. Those students who are scoring around 50% on the first two quizzes, however, are likely not in such good shape. In an effort to get them to confront this now, while there is still time to either a. drop the class before a significant investment of time and energy (which also detracts from their study for their other classes); or b. fully commit and bring up their grade, I had one of my teaching assistants go through the gradebook manually and identify students who had scored 50% or lower on both of the first two quizzes.
These "at risk" students--23 out of 387 (though there were about 20 more on the bubble)--were sent a short email expressing our concern and urging them to contact their assigned TA or me; and reminding them that we were there to help them be successful in the course. I have emailed all of the TAs a list of the students who received an "early warning" email and will check to see how many follow-up before the first midterm; and, also, how many are able to raise their scores on their own. We will likely do something similar after the first midterm. Our goal is to let these underperforming students know that there are resources to help them; and to do what we can to force them out of their denial, out of their belief that somehow everything will be ok without them actively making changes to their learning behaviors.
We also hope that some of these students will recognize that perhaps they should drop the course. We want them to drop as early in the term as possible, so that they aren't wasting time on a class they aren't going to finish. Nothing is more frustrating than signing a drop form for a student who has limped along for months and then, a week before the end of the semester, finally admitted that they aren't going to raise their grade. It is a waste of the student's time but also of the time and resources of the teaching time. Often, we will invest hours of time working with these students, knowing that they are likely to drop but unable to convince them to do so. One goal of these early warnings is to force students to be honest with themselves, to force them to decide whether they are willing to make the necessary changes to their learning behaviors for my class.
Someday, hopefully in the near future, this early warning process will be mechanized, built into any LMS. I will be able to set it to send warnings that I have written. I can even imagine a time when the system will be able to develop algorithms for particular courses, on the basis of several years of data. It will be able to identify less obvious at risk students because the data will tell us that students who perform a certain way on early quizzes are likely to score a certain way on later assessments. After a decade of teaching and several iterations of this Rome class, I have an intuitive sense of these algorithms. But I can imagine that data (not necessarily BIG DATA, but just a few years of 1000 students/year) will eventually be able to predict with some reasonable accuracy how students are likely to perform in a course after just a few assessments. I can imagine that this will give me a big stick to use in persuading at risk students to change their approach (or decide to drop the class).
As for the success of my low-tech early interventions, stay tuned....
The other significant benefit of these weekly quizzes is the information it provides to the teaching team. We are now starting the fifth week of the semester--1/3 of the way into the course. The class has its first midterm on the 19th of February. Those students who are scoring 10-12 on the quizzes are likely in good shape for the midterm and will be able to prepare with relatively little extra work, just a few hours of careful review. Those students who are scoring around 50% on the first two quizzes, however, are likely not in such good shape. In an effort to get them to confront this now, while there is still time to either a. drop the class before a significant investment of time and energy (which also detracts from their study for their other classes); or b. fully commit and bring up their grade, I had one of my teaching assistants go through the gradebook manually and identify students who had scored 50% or lower on both of the first two quizzes.
These "at risk" students--23 out of 387 (though there were about 20 more on the bubble)--were sent a short email expressing our concern and urging them to contact their assigned TA or me; and reminding them that we were there to help them be successful in the course. I have emailed all of the TAs a list of the students who received an "early warning" email and will check to see how many follow-up before the first midterm; and, also, how many are able to raise their scores on their own. We will likely do something similar after the first midterm. Our goal is to let these underperforming students know that there are resources to help them; and to do what we can to force them out of their denial, out of their belief that somehow everything will be ok without them actively making changes to their learning behaviors.
We also hope that some of these students will recognize that perhaps they should drop the course. We want them to drop as early in the term as possible, so that they aren't wasting time on a class they aren't going to finish. Nothing is more frustrating than signing a drop form for a student who has limped along for months and then, a week before the end of the semester, finally admitted that they aren't going to raise their grade. It is a waste of the student's time but also of the time and resources of the teaching time. Often, we will invest hours of time working with these students, knowing that they are likely to drop but unable to convince them to do so. One goal of these early warnings is to force students to be honest with themselves, to force them to decide whether they are willing to make the necessary changes to their learning behaviors for my class.
Someday, hopefully in the near future, this early warning process will be mechanized, built into any LMS. I will be able to set it to send warnings that I have written. I can even imagine a time when the system will be able to develop algorithms for particular courses, on the basis of several years of data. It will be able to identify less obvious at risk students because the data will tell us that students who perform a certain way on early quizzes are likely to score a certain way on later assessments. After a decade of teaching and several iterations of this Rome class, I have an intuitive sense of these algorithms. But I can imagine that data (not necessarily BIG DATA, but just a few years of 1000 students/year) will eventually be able to predict with some reasonable accuracy how students are likely to perform in a course after just a few assessments. I can imagine that this will give me a big stick to use in persuading at risk students to change their approach (or decide to drop the class).
As for the success of my low-tech early interventions, stay tuned....
Monday, February 11, 2013
Practice Makes Perfect
In his book Outliers, Malcolm Gladwell famously makes the claim that the key to success in any field is putting in around 10,000 hours of practice. Native intelligence matters, but not nearly as much as many people think. Practice--and the willingness to practice long hours--is often what sets apart the very highest achievers. Indeed, one of the guiding principles of the flipped classroom is that practice makes perfect--or, at least, makes for deeper learning and better grades. Yet practice is often what our students resist the most and are least well-trained to do. By shifting content delivery out of class, we instructors can be present with our students while they practice recalling and applying the content they learned outside of class.
This in class practice takes many forms: i>clicker questions and opinion polls; class discussion; and peer instruction. As more BYOD technologies come on the market, like Learning Catalytics, I expect that the in class experience will become ever more interactive and sophisticated. This in class practice of content is crucial to student learning. In a history class like mine, however, much of this practice depends first of all on a clear understanding and ability to recall the basic facts. Thus, I need to spend at least some time making sure that they are understanding key concepts and ideas (and highlighting certain people/events/concepts as important to know).
Unfortunately, these sorts of questions can become tedious very quickly, even when discussion questions are interspersed. In the fall, I saw that the students came to dislike the i>clicker questions and viewed them as nothing more than a way for me to take attendance (and catch them out for coming to class late or leaving early). Part of the problem was that my i>clicker questions focused too much on quizzing facts. In redesigning the class for the spring, I realized that I could kill two (or, really, about five) birds with one stone by integrating weekly short quizzes into the assessments. In addition to the graded 12 question quiz (10 min.), I also post a practice quiz of about 25 questions. I draw a few of the graded quiz questions from the practice quiz to incentivize the students to do the practice quiz. By shifting this kind of content practice to a highly structured online activity, I am able to use class time to focus i>clicker questions on things other than simple recall (though I still include 1-2 of those just to keep students on their toes). Now I use them to initiate a debate, to get a discussion of a complex issue started (e.g. should the Romans have sowed Carthage's fields with salt?)
Something I am realizing is that, for my subject, there are different kinds of practice that are necessary for success in the course. Some kinds, such as recall of basic facts, is better done online while practice with conceptual knowledge and application of those facts is best done in class or on the online discussion board. The challenge is motivating the students to engage in the practice when they aren't being observed. My solution, which seems to be working very well thus far, was to let them see how much easier the graded quiz is when they have done the practice quiz, seen some of the questions, etc. We can see on Blackboard which students have done the practice quiz (but not anything about their results--I deliberately made that hidden so that they would not feel self-conscious). One of my TAs is keeping track of this data so that we can correlate it with quiz performance.
On this same issue, we are releasing a data bank of possible short answer questions, about 100 for the first midterm. From this set of 100 questions, we will take 8 for the exam. Once again, the message we are trying to send is that learning is about practice and effort. We expect that the students will create a Google doc for the class and that some students will try to just memorize the answers. I am confident, though, that the number of potential questions is large enough that if one is just trying to memorize answers, they are unlikely to stick in the memory. Each week, we will add more and more questions to this databank, and the short answer portion of the exam is cumulative so they will have to keep that knowledge relatively fresh. But, if they put in the practice, they will almost certainly do well on the assessments.
This in class practice takes many forms: i>clicker questions and opinion polls; class discussion; and peer instruction. As more BYOD technologies come on the market, like Learning Catalytics, I expect that the in class experience will become ever more interactive and sophisticated. This in class practice of content is crucial to student learning. In a history class like mine, however, much of this practice depends first of all on a clear understanding and ability to recall the basic facts. Thus, I need to spend at least some time making sure that they are understanding key concepts and ideas (and highlighting certain people/events/concepts as important to know).
Unfortunately, these sorts of questions can become tedious very quickly, even when discussion questions are interspersed. In the fall, I saw that the students came to dislike the i>clicker questions and viewed them as nothing more than a way for me to take attendance (and catch them out for coming to class late or leaving early). Part of the problem was that my i>clicker questions focused too much on quizzing facts. In redesigning the class for the spring, I realized that I could kill two (or, really, about five) birds with one stone by integrating weekly short quizzes into the assessments. In addition to the graded 12 question quiz (10 min.), I also post a practice quiz of about 25 questions. I draw a few of the graded quiz questions from the practice quiz to incentivize the students to do the practice quiz. By shifting this kind of content practice to a highly structured online activity, I am able to use class time to focus i>clicker questions on things other than simple recall (though I still include 1-2 of those just to keep students on their toes). Now I use them to initiate a debate, to get a discussion of a complex issue started (e.g. should the Romans have sowed Carthage's fields with salt?)
Something I am realizing is that, for my subject, there are different kinds of practice that are necessary for success in the course. Some kinds, such as recall of basic facts, is better done online while practice with conceptual knowledge and application of those facts is best done in class or on the online discussion board. The challenge is motivating the students to engage in the practice when they aren't being observed. My solution, which seems to be working very well thus far, was to let them see how much easier the graded quiz is when they have done the practice quiz, seen some of the questions, etc. We can see on Blackboard which students have done the practice quiz (but not anything about their results--I deliberately made that hidden so that they would not feel self-conscious). One of my TAs is keeping track of this data so that we can correlate it with quiz performance.
On this same issue, we are releasing a data bank of possible short answer questions, about 100 for the first midterm. From this set of 100 questions, we will take 8 for the exam. Once again, the message we are trying to send is that learning is about practice and effort. We expect that the students will create a Google doc for the class and that some students will try to just memorize the answers. I am confident, though, that the number of potential questions is large enough that if one is just trying to memorize answers, they are unlikely to stick in the memory. Each week, we will add more and more questions to this databank, and the short answer portion of the exam is cumulative so they will have to keep that knowledge relatively fresh. But, if they put in the practice, they will almost certainly do well on the assessments.
Sunday, February 10, 2013
The Power (and Problem) of Discomfort
Educators know that moments of discomfort are very often also moments of significant learning. Still, it can be difficult to persuade our students of this principle, and even more difficult to persuade them that it is normal to experience discomfort during a class--and that this doesn't necessarily mean that the instructor is doing something wrong. At the same time, I am learning, we educators must also appreciate the value of comfort to the learning experience. The tricky part is finding a good balance between letting students settle in and feel comfortable in a class; and forcing them out of their comfort zones from time to time.
In the fall semester version of the flipped class, I did the equivalent of throwing the students into the deep end. I did explain to them what a flipped class was and how it would improve their learning, but did not really appreciate the extent to which their experience of a flipped class would be so disorienting. In part, the disorientation came from the space of the classroom itself--a very large and badly designed lecture hall. It is the kind of room one walks into and expects to hear a lecture from the sage on the stage. At the front of the class, there is a small podium and two projection screens. The design of the room encourages focus to be on that stage at the front, and it is very difficult to navigate the seats in the room. When I didn't lecture, students were confused--even if I had told them at the outset that I was not going to lecture, they somehow still expected it and wanted it. They also struggled to understand how the i>clicker and discussion questions connected to their out of class work. To them, it just seemed like a review. Even when I tried to explain that I was using the i>clicker polls to check comprehension of difficult concepts; and using peer discussion and class discussion to do higher order analytical thinking, they didn't get it.
Going into the fall semester, it never occurred to me that shifting lectures to online would result in such a high level of student disorientation and discomfort. Yet it clearly did. As well, it was at a level that was not conducive to learning. Rather, it was causing frustration, anxiety, and rebellion against the flipped class model. For my part, I was blindsided and struggled to make sense of their discomfort. It was difficult for me to understand why they would be so disoriented by such a simple shift of content. I didn't understand why they struggled to make what seemed to me to be a relatively small cognitive adjustment. Ultimately, I suspect that there were many causes of their discomfort, including their own choice to cram for exams instead of preparing the work as assigned. Still, it was incredible to observe how much power the space of the classroom exerts on student expectations.
In re-designing the class for the spring, I took very seriously the issue of student (dis)orientation. I knew that the only way I could push them up against the edges of their comfort zones was if I let them find a comfort zone. To that end, much of the first month of this semester has been devoted to orienting the students and meeting their explicit and implicit expectations. I am lecturing (though I will be gradually shifting away from in class lecture in the coming weeks); every exercise that the students are asked to do has a practice component to it (practice quizzes, they can post until they are satisfied with the grades that will count) and was modeled for them by the teaching team. I have created numerous written documents and recordings to explain to them expectations for different elements of the course. A substantial amount of structure has been added to the course, so that students know each week what they will be doing on a given day.
As we start the fifth week, I am pleased with the redesign (but am also keeping in mind that, at this point in the fall, everything was going smoothly too). I do feel like the students are much more oriented and that they have fallen into the weekly rhythm of the course. We are getting very few logistical questions (unlike last fall, when we got many for weeks after the start of the semester) and my sense is that the students' know my expectations of them and that I am meeting their general expectations as well. I have worked hard to let them settle in to the class and feel a sense of comfort, perhaps even make some friends. Then, once they have that sense of comfort, I can chip away at it from time to time. I can encourage them to move out of their comfort zone, into a state of discomfort, where most formative learning happens. I've come to appreciate, however, that the power of discomfort for the learning process only exists if the students start from a place of comfort.
In the fall semester version of the flipped class, I did the equivalent of throwing the students into the deep end. I did explain to them what a flipped class was and how it would improve their learning, but did not really appreciate the extent to which their experience of a flipped class would be so disorienting. In part, the disorientation came from the space of the classroom itself--a very large and badly designed lecture hall. It is the kind of room one walks into and expects to hear a lecture from the sage on the stage. At the front of the class, there is a small podium and two projection screens. The design of the room encourages focus to be on that stage at the front, and it is very difficult to navigate the seats in the room. When I didn't lecture, students were confused--even if I had told them at the outset that I was not going to lecture, they somehow still expected it and wanted it. They also struggled to understand how the i>clicker and discussion questions connected to their out of class work. To them, it just seemed like a review. Even when I tried to explain that I was using the i>clicker polls to check comprehension of difficult concepts; and using peer discussion and class discussion to do higher order analytical thinking, they didn't get it.
Going into the fall semester, it never occurred to me that shifting lectures to online would result in such a high level of student disorientation and discomfort. Yet it clearly did. As well, it was at a level that was not conducive to learning. Rather, it was causing frustration, anxiety, and rebellion against the flipped class model. For my part, I was blindsided and struggled to make sense of their discomfort. It was difficult for me to understand why they would be so disoriented by such a simple shift of content. I didn't understand why they struggled to make what seemed to me to be a relatively small cognitive adjustment. Ultimately, I suspect that there were many causes of their discomfort, including their own choice to cram for exams instead of preparing the work as assigned. Still, it was incredible to observe how much power the space of the classroom exerts on student expectations.
In re-designing the class for the spring, I took very seriously the issue of student (dis)orientation. I knew that the only way I could push them up against the edges of their comfort zones was if I let them find a comfort zone. To that end, much of the first month of this semester has been devoted to orienting the students and meeting their explicit and implicit expectations. I am lecturing (though I will be gradually shifting away from in class lecture in the coming weeks); every exercise that the students are asked to do has a practice component to it (practice quizzes, they can post until they are satisfied with the grades that will count) and was modeled for them by the teaching team. I have created numerous written documents and recordings to explain to them expectations for different elements of the course. A substantial amount of structure has been added to the course, so that students know each week what they will be doing on a given day.
As we start the fifth week, I am pleased with the redesign (but am also keeping in mind that, at this point in the fall, everything was going smoothly too). I do feel like the students are much more oriented and that they have fallen into the weekly rhythm of the course. We are getting very few logistical questions (unlike last fall, when we got many for weeks after the start of the semester) and my sense is that the students' know my expectations of them and that I am meeting their general expectations as well. I have worked hard to let them settle in to the class and feel a sense of comfort, perhaps even make some friends. Then, once they have that sense of comfort, I can chip away at it from time to time. I can encourage them to move out of their comfort zone, into a state of discomfort, where most formative learning happens. I've come to appreciate, however, that the power of discomfort for the learning process only exists if the students start from a place of comfort.
Friday, February 8, 2013
The Teaching Team, or Adventures in Management
Another significant and (for me, at least) unanticipated challenge of teaching 400 students in a flipped class is managing the team of teaching assistants and graders. The first challenge was to persuade my college that I needed more than two graduate student teaching assistants. I spent the better part of two months campaigning myself and getting every administrator I could find to send emails to my dean explaining that two teaching assistants for 400 students was not going to work. Finally, after several weeks of sweating it out, I was informed that I would have four teaching assistants. I also persuaded my department chair to pay two advanced undergraduate students a small stipend to work as graders. This meant that, in the fall, I had a team of 6 to work with: 4 graduate TAs and 2 undergraduate graders (who only graded and otherwise had no interactions with the students). I depend on my department chair to assign me specific students and I was given two fairly advanced students and two brand new MA students to work with.
I understood early on that, with a team of this size and with a range of jobs needing done, it would make sense to distribute duties. I decided that one of the two advanced grad students would be in charge of conducting the weekly review sessions (held during classtime on Fridays). The other would be the grade czar, coordinating all the grading and dealing with students who had questions about exam grading. The MA students would come to class and help with in class activities; and would grade 1/4 of the exams apiece. The two graders would grade 1/4 of the exams each. This system worked fairly well at first, until suddenly my "Friday Review" TA was reassigned to another course. This meant that I was now in front of the class MWF. It also meant that I had one fewer TAs to assist with classroom management, and as the semester progressed, it became clear that I really needed to have a second, experienced TA. Losing this TA meant that I was now spending a lot of time dealing with issues and questions that should have been directed to an experienced TA (this became even more of an issue because of the inexperience of the two MA students).
The other complication that came up was in grading non-exam work. During the semester, I offered the students the chance to do extra credit twice, in an effort to get them to learn some key concepts. Unfortunately, this meant that 400 written assignments needed to be graded. In retrospect, I should have involved several of the other TAs. Instead, I graded them myself. I spent the final 5 weeks of the course grading 800 written assignment + a final exam with an extensive written section. I did have some help with the final exam grading, thank heavens. But, by the time grades were submitted, I was completely exhausted. I had also learned some very important lessons about managing a teaching team in such a large class.
This spring, I once again have a team of 6: 4 graduate TAs and 2 undergraduate graders. In this revised version of the course, however, I have made significant changes also to how I utilize my teaching team. I have a "head TA", who is in charge of grades (the same TA I had in the fall, which has been a huge boon). I have another very experienced graduate student who manages all things ethics. When the students hand in worksheets connected to the ethics case studies, she records them and comments on them. She will review their ethics portfolios mid-semester and grade them (with assistance from me and another TA) at the end of the semester. She is the person to whom students can take their ethics questions. She also hands out candy to students who participate during class. A third TA will be working virtually (she is based in Iowa this semester). Her job is to manage and grade the discussion threads on Piazza. She has some help from a volunteer TA--a former student of mine who is now in medical school but enjoys keeping up with her Roman history/culture. My fourth TA helps with things like bringing scantrons for quizzes, making copies of exams, and he will grade 1/3 of the exams. The two undergraduate graders will each grade 1/3 of the exams. As well, I have divided the class into thirds and assigned each third to one of the three campus-based TAs to serve as their "point-person", that is, the TA they should think of as their own.
So far, I am very pleased with my new system of role distribution. Each TA knows very clearly what their responsibilities are; they are very good about doing their job with minimal drama; and thus far, I have not been inundated with questions from students that should be taken to the TAs. It also helps, I suspect, that I set up a special email account for this class, so that I can easily find (and forward to the right TA) the emails that come in from students. Certainly, some issues have to be handled by me. But many are things that are easily handled by a teaching assistant.
I don't think I appreciated at all how important it would be to develop some basic "best practices" for managing a larger team of TAs. With two TAs, it was pretty simple to just have them do the same thing and share the workload. But with a larger class and a more complex course design, it suddenly requires a much clearer division of specific roles. The other thing it requires is for me to leave the TAs alone to do their job. I have a tendency to micromanage, to fret. I am slowly but surely learning that, for me to do my part of the course well, I have to assume that my TAs are doing their parts and not waste mental energy double-checking or intervening. Or, worse, deciding that I am the only one who can do a particular task (as happened when I graded 800 written assignments in the fall).
As faculty, not only are we not particularly trained in pedagogy; but we are not trained in basic management. It can be a pretty big adjustment to work with teaching assistants, and to figure out how to make good use of them (without overusing them, of course!). That challenge becomes even greater as the class size and, consequently, the size of the teaching team, increases. Eventually, I'd like to have an even larger teaching team, probably by adding more undergraduate student TAs who would come to class and help with discussion activities. Other departments at my university have done this very successfully; yet it is clear that this is successful because a lot of time is devoted to mentoring and training. Still, as we move away from traditional lecture and towards more active, student-centered classrooms, it will be imperative to have a fairly large teaching team--and that will require substantial management, division of roles, and basic oversight on the part of the instructor.
I understood early on that, with a team of this size and with a range of jobs needing done, it would make sense to distribute duties. I decided that one of the two advanced grad students would be in charge of conducting the weekly review sessions (held during classtime on Fridays). The other would be the grade czar, coordinating all the grading and dealing with students who had questions about exam grading. The MA students would come to class and help with in class activities; and would grade 1/4 of the exams apiece. The two graders would grade 1/4 of the exams each. This system worked fairly well at first, until suddenly my "Friday Review" TA was reassigned to another course. This meant that I was now in front of the class MWF. It also meant that I had one fewer TAs to assist with classroom management, and as the semester progressed, it became clear that I really needed to have a second, experienced TA. Losing this TA meant that I was now spending a lot of time dealing with issues and questions that should have been directed to an experienced TA (this became even more of an issue because of the inexperience of the two MA students).
The other complication that came up was in grading non-exam work. During the semester, I offered the students the chance to do extra credit twice, in an effort to get them to learn some key concepts. Unfortunately, this meant that 400 written assignments needed to be graded. In retrospect, I should have involved several of the other TAs. Instead, I graded them myself. I spent the final 5 weeks of the course grading 800 written assignment + a final exam with an extensive written section. I did have some help with the final exam grading, thank heavens. But, by the time grades were submitted, I was completely exhausted. I had also learned some very important lessons about managing a teaching team in such a large class.
This spring, I once again have a team of 6: 4 graduate TAs and 2 undergraduate graders. In this revised version of the course, however, I have made significant changes also to how I utilize my teaching team. I have a "head TA", who is in charge of grades (the same TA I had in the fall, which has been a huge boon). I have another very experienced graduate student who manages all things ethics. When the students hand in worksheets connected to the ethics case studies, she records them and comments on them. She will review their ethics portfolios mid-semester and grade them (with assistance from me and another TA) at the end of the semester. She is the person to whom students can take their ethics questions. She also hands out candy to students who participate during class. A third TA will be working virtually (she is based in Iowa this semester). Her job is to manage and grade the discussion threads on Piazza. She has some help from a volunteer TA--a former student of mine who is now in medical school but enjoys keeping up with her Roman history/culture. My fourth TA helps with things like bringing scantrons for quizzes, making copies of exams, and he will grade 1/3 of the exams. The two undergraduate graders will each grade 1/3 of the exams. As well, I have divided the class into thirds and assigned each third to one of the three campus-based TAs to serve as their "point-person", that is, the TA they should think of as their own.
So far, I am very pleased with my new system of role distribution. Each TA knows very clearly what their responsibilities are; they are very good about doing their job with minimal drama; and thus far, I have not been inundated with questions from students that should be taken to the TAs. It also helps, I suspect, that I set up a special email account for this class, so that I can easily find (and forward to the right TA) the emails that come in from students. Certainly, some issues have to be handled by me. But many are things that are easily handled by a teaching assistant.
I don't think I appreciated at all how important it would be to develop some basic "best practices" for managing a larger team of TAs. With two TAs, it was pretty simple to just have them do the same thing and share the workload. But with a larger class and a more complex course design, it suddenly requires a much clearer division of specific roles. The other thing it requires is for me to leave the TAs alone to do their job. I have a tendency to micromanage, to fret. I am slowly but surely learning that, for me to do my part of the course well, I have to assume that my TAs are doing their parts and not waste mental energy double-checking or intervening. Or, worse, deciding that I am the only one who can do a particular task (as happened when I graded 800 written assignments in the fall).
As faculty, not only are we not particularly trained in pedagogy; but we are not trained in basic management. It can be a pretty big adjustment to work with teaching assistants, and to figure out how to make good use of them (without overusing them, of course!). That challenge becomes even greater as the class size and, consequently, the size of the teaching team, increases. Eventually, I'd like to have an even larger teaching team, probably by adding more undergraduate student TAs who would come to class and help with discussion activities. Other departments at my university have done this very successfully; yet it is clear that this is successful because a lot of time is devoted to mentoring and training. Still, as we move away from traditional lecture and towards more active, student-centered classrooms, it will be imperative to have a fairly large teaching team--and that will require substantial management, division of roles, and basic oversight on the part of the instructor.
Thursday, February 7, 2013
Weekly Quizzes
One of the most significant changes I made to my 400 student Intro to Rome course this spring was to the assessment activities and structure. I realized that if I wanted to send the mess age to students that, in a history class, learning was cumulative and not something that could be done effectively 48 hours before a midterm exam, I needed to put my percentages where my mouth was. So I drastically reduced the weight placed on midterm exams and added in 10-minute, weekly quizzes. The three cumulative midterms are worth 40% of the final grade. The 9 quizzes (8 of which count towards the final grade) are also worth 40% of the final grade. My students are very attuned to grades. They care a lot about them and will, generally, work hard to get a good one. By setting up the assessment this way, I am taking a characteristic of their learning behavior and using it to try to inculcate good study skills (or, at least, an approach to the course that is far more likely to help them learn the content in the short run and remember more of it in the long run). I don't love the fact that years of standardized testing has trained my students to be overly focused on outcome, at the expense of the process. Yet, with a lot of hard thinking and a bit of creativity, I feel like I have found a way to turn that potentially negative learning behavior into a positive. They care about outcomes, so channel that into weekly studying; and give them weekly feedback on how well their learning strategies are working so that they can make changes (if necessary) before the midterm exam.
In the fall, it was clear that I needed to implement weekly quizzes to motivate the students to stay caught up and, therefore, able to be active and engaged learners. With weekly quizzes, they really cannot get more than a week behind without significant consequences for their course grade. It was interesting that, in one of the fall student surveys, a common suggestion for improving the course was to incorporate weekly quizzes. As well, in information conversations in my office with various students from the fall course, they were clear that they needed the outside motivation of a graded quiz to "force" them to do assigned work. They knew that it was better for their learning to do the assigned work incrementally rather than try to cram for the exam, but seemed incapable of managing their time in a way that allowed them to avoid cramming.
The real obstacle to incorporating graded, weekly quizzes into the class was logistics. It is a big deal (and takes about 5-7 minutes) to hand out 400 scantron sheets and then collect them. It uses a lot of paper, especially if we also hand out copies of the quiz. I talked to a lot of people about possible ways to administer a short quiz to 400 students. I was less worried about the time since the spring class is T/TH (75 minutes rather than 50). I was more worried about all the paper--including making sure we got the quiz itself back from the students (though, as it happens, I've decided to post them on Bb for the students to review). Initially, I thought I might embed the quizzes in the Learning Management System and have them take them in class. My classroom has the technological capacity for that. Soon enough, though, we realized that a. there were bound to be tech issues ("I can't get online"; "Bb ate my quiz"); and b. we had no way of preventing someone from taking the quiz from another location. To prevent that would have required something like having the students register their presence with an i>clicker and then us having to compare that to the names of 400 quizzes. Not reasonable.
Finally, we settled on a fairly basic plan: we would hand out the scantrons and then use a doc cam to project the questions and a PPT slide for images. This seemed like a good idea in theory; in practice, it didn't work very well, for reasons entirely out of our control. First, the doc cam could only focus and magnify 2-3 questions at a time and I hadn't worked out the timing for individual questions. I had to play it by ear (which seemed fine--I gave them more time than they needed). Second, separating the images from the question meant that we had an image on one screen and the question on another. The architecture of my classroom made it nearly impossible for large swaths of students to see either the image or the question. I ended up reading the answer options aloud twice, which was a good bandaid. Still, after the first quiz, it was clear that we needed to find another way to administer the questions. Fortunately, one of my TAs (who was with my in the fall as well) came up with the perfect solution: a PPT slide show with timed slides. This way, we could project everything on both screens at all times. The images and the questions about them were on the same slide; and I could add timings so that easy questions got less time while harder questions allowed for more time. This ended up shortening the 12-question quiz to about 8.5 minutes. This second quiz went very smoothly. I seemed to get the timing right for individual questions and nobody had any trouble seeing the questions.
With the first quiz we had a somewhat unusual issue--four quizzes went missing. I have several procedures in place to ensure that, once they are in our hands, we don't lose them without knowing it. This meant that they must have gone missing during the collection process--a reminder that, even with this, we need to take special care. An enormous challenge of a large enrollment class is that you can't know if an individual student is lying to you; or really was the victim of some freak circumstance. In this particular case, evidence is mounting that someone funny happened. When we ran the scantrons for the second quiz, those four quizzes magically re-appeared. We know they had to have been added by the students, either the same students or some other student. It's difficult to know what to do since we can't prove who added the quizzes (perhaps they fell into a student's bag and, rather than giving them to us, he thought he would slip them into the stack?)
More than likely, these four students are trying to pull a fast one, but I do feel like I have to a. let them know that I know; but still let them retake the quiz. In the future, a TA will leave the room after we collect the quizzes and count them. And I will be clear that we will only run the quizzes we have in our possession after they are collected. This episode is a reminder to me of just how much effort, care, and time it takes to incorporate something as simple as a weekly, 10 minute quiz into a large enrollment class. At the same time, the weekly has proven to be very effective at getting the students--totally unaware of what they are doing--to engage in a flipped class. In addition, it gives the students substantive and relevant feedback on their learning strategies long before the first midterm.
In the fall, it was clear that I needed to implement weekly quizzes to motivate the students to stay caught up and, therefore, able to be active and engaged learners. With weekly quizzes, they really cannot get more than a week behind without significant consequences for their course grade. It was interesting that, in one of the fall student surveys, a common suggestion for improving the course was to incorporate weekly quizzes. As well, in information conversations in my office with various students from the fall course, they were clear that they needed the outside motivation of a graded quiz to "force" them to do assigned work. They knew that it was better for their learning to do the assigned work incrementally rather than try to cram for the exam, but seemed incapable of managing their time in a way that allowed them to avoid cramming.
The real obstacle to incorporating graded, weekly quizzes into the class was logistics. It is a big deal (and takes about 5-7 minutes) to hand out 400 scantron sheets and then collect them. It uses a lot of paper, especially if we also hand out copies of the quiz. I talked to a lot of people about possible ways to administer a short quiz to 400 students. I was less worried about the time since the spring class is T/TH (75 minutes rather than 50). I was more worried about all the paper--including making sure we got the quiz itself back from the students (though, as it happens, I've decided to post them on Bb for the students to review). Initially, I thought I might embed the quizzes in the Learning Management System and have them take them in class. My classroom has the technological capacity for that. Soon enough, though, we realized that a. there were bound to be tech issues ("I can't get online"; "Bb ate my quiz"); and b. we had no way of preventing someone from taking the quiz from another location. To prevent that would have required something like having the students register their presence with an i>clicker and then us having to compare that to the names of 400 quizzes. Not reasonable.
Finally, we settled on a fairly basic plan: we would hand out the scantrons and then use a doc cam to project the questions and a PPT slide for images. This seemed like a good idea in theory; in practice, it didn't work very well, for reasons entirely out of our control. First, the doc cam could only focus and magnify 2-3 questions at a time and I hadn't worked out the timing for individual questions. I had to play it by ear (which seemed fine--I gave them more time than they needed). Second, separating the images from the question meant that we had an image on one screen and the question on another. The architecture of my classroom made it nearly impossible for large swaths of students to see either the image or the question. I ended up reading the answer options aloud twice, which was a good bandaid. Still, after the first quiz, it was clear that we needed to find another way to administer the questions. Fortunately, one of my TAs (who was with my in the fall as well) came up with the perfect solution: a PPT slide show with timed slides. This way, we could project everything on both screens at all times. The images and the questions about them were on the same slide; and I could add timings so that easy questions got less time while harder questions allowed for more time. This ended up shortening the 12-question quiz to about 8.5 minutes. This second quiz went very smoothly. I seemed to get the timing right for individual questions and nobody had any trouble seeing the questions.
With the first quiz we had a somewhat unusual issue--four quizzes went missing. I have several procedures in place to ensure that, once they are in our hands, we don't lose them without knowing it. This meant that they must have gone missing during the collection process--a reminder that, even with this, we need to take special care. An enormous challenge of a large enrollment class is that you can't know if an individual student is lying to you; or really was the victim of some freak circumstance. In this particular case, evidence is mounting that someone funny happened. When we ran the scantrons for the second quiz, those four quizzes magically re-appeared. We know they had to have been added by the students, either the same students or some other student. It's difficult to know what to do since we can't prove who added the quizzes (perhaps they fell into a student's bag and, rather than giving them to us, he thought he would slip them into the stack?)
More than likely, these four students are trying to pull a fast one, but I do feel like I have to a. let them know that I know; but still let them retake the quiz. In the future, a TA will leave the room after we collect the quizzes and count them. And I will be clear that we will only run the quizzes we have in our possession after they are collected. This episode is a reminder to me of just how much effort, care, and time it takes to incorporate something as simple as a weekly, 10 minute quiz into a large enrollment class. At the same time, the weekly has proven to be very effective at getting the students--totally unaware of what they are doing--to engage in a flipped class. In addition, it gives the students substantive and relevant feedback on their learning strategies long before the first midterm.
Subscribe to:
Posts (Atom)