The process of designing and building Online Rome was filled with obstacles. A big one--perhaps the biggest and most challenging one--was figuring out a design that would allow the course to scale well enough that my university would be willing to create and fund future instructors. I discovered only in Fall 2014, more than a year into the project, that there was no plan in place for funding the instruction of Online Rome after Fall 2014 (when, as part of the terms of the grant, I hired and paid all instructors and graders). I gather that there was the expectation that departments would take on the instructional costs. Yet, with no clear incentives to do so, that was unlikely to happen.
A savvy department leadership might understand that there were long-term (and even some short-term) benefits to working their soft money budget to fund the instruction of an online course but, given the general lack of experience with or knowledge about online teaching and learning in academic units, this was not likely to happen. My own department certainly had no interest in supporting the ongoing instruction of Online Rome without additional soft money from the College of Liberal Arts. Thankfully, with the help of our fabulous Liberal Arts ITS department head and my project manager, I was able to get my college to foot the bill for the course instruction in Spring 2015. I then spent this Spring advocating tirelessly for my college to find a longer term solution to the funding of the course's instruction--which, thankfully, they did in the form of giving my department instructional funds to hire a lecturer on the condition that at least three sections of Online Rome were offered each semester.
One of the ways that I was able to "sell" my college on continuing to support the instruction of Online Rome was by designing a course that is, in fact, less expensive to teach than the classroom based version. For me, the challenge was to navigate between the Scylla of deans who wanted efficiency; and the Charybdis of quality. If I could not find a way to design a course that also produced high quality learning, I had no interest in extending the life of the course. The design of the Fall 2014 version was labor intensive and totally unsustainable. It also did not really produce the kind of quality critical thinking that I wanted to see.
My main task during this past spring semester, on the development side, was finding a way to reduce the labor demands on the instructor; and ensure that the instructor's time was being spent as much as possible on high impact activities. In brief, it was about finding a balance between questions that could be automatically graded but still required students to do more than regurgitate content; and instructor-graded activities--in our case, short essays. In the Fall 2014 version, I tried to use short answer question inside the modules. I then wanted the instructional team to read, grade and respond to these short answer questions. This might have worked for a class of 30 but was unmanageable in a class of 300+, also because Canvas is not set up very well for this. Speed Grader only works for a graded quiz but, for other reasons, we needed to categorize the modules as practice quizzes. It rapidly became clear that the module grading had to be automated if the course was going to scale at all.
In order for the module grading to be automated, the short answer questions had to either be removed or be ungraded. I did not want to remove them. They served several purposes, the most important of which was to require active thinking/writing at frequent intervals. They also allowed me to ask certain kinds of questions that could not be reduced to a multiple choice/matching/ranking, etc. question. My solution was to retain the short answer questions and provide extensive feedback on each one. In the course orientation module, I spent a good amount of time introducing the students to the concept of self-regulated learning; and explained how the concept was used in the design of the course. Of course, students could skip over the short answer questions or write nonsense. They could paraphrase the feedback. I did test some of the content in the short answer questions later in the module. I would love to be able to work with a programmer to design an LTI that makes it easier to spot the short answer questions in the modules.
The course modules were worth 35% of the final course grade. Each module was worth 3 points. 2 points were awarded for getting 90% or higher on the questions in the module. Students could redo the module questions as many times as they wanted, until they earned that 90%. The third point was awarded for earning a 90% or higher on a graded, 15 question quiz at the end of the module. A quiz grade of 70-89% earned a student 1/2 point. Anything under 70% earned no points. These graded quizzes were intended primarily as a way for the students to get feedback on their learning before the midterm exam. They were graded to encourage them to take them seriously--which worked. They were not weighted very heavily because they were intended to be formative; and because they were not proctored. We did have a database of questions and no two students would get the same quiz; and the questions were not easy to Google. At the same time, to remove the temptation to cheat, we de-emphasized the graded quiz. Students also had an optional practice quiz that they could take; and many of the quiz questions came directly from the module and practice quiz. The emphasis and incentives were entirely on effort, persistence, self-correction, and time on task.
45% of the course grade was 3 proctored midterms, administered on campus in the evening. The midterms consisted of multiple choice questions, of the same sort that they saw in the modules, practice quizzes, and graded quizzes; and then short answer questions. The instructor provided a detail study guide for the short answer questions and took the exam questions from the study guide. Again, the emphasis was on effort and focused work. In reality, the study guide was so comprehensive that nothing was being given away--it was simply a way to soothe anxieties and help the students focus their study.
In place of the time-consuming and low-return short answer questions, we added 500-750 word essays to the course. The essays were worth 10% of the total grade. Each student had to submit 5 essays over the course of the semester. All students completed a final, summative essay with the last module. For modules 2-9, we divided the students into sections and, for each module, half of the sections had an essay. This meant that, for each module, the instructor had 50 essays to grade rather than 100. Five essays was plenty of writing and practice with critical thinking. The instructor spent a lot of time crafting and refining the essay prompts. On the whole, the students did an outstanding job with the essays. The instructor reported that he enjoyed reading them and, frequently, found the students thinking critically and deeply about Roman history and culture. This change in the course design was a resounding success.
The final component of the course grade, a new addition in the spring, was a movie module that was worth 10%. We chose Stanley Kubrick's Spartacus, partly because it's a good way to introduce students to the ways in which Ancient Rome has been used to talk about contemporary political issues. An area ripe for future development is the addition of more movie modules (Gladiator, Pompeii, etc.).
We introduced the module with an outline of the module as well as a reminder to review their earlier work on the historical figure of Spartacus and his revolt. There were two in-module quizzes. The first focused on slavery in Ancient Rome, so that the students would understand that aspect of the film and also understand the ways that Roman slave practices differed from what they might have learned about slavery in the US.
The second in-module quiz focused on Kubrick's film and the political background of the McCarthy Hearings--a topic that was unfamiliar to nearly all the students.
The module ended with an essay prompt. All students were required to write this essay (so, in fact, students wrote 6 essays in total during the course).
In the end, I am happy with the end result. Students engage in a substantial amount of critical thinking and writing, especially in the essays. Effort and persistence are highly rewarded, two things that we know are crucial to learning. The scores on the proctored midterms were very high over the semester, indicating that the work that the students were doing on the modules was leading to real learning that they were then able to demonstrate on exams.
No comments:
Post a Comment