Reflection on Chapter 14: How Do I Know If My Courses Are Designed Well?
"Just because the course is live doesn't mean the work is done."
I feel that in my bones. Throughout my days as a primary, secondary, and undergraduate school student, I had numerous teachers and professors who had not updated their course materials in years (sadly even down to some geography lessons where the worksheets with maps were quite dated). I swore I'd never be that teacher and I was constantly refining individual lessons and overall course maps throughout my fourteen years teaching middle and high school students.
Throughout this chapter, Dr. Hobson discusses ways we can make sure that a course and its content resonate with the learners. While he does mention having a pilot program is ideal, he also acknowledges that it isn't always possible to do based on the timeline and nature of a project. I haven't had the opportunity to utilize a pilot program for a course before going live, either as an educator or an instructional designer, but hopefully that day will come sooner or later.
Hobson then goes on to share his favorite method of research, an explanatory sequential mixed methods approach, to improve his courses. I accidentally used a similar method in one of my graduate school assignments recently and reading this chapter made me smile. It's always nice to have an idea validated by an expert in the field, even if they don't realize they've done so. With this model, you start by collecting quantitative data from surveys and then use that data to develop questions for follow-up interviews where you can collect qualitative data.
Some other suggestions he had for gathering data for course improvement included:
Include a "Tell Us Anything" section in the survey so learners can provide you with information you may not have thought of otherwise.
Ask learners how long it took them to complete the course or module.
Gather real-time data by placing surveys at the end of each module to ensure you get more specific information.
Hobson ends this chapter by posing the following reflection question:
Think of a time you made an assumption about your students' understanding of a subject and then realized it was inaccurate. How did you adjust your content based on this new information?
When I taught social studies courses to upperclassmen in high school, I made an assumption that the students knew how to do a self-check and a peer review of their written work. We had been working on document-based questions (DBQs) in class and students had the first drafts of their papers written and were reading through and providing feedback for one another's work.
As I was walking around the room it became apparent, they had no idea what they were doing. I had the class put their pencils down and started asking them questions to try and determine what exactly was going on. Was it apathy? Did they know what to do but didn't care to do it? Or were they confused about what they should be doing during the peer feedback stage? Naturally, it was both.
For the rest of that day in all of my classes, we did a more structured peer feedback session. After students traded papers, we read the first paragraph together and one by one, I gave a list of things for the reviewer to look at and write feedback for on their papers. After that, we did the three body paragraphs and the conclusion in the same manner. It wasn't an ideal scenario, but it was the best I could do at the moment given the stage of the process we were already in.
In order to improve this process in future iterations, I developed a Self-Check and Peer Review document. Students would write each paragraph of their rough draft on one of these sheets. After finishing their draft, they would answer the five self-check questions on the left side of the page. After that, they would hand trade papers with a peer who would read the draft and answer the four peer review questions on the bottom half of the page.
Providing students with that structure helped students get in the habit of providing better feedback to each other while also providing accountability for those who knew what to do, but did not care to do so.
References
Hobson, L. (2021). What I Wish I Knew Before Becoming an Instructional Designer. Independently published.