About this Course
When you register or take the course, please use the latest version (or the one before that) of Microsoft Edge, Firefox, Safari, Chrome, or a modern mobile browser such as Mobile Safari. For more information, see https://ruzuku.helpscoutdocs.com/article/487-web-browsers-and-operating-systems.
Do Your Multiple-Choice Questions Drive Deeper Learning?
Consider this multiple-choice question:
To drive deeper learning, multiple-choice questions must __________________: (Select the best answer.)
a) ask participants to recall content.
b) ask participants to determine which details apply.
c) measure whether participants have met the learning objectives.
d) measure whether participants have read all the content.
If you're not sure of the answer, you’re not alone. Research shows that multiple-choice questions are difficult to write well. Difficult, that is, until you know and have practiced the evidence-based tactics that make them clear and valid for their intended purpose.
Too many multiple-choice questions are unclear, frustrating people who try to answer them and making results less valid. Too many multiple-choice questions measure only recall of course content, which is not usually what we need to measure. We must write questions so they effectively measure achievement of the learning objectives. (Therefore, the answer to the multiple-choice question above is c.)
Writing well-written, valid multiple-choice questions that measure achievement of the learning objectives is a critical skill for people who design instruction. And yet research shows that most multiple-choice questions are poorly written. This is a big problem because poorly written learning objectives provide bad information (GIGO or garbage in garbage out) and can lead to legal problems.
Who is this course for?
We designed the Write Learning Assessments for Deeper Learning course specifically for workplace-learning professionals and others who need to write valid job-related assessments. This includes instructional designers, trainers, instructors, content experts, technical writers, and others. In Patti Shank, PhD’s Write Learning Assessments course, we use actionable research tactics for writing valid multiple-choice questions that measure learning objectives. The course is best suited for people who have some experience designing adult instruction and writing multiple-choice questions.
If writing multiple-choice assessments is something you need to do and you don't know the critical evidence-based tactics, you need this course! Take this (anonymous) multiple-choice question quiz to gauge whether you know the basics.
What does this course include?
The Write Learning Assessments for Deeper Learning course is equivalent to a 3-day classroom workshop, but we spread it out over 3 weeks to give you more time to process and practice. And it’s online, so no travel or travel expenses, plus you get better access to Dr. Shank and your classmates!
The course learning objectives for this course are:
- Make sure learning objectives are written correctly.
- Analyze which actions improve assessment validity and reliability.
- Evaluate which objectives are appropriately assessed using multiple-choice items and which objectives are appropriately assessed using performance assessments.
- Analyze multiple-choice questions to identify flaws that should be fixed.
- Write appropriate multiple-choice items that measure given objectives.
The course includes:
- Substantial interaction, including:
- 4 live online sessions with Patti and your classmates on important insights, problems, commonly asked questions, and practice issues
- Ongoing asynchronous discussions to ask and discuss questions, and share insights
- Feedback from Patti and your classmates
- Access to live-session recordings, in case you can’t attend live or you want to review
- Patti’s assessment-writing manual (only available through this course)
- A master job aid to help you write more effective multiple-choice questions
- Practice evaluating and writing learning objectives so you can more easily write aligned multiple-choice questions
- Practice writing multiple-choice questions that assess achievement of learning objectives
- Three quizzes (with effective multiple-choice questions, of course!) to check your understanding and help you process and remember critical insights
While this course concentrates on writing multiple-choice test questions, it does include high-level information on the overall assessment process and on performance assessments, as well as the legal aspects of testing. We include this information because you need it to understand where multiple-choice questions fit into the bigger picture, and why writing good multiple-choice questions is so very important.
Praise for Write Learning Assessments
Multiple-choice questions are everywhere in learning, but they can be fiendishly difficult to write well. Nobody is doing a better job than Patti of distilling out research-based principles for writing well, and turning them into practical, actionable guidelines. Highly recommended.
- Julie Dirksen, author, Design for How People Learn
This training gave me insight into the many factors involved in creating effective multiple-choice questions. It has prepared me to better advise instructors on whether multiple-choice questions are the best tool, and how they can be crafted to generate reliable, valid data. Course participant contributions created a robust conversation and led to many insights.
- Anna Landes Benz, Program and Faculty Developer, private university, MN, USA
I joined Patti’s course this July and loved it! Her content matches her facilitation; both excellent. I would totally recommend it.
- Taruna Goel, Learning & Performance Specialist, Vancouver, BC, Canada
The weekly live sessions opened my mind and upped my question-writing game. Patti knows her topic and added the wow factor that I was looking for by offering real-time feedback to participants. Comments from other participants helped me see different perspectives. I strongly recommend this course for anybody who must create multiple-choice question learning assessments.
- Peter Sage, Trainer, iLAB, South Africa
What's different about THIS course?
Here’s what’s different about this course: Patti. Real practice. Feedback. Support.
Patti Shank, PhD is an acknowledged leader in the use of evidence-based tactics for deeper learning in the workplace. She reads the research, interprets it for use in workplace learning, and helps people leverage the most critical insights in an easy-to-apply and practical way. And she has 12+ years of experience teaching in the virtual classroom.
If you need to write understandable and valid multiple-choice questions, this is the course you need. You’ll learn when to use multiple-choice questions, when other assessment types are more appropriate, and how to design understandable and valid multiple-choice questions.
Supporting Patti’s courses are:
- Karen Hyder, with her 20 years of virtual-classroom training, coaching and production work
- Chris Benz, with his 30 years of designing and producing successful live online events, training materials, and technical-communication products
What’s the big deal with “validity?”
The reason the word “validity” keeps coming up is because it is one of the most important issues when writing multiple-choice questions. Validity is a critical outcome of the assessment process. Valid questions require clear writing and clear ties to learning objectives. When they are valid, questions can measure deeper levels of learning and create deeper processing, which improves ability to apply.
Assessments that are less valid invite frustration, irrelevant results, and legal battles. It’s far better to do a good job with this key skill than risk these bad outcomes.
Why should I register as early as possible?
Register early! As soon as you register, you’ll get early access to course materials, which gives you a head start on reading and processing important concepts. The first week of the course is the “official” time allotted for participants to read and ask questions, but past participants said they wanted more time.
To encourage you even more to register early, we are offering an early-registration discount of $100. Use the code EARLYBIRD for the $100 discount before October 15. If you can’t register before October 15, you will still receive early access as soon as you register.
One more reason to register early: For maximum interaction, we purposely limit enrollment, so if you wait too long, you might miss out.
What are the dates and times?
The next course is November 4-22, 2019, with an introductory live online session on November 4 and then on the three following Fridays, November 8, 15, and 22. (For your reference, Thanksgiving in the US is on November 28.) All live sessions start at 11:00 AM US Eastern Time / 8:00 AM US Pacific Time and run for up to 90 minutes each depending on participant needs.
What are the activities for each week?
Week 1: Foundations of assessments and multiple-choice questions
Patti's assessment-writing manual
Two live sessions
Week 2: Practice writing multiple choice questions
Patti's multiple-choice writing job aids
Practice and feedback
One live session
Week 3: Foundations of assessments and multiple-choice questions
Practice and feedback
One live session
Where and when are the live sessions?
We hold an introductory live session the first day of the course to meet, answer questions, and get a tour of the course. All other live sessions are held on the three consecutive Fridays of the course. The URL for the live sessions are posted in the course materials.
Live session dates and times (each session runs for up to 90 minutes each depending on participant needs):
|Monday, November 4||11:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time|
|Friday, November 8||11:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time|
|Friday, November 15||11:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time|
|Friday, November 22||11:00 AM-12:30 PM US Eastern Time / 8:00-9:30 AM US Pacific Time|
What is the time commitment?
You should set aside 7 hours each week for the reading, live sessions, and self-paced course activities (approximately 25 hours total). (Enroll early to spread essential reading time.) To gain the maximum benefit, take the course when you can dedicate this much time. Except for the live sessions, you can participate in the course activities at whatever times work best for you.
What's my investment?
Before October 15, 2019: US$995 using the coupon EARLY BIRD
After October 15, 2019: US$1,095
Register today! Enrollment is limited for maximum interaction, and we expect the course to fill up quickly. We currently accept only credit cards and PayPal for payment. To see if other payment arrangements are possible, please contact us at firstname.lastname@example.org.
Can I share the materials and live/recorded sessions with my team/friends/pets?
Each registration allows only the person registering to access the course, materials, and activities. But it’s a smart idea to train your entire team and give them access to everything the course has to offer! See the last question below for options and discounts for training your team.
How do I convince my workplace to pay for this course?
Assessment writing is a critical skill but is rarely taught to learning practitioners. As research shows, many workplace-learning assessments are poorly written and invalid for the desired use.
Poorly written and invalid assessments do not provide accurate information about the effectiveness of the instruction. Invalid assessments can create morale and legal problems. The bottom line is that everyone who builds assessments must be able to design valid assessments!
Can I get a refund?
To offer personal attention, this course has limited enrollment. Once you register, we are holding your spot. As a result, we are unable to offer refunds. However, you can substitute another person in your organization up to three days before the course starts.
Can you deliver a private class just for my team?
We can and we do! Please contact us at email@example.com to discuss. We offer discounts for three or more people from the same organization (paid at the same time) attending the same public class or for 12 or more people from the same organization attending a private class.
Register for the course or our mailing list
Ready to register? Don’t delay; register today! Enrollment is limited for maximum interaction and we expect the course to fill up quickly. We currently accept credit cards and PayPal. To see if another payment arrangement is possible, please contact us at firstname.lastname@example.org. To attend, we require that payments be paid in full before the course start date.
Not quite ready to register? Sign up for the Deeper Learning At Work mailing list to stay informed about:
Future offerings of this course,
Other upcoming Deeper Learning At Work (DL@W) courses, webinars, and conference presentations by Patti and other DL@W thought leaders, and
Ongoing information about DL@W principles, strategies, tactics, and research
Have questions? Please email us: email@example.com.
When you register or take the course, please use the latest version (or the one before that) of Microsoft Edge, Firefox, Safari, Chrome, or a modern mobile browser such as Mobile Safari. For more information, see https://ruzuku.helpscoutdocs.com/article/487-web-browsers-and-operating-systems. If you have any problems enrolling, contact firstname.lastname@example.org and email@example.com.