Advice on Quiz Assessments: 5 Things to Do

December 4, 2009

What this post is about: This post provides tips on developing quiz assessments for instructional media.

What this post is not about: This is not an in-depth discussion on the various types and methods of assessment. Rather, it provides advice on what you need to consider when developing quizzes to test for knowledge retention and understanding.

Here is a summary: Quizzes are not the only method of assessment available. However, they are the most popular, as they are easy to develop and manage. Unfortunately, quizzes are often seen as an add-on to learning content, rather than being central to its purpose. In this post, I discuss this in more detail, and provide 5 useful tips that can help you develop better quizzes that really assess learner knowledge retention and understanding.

Here begins the post

Quizzes are probably the most popular form of assessment for online/digital learning materials. They are easy enough to write, develop and deploy. From a learner administration point of view, they are also easy to track and measure. They seldom require intervention, as they run from an LMS, assessment engine or other automated system.  Once learners have taken a quiz, the (reliable) results are provided, letting administrators know which learners have done well, and which have not.

However, many quizzes I have seen are poorly implemented. They are an afterthought, created as a piece that slots in toward the end of the project: an administrative effort, required by the Subject Matter Expert, Curriculum Developer, Client or the Project Manager.

This is the wrong approach.  Quizzes – or any assessment, for that matter – should be much more central to your learning product. It is the assessment that indicates how much knowledge learners have retained and how well they have understood it. The questions you pose also offer the opportunity to test how well learners will be able to apply the knowledge they have learned.

5 Things to do

Test to your objectives (verbs, purpose, etc.)

It is crucial that your quiz questions relate back to the course objectives you have set. As obvious as this seems, it is often not implemented. Instructional Designers will much more often work off their scripts or content samples, tying quiz questions to specific content in the learning product. The logic is that “This is what they have learned, so it is safe to test on it”. While this point makes some sense, it loses sight of the purpose of both the quiz and the learning content that has been developed.

Learning content is fundamentally about improving people’s personal or professional ability. The specific improvement to be achieved is broken down into constituent objectives. For most subjects and learning content, it would be impossible to test whether the learner has improved their skill by asking them to display it. Therefore, it is the objectives that are tested, with the fair assumption that if the learner can achieve those actions described by the objectives, they should have improved their skill.

This means considering:

  1. The context of each objective (are there specific conditions under which the action described in an objective should be performed?)
  2. The verb of the objective – this is really important. I’ll elaborate more on verb objectives in another post, but for now, take it as read that to “describe”, “identify”  or “demonstrate” are completely different things. While this is obvious when written like this, you might be surprised to learn that they often become interchangeable when being developed in learning content
  3. The subjects and objects of the objectives. What should the learner be able to do, will they do this with an object, a tool, or a piece of software?

Answering these questions about your objectives will go some distance to helping you develop really good quiz assessments.

Decide on quiz items when developing an IDD

An important way to decide on the design for your quizzes is to outline them as part of your initial Instructional Design Document. I know this is standard in most companies. I also know it’s standard to give assessment little more than an afterthought or consider it something of an onerous task, a push to get the IDD out and signed off.  However, carefully considering your quiz assessments, based on the course objectives at this point will actually provide a couple of benefits:

  1. They will give you a better idea of how best to assess learner’s retention of knowledge and/or understanding. This in turn can help to suggest ways in which the content itself should be developed to improve retention and understanding
  2. If you consider your quiz questions ‘blind’ (i.e. without looking at the content itself), then look back on how you will develop your content, you can check that the content does address all the objectives. If you have questions that ask something not covered by the content – you need to include it

Ask questions as closely relevant to the job/task/skill you are training on

Again, this may seem obvious, but it is advice that is often overlooked, rather than ignored. In an attempt to question objectives, instructional designers will often overlook the real purpose of the objective. Should someone know the definition of a term, or what that definition actually implies when using a tool, software or piece of information in their job? Usually, the latter is the case.

Here is an example: Pressing the Shift key will display letters in uppercase. So, what is the Shift key for? Well, two possible answers are immediately apparent:

  • displaying letters in upper case
  • capitalising words (both at the start of a sentence, or mid sentence)

The first option here is very literal, but the second provides a useful context in which the information is used. From my viewpoint, this makes the second option more ‘correct’ than the first.

Of course, the treatment you provide will depend on the objectives, which is (yet) another reason why your objectives should be carefully drawn out. However, in most cases, the application of information is more important to test than its abstract truth. This is especially true when developing and assessing content for specific work tasks.

Consider the assessment as an item in itself – not just a string of questions!

Quizzes in particular are often seen as a string of questions. Instructional Designers bash them out and send them off for review. However, a much more interesting way to develop your quiz assessments is to try and consider the whole piece. Consider the following:

  • In what you are testing, is there a beginning, middle and end that relates to your objectives? Could you use this to provide a ‘development’ for your quiz?
  • Does the information you provide in the learning content tie together in an interesting way? Can you ask one question to assess fundamentals, then go on to ask other questions that assess the ability to apply the information?
  • Is it possible to ask questions in a structured fashion that relates to how learners will use information, tools or software in their jobs?
  • Is there a typical scenario you could use to develop questions from? Ask the SME!

Add some colour!

If I had a penny for every time I’ve seen a quiz assessment made up entirely of multiple choice questions… Most VLEs and assessment engines now offer an excellent range of question types for quizzes, and you should try to exploit this. Don’t allow learners to passively click their way from one end of a quiz to the other- even assessments are learning opportunities. When considering question types, it is also important to ensure the type of question you are asking relates to the type of thing you are assessing. Here are some examples:

  • Fill in the blanks, or a series of them can be a useful way to test steps in a process
  • Matching questions are a helpful way of checking relationships between concepts, or the where one might find a menu command, information or perform an action
  • Graphic based click answer questions are great for testing people’s understanding of situations, scenarios or the actions they need to take in software applications to perform tasks
  • True or false – often considered the sledgehammer of quizzes – can be used to test understanding of conditions, situations or differences in functions or features. As long as your question is fair, you can ask learners to identify subtle implications of information outlined in the learning content.
  • Just about every question type can be used to ask learners about definitions

If you’d like to discuss this, please feel free to leave a comment below. Alternatively, you can email me about this and other opinions at brendan dot strong at gmail dot com.

Tune in next time for 5 things you should avoid when developing quiz assessments.

Advertisements

2 Responses to “Advice on Quiz Assessments: 5 Things to Do”

  1. Great information. Thanks for sharing the link!

  2. Brendan Strong said

    Thanks for reading and taking the time to comment! I hope it’s helpful for you.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: