Principles of Assessment – Part 1 (Reliability)
There are four Principles of Assessment – Reliability, Fairness, Flexibility and Validity. Here we discuss Reliability. This will be followed by additional Blogs which will discuss the remaining Principles of Assessment.
Principle of Reliability
Reliability refers to the consistency of the interpretation of evidence and the consistency of
38 comments
Leila Regina Kuzmiuk
Reliability is essential to the outcome of competence. Assessment tools that support and guide the assessor to standardise what is required will reduce bias or influence judgment. This leads to consistency and validity of assessment and competence outcomes.
Anne T
Consistency is important so that all the learners are given equal opportunity and assessed according to the same criteria.
Melanie Oliver
I also agree it is important that assessors come to the same or similar conclusion about a candidate’s competency. Otherwise it is unfair.
Keryn
Really interesting article. I agree it is vitally important that all assessors are able to come to the same conclusion about a candidate’s competency.
Rachel Popple
Great article, Sandy. Sets out quite clearly the importance of creating reliable assessment tools and also assessor guides to accompany the tool. I thought the differences in creating the assessment tools/assessor guides depending on the level of the qualification particularly helpful. It can clearly be seen from your points above that the Principle of Reliability is so very important, especially if you are creating an assessment tool that will be used by multiple assessors. By setting out appropriate answers/responses in the assessor guide, this should minimize any interpretation required by assessors and result in the same outcome regardless of who the assessor is.
Jen Good
Thanks Sandy, a great explanation! It just makes sense, hoping it will make sense when I put my pen to the task! I know ‘first time around’ I’ve struggled and got a bit buried in this (as an over thinker!), but reading (& re-reading) reinforces just how ‘airtight’ the 3 principles of Reliability fit together. They should flow to the exact same outcome no matter other ‘influences’ that may be involved. My thoughts go directly to the type of learner the assessor may be and given we are all different in our learning styles, thus are we in our thought processes and approach. As an over thinker I get it! Now to just get over my over thinking!
Scott M
Certainly highlights the lack of consistency in making assessments that can creep in if clear guidance is not given to the assessor
If the instrument is well designed then this reliability will be possible between assessors in different RTO’s, different geographical locations, different teaching backgrounds and with varying target groups. And the students gain fairness in how they are assessed regardless of variation in background or personality.
Sarah Walker
I completely agree with this. It is extremely important to have guidance for the assessor to ensure fairness across the board. Where there are multiple assessors in an organisation, it removes personal judgments and teaching styles.
Matt
Thank you for the article. A very valuable reminder of the importance of reliability in assessment instruments and for assessor guides, to ensure all assessors are making the same judgements on what is deemed ‘competent’.
amber
The Principle of Reliability ensures that all students have fair and equitable chances of succeeding in their study, without having to worry about differences in teaching styles or approaches, or a teacher’s personality or expectations. The rules are determined and that’s helpful for both teachers and students.
Emily Dever
I wish I had read this article prior to submitting my last assignment! (Some points I could have used!)
The Principle of Reliability provides a framework for integrity that supports both the assessor and candidate and further down the line future employers and service recipients. I think it is enormously important that there is reliability in assessment to ensure a uniform baseline for performance, knowledge, skills and wider trust in qualifications.
Philip Bancroft
As the old adage says “consistency is key” and in the field of VET training and assessment, this is absolutely vital. Consistency provides reliability which in turn provides businesses, enterprises, trainers and trainees alike with the confidence that the training and assessment process can be trusted.
Tahlia Oxenbridge
This article really highlights the importance in creating instruments for both students and assessors that are detailed and informative. They need to be created with careful and close considerations of all requirements to ensure the reliability of the instruments.
Shari Dearman
Understanding the Principal of Reliability is extremely important. It gives measure to everyone in understanding what is required and assurances that we should be able to draw on the same outcome.
Claire Holloway
Thanks for the article Sandy. To summarise, designing assessment requires that questions and answers result in any trainer being able to rely on the instrument to assess the candidate similarly to another trainer including details such as what level of grammar, technical detail etc should be achieved.
John Logan
Oh wow, maybe everybody gets it but me. I found this piece heavy on jargon and was hoping that for an introductory piece on this task that it could have been put in simpler terms. I read it several times, & it started to clear the fog but still felt there could have been a way to explain it in a more straight forward manner.
Gabrielle Jones
Thanks for this article. The clearer the instructions for the candidate and assessor, the easier it will be for everyone. There is fairness in reliability for learners. When designing assessment instruments I need to keep in mind not only the Principles of Assessment but the Standards for RTOs. Having the big picture in mind will help me to express the instructions in a meaningful way. For example, Standard 1 “The RTO’s training and assessment strategies and practices are responsive to industry and learner needs and meet the requirements of training packages and VET accredited courses”. The key idea – responsiveness – should be reflected in learner and assessor guides and the Assessment Plan itself.
The ideas underpinning each Principle and Standard could be woven into the words we choose to explain our assessment instruments and their guide book tools. In this way, reliability becomes an action and ideal not only accountable to the requirements of the training packages and courses, but to a commitment of responsiveness necessary to meet Standard 1. Reliability to me means mapping of a sort to ensure Standards shine through every document we create to collect evidence and assess competency, and that these instruments can reliably be repeated to produce the same assessment results within the same assessment conditions.
Darren Mumford
This article provided valuable information, helping outline important guidelines that help ensure assessment is consistent and fair. Providing clear boundaries that help the assessor and learner understand the requirements may seem obvious but this can easily be forgotten in the process. I appreciate the recommendations in this article and will take them on board as I use and prepare assessment tools.
Zoe
Thanks for the great article Sandy. Reliability is so important in VET assessment, so it’s good to go over the key points regularly.
Kym McDonald
The principle of reliability is one of upmost importance in keeping with the integrity and consistency of the unit of competency. A great deal of information these days is ‘open to interpretation’, so having consistent principals like this embedded in the framework is paramount.
Elizabeth Cooper
In the sector that I work, the trainees that I work with all have there assessments marked by different assessors. So for me, The Principle of Reliability is extremely important, so that it lays a clear benchmark for what the candidate needs to produce – no matter who is assessing their work – making the assessment process fair for all
Colin Meyer
I must admit that with my ‘shiny ball syndrome’ I went off on my own to find out how many and what type of Assessment tools existed and found 9 different ones including pre-assessments, they have yet more insights for me to learn about, however the two that Sandy has written about here are well worded and nicely explained. I agree that as senior VET folk we must not only know but apply Principles of Assessment as we develop tools for each of our respective work places. Reliability has earnt its place in the workspace.
The examples that have been given here certainly provided me with a greater understanding of the concept of consistency and therefor useability.
Thank you, Sandy,
p.s this comes from someone one hasn’t studies for some time now.
Claire Latham
The Principle of Reliability is one of the key features of VET that makes it so valuable and equitable. Given that different assessors may have varying levels of experience and different preferences about “style”, the requirement that there be clear benchmarks and agreement about acceptable answers should ensure that assessment is of sufficiently high standard across the board.
Claire Latham
The Principle of Reliability is one of the key features of VET that makes it so valuable and equitable. Given that different assessors may have varying levels of experience and different preferences about “style”, the requirement that there be clear benchmarks and agreement about acceptable answers should ensure that assessment is of sufficiently high standard across the board.
Lisa
Thanks for the article reiterating the importance of a reliable instrument for assessor’s to use to guide them to assessing competency in a consistent way.
Lisa
Thanks for the article reiterating the importance of a reliable instrument for assessor’s to use to guide them to assessing competency in a consistent way.
Warwick Maloney
Reliability in an assessment instrument will see that a students answers are repeatable and thus demonstrate appropriate competence.
Warwick Maloney
Reliability in an assessment instrument will see that a students answers are repeatable and thus demonstrate appropriate competence.
Hannah Pringle
Guidance for the assessor in the form of benchmark answers is crucial to ensuring that consistency in assessment is maintained not only by different assessors, by also by the same assessor for different candidates.
Sandy Welton
Hannah this is very true. It is not just a requirement in order for a tool to pass an ASQA audit. It is essential for reliability and consistency of assessment.
Further, reliable and consistent assessment means that we are being fair and without bias. Everyone is assessed to the same benchmark.
Hannah Pringle
Guidance for the assessor in the form of benchmark answers is crucial to ensuring that consistency in assessment is maintained not only by different assessors, by also by the same assessor for different candidates.
Sandy Welton
Hannah this is very true. It is not just a requirement in order for a tool to pass an ASQA audit. It is essential for reliability and consistency of assessment.
Further, reliable and consistent assessment means that we are being fair and without bias. Everyone is assessed to the same benchmark.
Sikiki
Model answers and/or benchmarks are a great guide to determine a candidate’s knowledge.
Sikiki
Model answers and/or benchmarks are a great guide to determine a candidate’s knowledge.
Beth
Thanks for the article, it’s great to be able to read the information in detail and in-depth.
Beth
Thanks for the article, it’s great to be able to read the information in detail and in-depth.
Carolyn Laycock
Great article, and shows how important model answers and benchmarks are in reliability.
Carolyn Laycock
Great article, and shows how important model answers and benchmarks are in reliability.