There are four Principles of Assessment – Reliability, Fairness, Flexibility and Validity.  Here we discuss Reliability.  This will be followed by additional Blogs which will discuss the remaining Principles of Assessment.

Principle of Reliability

Reliability refers to the consistency of the interpretation of evidence and the consistency of assessment outcomes.

Reliability is one of the four Principles of Assessment.  In practice, it means that under the same conditions for the same unit of competency, all assessors should reach the same decision as to whether the candidate is competent, based upon the evidence collected.

Therefore, your assessment tool must provide guidance for the assessor.  In reality this means that, for every assessment instrument provided to the candidate, there should be a “sister” instrument for the assessor (the assessor guide).  The assessor guide must provide instructions to the assessor to guide their judgement of satisfactory performance or answers to questions.  Thus:

• For an oral questioning instrument, you need to provide the questions and the answers expected, together with any necessary guidance on how far the answers can deviate from those provided. There must be a space for the assessor to write the actual answers given and another space for their comments.
• For a written questioning instrument, you also need model answers. Where the answer is likely to vary in wording, this should be stated (Response can vary).  Then, instead of a precisely worded answer you can list key points that need to be addressed in the candidate’s response.

Note that for Cert I to Cert III, you are more likely to be able to provide precise written answers.  When you are writing assessment guidance for higher AQF levels, the answers are expected to be more complex and key points may be all you can provide.

• For observation of performance, you need an observation form for the assessor to complete during the observation. The form must include the instructions to the candidate and a list of precisely what the assessor should observe during the task.  For instance, if it is a pre-start check of an item of plant, what should be checked?  Always break the task down into everything the assessor must be able to see.
• Where the instrument relates to performing a task and providing evidence of having done so (e.g. a report) there must still be guidance for the assessor on the key points to look for. It is important to be precise.  For instance, if the unit is Cert II and involves typing a letter, are typing, spelling or grammatical errors permitted and if so, what percentage?  Alternatively, if the unit is Cert IV and requires the candidate to document a report to the company director, the assessor should be directed to require professional language appropriate to the audience, with correct grammar and spelling.

As trainers and assessors within the Vocational Education and Training (VET) sector, it is important that we not only understand the Principles of Assessment but that we also apply the Principles of Assessment when designing and developing assessment tools and conducting assessment.  TAEASS502 Design and Develop Assessment Tools is a core unit in the TAE40116 Certificate IV in Training and Assessment and TAE50116 Diploma of Vocational Education and Training.

Written by Sandy Welton

 

Please post your thoughts below on the Principle of Reliability.