| The aim of the project is to gain insight into specific problems concerning validity and reliability in the assessment of teachers? competences based on the combined use of (authentic) instruments, and to derive implications for the way these problems should be resolved. In this project, two main problems concerning the quality of a teacher assessment procedure will be analysed and addressed: (1) combining data from different authentic instruments and (2) monitoring the quality of assessors? decision-making (i.e., primarily the quality of the assessors? arguments). The central question of this study is ?How should assessors? decision-making be monitored when a combined set of different authentic instruments is used in a teacher assessment procedure??. More specific research questions are the following: (1) how and to what extent can content criteria be used to monitor assessors? decision-making? (2) how can different kinds of data from the instruments be combined into an overall judgement? (3) what effect do benchmarked performance standards have on the quality of the decision-making process? and (4) which guidelines and tools for assessors? decision-making improve the quality of the decision-making process? Respondents are assessor-pairs. Data has been collected of audio-taped assessors? discussions during decision-making, their reflections on the quality of their decision-making process - particularly the way data from different instruments are combined -, and the final assessors? product: feedback reports. The development and evaluation of an assessment procedure for beginning teachers? interpersonal competence will serve as research context. Teachers? interpersonal competence involves the relationship between the teacher and his or her students. A qualitative content analysis will be used to analyse the data. Research questions will be addressed by analysing the decision-making processes of assessor-pairs (N=12) who are systematically assigned to different conditions (i.e., different guidelines and tools). For each condition, the analysis should first result in an extensive description of problems pertaining to validity and reliability, perceived by both assessors and the researcher. Second, the perceived quality of the decision-making process will be described. Finally, a conclusion will be drawn on the impact of the different guidelines and tools on the quality of the decision-making process. |
