Italian University student evaluation of teaching: does it changes over time?

This paper concerns the measurement of the Student Evaluation of Teaching (SET) by undergraduates. For almost 15 years Italian Universities have been monitoring that aspect, performing on students’ opinions about teaching, e.g. on teacher punctuality, clearness and willingness, and on suitability of the facility, logistics, and on course scheduling and managing. Each University carries out a survey with own procedures and through a questionnaire consisting of two set of items: a set of items equal for all Italian Universities, and a set of items built by each University. The survey basic aim is to gather information on teaching skill, educational goals, coordination among teachings, and adequacy of resources. This information is useful to policy makers to improve the SET and the Degree Courses. The main aim of this paper is to investigate changes over time of the SET. Teacher’s quality is obviously a concept that cannot be directly measurable. It is a latent concept and its measurement needs the specification of convenient measurable dimensions that are strictly correlated to the concept. The above mentioned questionnaire includes items that refer to SET. The items are measured by a Likert-type scale with 4 response categories. To analyse SET and its changes over the time, a Rasch model (Rasch, 1966) is used: it is necessary to transform the data from ordinal scale to interval one and to preserve the rank ordering (Doig and Groves, 2006). Rasch model gives items and persons measures on the same continuum: this is very useful to make comparisons among items, among students satisfaction levels, and among items and satisfaction levels, and finally among different points of time. The partial credit model for rating scales (Masters, 1988; Curtis and Bowman, 2007) appears the useful procedure for the analysis of these items. The dataset regards the questionnaires administrated and collected by the Engineering Faculty of the University of Pavia during three academic years - from 2004/05 to 2006/07. The 3-years period is taken into consideration because it is the longest period over the questionnaires structure and items have not been modified. The statistical unit is the teaching. The focus is on 5 teachings chosen among those performed by the same teacher during the considered period and with many students. Results show: 1) the satisfaction mean level is mainly medium-high, and seems varying over time, 2) the items and person measures are graphically compared to emphasize overall satisfaction vs. single item perceived quality, and more and less appreciated aspect of SET, and their variation over the time.