The ordinal inter-rater agreement in the evaluation of University courses

In this paper we consider the problem of the evaluation of undergraduate and graduate courses, in the context of Italian universities. We present a descriptive measure of ordinal inter-rater agreement that can be associated with a suitable index of satisfaction. This measure is a modification of a previously proposed index, which avoids the problem of paradoxes of both Cohen’s and Fleiss’ kappa statistics. We apply our results to the evaluation of the Bachelors and Masters of Sciences in Economics at the University of Milan-Bicocca.