A new metric of consensus for Likert-type scale questionnaires: An application to consumer expectations

Abstract

In this study, we present a metric of consensus for Likert-type scales. The statistic provides the level of agreement for any given number of response options as the percentage of consensus among respondents. With this aim, we use a geometric framework that allows us to analytically derive a positional indicator. The statistic is obtained as the relative weight of the distance from the point containing the proportions of observations that fall in each category to the centre of a regular polygon with as many vertices as categories, which corresponds to the point of maximum dissent. The polygon can be regarded as the area that encompasses all possible answering combinations. In order to assess the performance of the proposed metric of consensus, we conduct an iterated forecasting experiment to test whether the inclusion of the degree of agreement in households' expectations improves out-of-sample forecast accuracy of the unemployment rate in seven European countries and the Euro Area. We find evidence that the level of consensus among households contains useful information to predict unemployment rates in all cases. This result shows the potential of agreement metrics to track the evolution of economic variables. Finally, we design a simulation experiment in which we compare the sampling distribution of the proposed metric for three- and five-response alternatives, finding that the distribution of the former shows a higher level of granularity and dispersion.