Subjective self-assessments are a convenient and widespread method of comparing many aspects of well-being. A potential problem with subjective self-assessments is that people in different countries and socio-economic groups may use different response scale. The anchoring vignette method is a popular tool for adjusting self-assessments for the heterogeneity in reporting behavior. The method rests upon two basic assumptions – response consistency (there are no systematic differences between response scales for self-report and vignette evaluation for the same respondent) and vignette equivalence (different respondents interpret the same vignette in the same way). They are two approaches how to use anchoring vignettes for data analysis. The nonparametric approach which essentially compares the distribution in different socio-economic groups of the rank of the respondent’s self-evaluation among the same respondent’s vignette evaluations. This approach rests upon the two basic assumptions (and the assumption about the order of vignettes if more than two vignettes are used). The second approach is the commonly used parametric model of the anchoring vignette method (CHOPIT model) which requires additional assumptions like modeling the objective reality as a linear function of observed characteristics and an unobserved component, a specific functional form of the thresholds and joint normality of errors.
In the first part of the seminar, a test for the specification of parametric models by using anchoring vignette will be introduced. A basic idea of the test is to compare the rankings that are implied by the parametric model with non-parametric rankings that come directly from the raw data. The test reject the parametric model if the misspecification implies that using the parametric model leads to biased conclusions concerning ranking comparisons across socio-economic groups. We use SHARE 2004 data and apply the test to six health domains. The test always rejects the standard CHOPIT model, but an extended CHOPIT model performs much better. It implies a need for more flexible (parametric or semi-parametric) models than the standard CHOPIT model.
The model misspecification may be less important than the validity of the two identifying assumptions – response consistency and vignette equivalence. In the second part of the seminar, a test of response consistency will be introduced. Its basic idea is to construct a replica vignette which is a vignette reflecting respondent’s own health. Under the response consistency assumption, there should be no systematic difference between the respondent’s self-reported health and the respondent’s evaluation of the replica vignette. We use American Life Panel to test the assumption for five health domains with mixed results.
In the final part of the seminar, the idea of our response consistency test is compared with ideas of other tests (of both identifying assumptions) published in the current literature. The tests are also discussed in the light of the potential misspecification of the parametric CHOPIT model which is usually held as an identifying assumption for testing the response consistency and vignette equivalence. The implication for further research are given.
