In collaboration with Dana Garbarski (Loyola University Chicago), the UWSC Faculty Director Nora Cate Schaeffer and Senior Scientist Jennifer Dykema present findings from a web survey assessing the impact of response option order and question order on the distribution of responses to the self-rated health (SRH) question in “Online First” for the journal Quality of Life Research.
In an online panel survey sponsored by Time-sharing Experiments for the Social Sciences (TESS), the authors implemented a 2-by-2 between-subjects factorial experiment, manipulating the following levels of each factor: (1) order of response options (“excellent” to “poor” versus “poor” to “excellent”) and (2) order of SRH item (either preceding or following the administration of domain-specific health items). Results indicated mean SRH was higher (better health) and proportion in “fair” or “poor” health lower when response options were ordered from “excellent” to “poor” and SRH was presented first compared to other experimental treatments. Presenting SRH after domain-specific health items increased its correlation with these items, particularly when response options were ordered “excellent” to “poor.” Among participants with the highest level of current health risks, SRH was worse when it was presented last versus first.
“While more research on the presentation of SRH is needed across a range of surveys, we suggest ordering response options from ‘poor’ to ‘excellent’ in order to reduce positive clustering, and given the question order effects found, we suggest presenting SRH before domain-specific health items in order to increase inter-survey comparability, as domain-specific health items will vary across surveys,” advises Dr. Garbarski.
(Users within the University of Wisconsin can download the full article here. Others should contact Jennifer Dykema for a copy.)