Efectos de respuesta en encuestas administradas: diferencias entre la aplicación visual y la oral

Vidal Díaz de Rada

doi:10.5565/rev/papers.2920

Response effects in interviewer-administered surveys: Differences between visual and oral modes of survey application

Authors

Vidal Díaz de Rada Universidad Pública de Navarra https://orcid.org/0000-0002-9638-3741

PDF (ES)

Abstract

This paper discusses the complacency and satisficing effect in surveys conducted in Spain, specifically analysing the more frequent selection of the first response options when the respondent has visual information, and a bias towards the last response options when the information is transmitted orally. In addition, it examines whether these effects are similar in the questions with (fully-labelled) vertical response choices and in those displayed on a horizontal scale (labelled at the ends). Using a face-to-face survey that employed show cards, a comparison was made between the answers of respondents who used show cards and the answers of respondents who did not use these cards.
The analysis of results revealed the presence of primacy and recency effects in attitu- dinal questions. These effects were found to be much stronger in questions with vertical response choices than in those presented horizontally. While the cards included an «I don’t remember / I don’t know, I can’t say» response choice at the end, this was not the most frequently chosen by those who responded using cards. In fact, there was a greater number of «I don’t know» answers among the respondents who did not use show cards. There were very minor response changes according to the type of question used, specifically considering multiple choice questions and rating scale batteries. The subject of the research, which is a complex concept that ha received little attention in Spanish society, could explain the
effects found in the study.

Keywords

acquiescence, response effects, face-to-face survey, battery questions, response card

References

Alvira, Francisco y Serrano, Araceli (2015). «Diseños y estrategias de investigación social». En: García Ferrando, M. et al. (eds.). El análisis de la realidad social. Madrid: Alianza.

Alwin, Duane y Krosnick, Jon (1985). «The Measurement of Values in Surveys: A Comparison of Ratings and Rankings». Public Opinion Quarterly, 49, 535-552. https://doi.org/10.1086/268949

Ayidiya, Stephen A. y McClendon, Mckee J. (1990). «Response effects in mail surveys». Public Opinion Quarterly, 54, 229-247. https://doi.org/10.1086/269200

Bogner, Kathrin y Landrock, Uta (2016). Response Biases in Standardised Surveys. GESIS Survey Guidelines. Mannheim: GESIS – Leibniz Institute for Social Sciences. https://doi.org/10.15465/gesis-sg_en_016

Bradburn, Norman M. (1983). «Response Effects». En: Rossi, Peter H.; Wright, James D. y Anderson, Andy B. (eds.). Handbook of Survey Research». Nueva York: Academic Press.

Camacho, Carlos; Romero, Bélgica; Valero, Sergi y Camacho, Carmen (2000). «¿Cómo hacen encuestas los encuestadores?». Metodología de Encuestas, 2 (1), 117-133.

Centro de Investigaciones Sociológicas (2014a). Estudio ciudadanía (ISSP), estudio número 3020.

Centro de Investigaciones Sociológicas (2014b). Informe metodológico del estudio 3020: Ciudadanía (ISSP).

Díaz de Rada, Vidal (2015). Manual del trabajo de campo en la encuesta (presencial y telefónica). Madrid: Centro de Investigaciones Sociológicas.

Díaz de Rada, Vidal y Núñez Villuendas, Adoración (2008). Estudio de las incidencias en la investigación mediante encuesta. Madrid: Centro de Investigaciones Sociológicas.

Dillman, Don A.; Smyth, Jolene D. y Christian, Leah Melanie (2009). Internet, Phone, Mail and Mixed-Mode Surveys: The Tailored Design Method (3.ª ed.). Nueva York: Wiley.

Dillman, Don A.; Smyth, Jolene D. y Christian, Leah Melanie (2014). Internet, Mail and Mixed-Mode Surveys: The Tailored Design Method (3.ª ed.). Nueva York: Wiley.

Fricker, Scott; Galesic, Mirta; Tourangeau, Roger y Yan, Ting (2005). «An Experimental Comparison of Web and Telephone Surveys». Public Opinion Quarterly, 69, 370-392. https://doi.org/10.1093/poq/nfi027

García Ferrando, Manuel y Llopis, Ramón (2015). «La encuesta». En: García Ferrando, M. et al. (eds.). El análisis de la realidad social. Madrid: Alianza.

Gilljam, Mikael y Granberg, Donald (1993). «Should we take don’t know for an answer?». Public Opinion Quarterly, 57, 348-357. https://doi.org/10.1086/269380

Gobierno de los Estados Unidos de América, Oficina del Censo (2017). United States Census Bureau 2016.

Gwartney, Patricia A. (2007). The Telephone Interviewer’s Handbook: How to Conduct Standardized Conversations. San Francisco: Jossey-Bass.

Häder, Sabine; Häder, Michael y Kuhne, Mike (2012). «Introduction: telephone surveys in Europe». En: Häder, Dsninr; Häder, Michael y Kuhne, Mike (eds.). Telephone Surveys in Europe: Research and Practice. Manheim: Springer.

Hamilton, Lawrence (2006): Statistics with STATA. Toronto: Thomson.

Heerwegh, Dirk (2009). «Mode differences between face-to-face and web surveys: An experimental investigation of data quality and social desirability effects». International Journal of Public Opinion Research, 21, 111-120. https://doi.org/10.1093/ijpor/edn054

Heerwegh, Dirk y Loosveldt, Geert (2008). «Face to face versus web surveying in a high internet coverage population». Public Opinion Quarterly, 72 (5), 836-846. https://doi.org/10.1093/poq/nfn045

Herzog, A. Regula y Rodgers, Willard L. (1988). «Interviewing Older Adults: Mode Comparison Using Data from a Face-to-Face Survey and a Telephone Survey». Public Opinion Quarterly 52, 84-99. https://doi.org/10.1086/269083

Holbrook, Allyson L.; Green, Melanie C. y Krosnick, Jon A. (2003). «Telephone versus face-to-face interviewing of national probability samples with long questionnaires». Public Opinion Quarterly, 67 (1), 79-125. https://doi.org/10.1086/346010

Holbrook, Alyson L.; Krosnick, Jon A.; Moore, David y Tourangeau, Roger (2007). «Response order effects in dichotomous categorical questions presented orally: the impact of question and respondent attributes». Public Opinion Quarterly, 71, 325-348. https://doi.org/10.1093/poq/nfm024

Kish, Leslie (1995). Diseño estadístico para la investigación. Madrid: Centro de Investigaciones Sociológicas.

Krosnick, Jon A. (1991). «Response strategies for coping with the cognitive demands of attitude measures in surveys». Applied Cognitive Psychology, 5, 213-236. https://doi.org/10.1002/acp.2350050305

Krosnick, Jon A. (1999). «Survey research». Annual Review of Psychology, 50, 537-567. https://doi.org/10.1146/annurev.psych.50.1.537

Krosnick, Jon A. (2015). «Improving Question Design to Maximize Reliability and Validity». En: Krosnick, Jon (ed.). The Future of Survey Research: Challenges and Opportunities (conferencia), Stanford University, octubre-noviembre 2012.

Krosnick, Jon y Alwin, Duane (1987). «An Evaluation of a Cognitive Theory of Response Order Effects in Survey Measurement». Public Opinion Quarterly, 51, 201-219. https://doi.org/10.1086/269029

Krosnick, Jon A.; Narayan, Sowmya y Smith, Wendy R. (1996). «Satisficing in Surveys: Initial Evidence». New Directions for Evaluation, 70, 29-44. https://doi.org/10.1002/ev.1033

Krosnick, Jon A.; Holbrook, Allyson L.; Berent, Matthew K.; Carson, Richard T.; Hanemann, W. Michael; Kopp, Raymond J.; Mitchell, Robert Cameron; Presser, Stanley; Ruud, Paul A.; Smith, V. Kerry; Moody, Wendy R.; Green, Melanie C. y Conaway, Michael (2002). «The Impact of “No Opinion” Response Options on Data Quality: Non-Attitude Reduction or an Invitation to Satisfice?». Public Opinion Quarterly, 66, 371-403. https://doi.org/10.1086/341394

Krosnick, Jon A. y Presser, Stanley (2010). «Question and Questionnaire Design». En: Mardsen, Peter V. y Wright, James D. (eds.). Handbook of Survey Research. Emerald Group Publishing Limited.

Leeuw, Edith D. de (1992). Data Quality in Mail, Telephone and Face to Face Surveys. Ámsterdam: TT-Publikaties.

Leeuw, Edith D. de; Suzer-Gurtekin, Z. Tuba y Hox, Joop J. (2019). «The design and implementation of mixed-mode surveys». En: Johnson, Timothy P.; Pennell, Beth-Ellen; Stoop, Ineke y Dorer, Brita (eds.). Advances in Comparative Survey Methods: Multinational, Multiregional, and Multicultural Contexts (3MC). Hoboken: Wiley.

Lundmark, Sebastian; Gilljam, Mikael y Dahlberg, Stefan (2016). «Measuring generalized trust: An examination of question wording and the number of scale points». Public Opinion Quarterly, 80 (1), 26-43. https://doi.org/10.1093/poq/nfv042

Ministerio de Educación, Cultura y Deporte (2017). Sistema estatal de indicadores de educación 2016. Madrid: Gobierno de España.

Narayan, Sowmya y Krosnick, Jon (1996). «Education Moderates some Response Effects in a Attitude Measurement». Public Opinion Quarterly, 60, 58-88. https://doi.org/10.1086/297739

Narayan, Sowmya; Krosnick, Jon A.; Mulligan, Kenneth; Smith, Wendy R.; Green, Melanie C. y Bizer, George Y. (2006). Non-differentiation in Ratings: Tests of Satisficing Theory Predictions. Ohio: Department of Psychology, Ohio State University.

Nicolaas, Gerry; Thomson, Kert y Lynn, Peter (2000). The feasibility of conducing Electoral Surveys in the UK by telephone. Londres: NCSR.

O’Rourke, Diane; Sudman, Seymour; Johnson, Timothy Johnson y Burris, Jane (1999). «Cognitive testing of cognitive functioning questions». En: Schwarz, Norbert et al. (eds.). Cognition, Aging, and Self-Reports. Filadelfia: Psychology Press.

Peytchev, Andy; Couper, Mick P.; McCabe, Sean E. y Crawford, Scott D. (2006). «Web survey design: paging versus scrolling». Public Opinion Quarterly, 70 (4), 596-607. https://doi.org/10.1093/poq/nfl028

Revilla, Melanie; Saris, Willem E. y Krosnick, Jon A. (2013). «Choosing the Number of Categories in Agree–Disagree Scales». Sociological Methods & Research, 43 (1), 73-97. https://doi.org/10.1177/0049124113509605

Roberts, Caroline; Gilbert, Emily; Allum, Nick y Eisner, Léïla (2019). «Research Synthesis: Satisficing in Surveys: A Systematic Review of the Literature». Public Opinion Quarterly, 83 (3), 598-626. https://doi.org/10.1093/poq/nfz035

Schaie, K. Warner (1996). «Intellectual Development in Adulthood». En: Birren, J. E. y Schaie, K. W. (eds.). Handbook of the Psychology of Aging. San Diego: Academic Press.

Schuman, Howard y Presser, Stanley (1981). Questions and Answers in Attitude Surveys. San Diego: Academic Press.

Sudman, Seymour y Bradburn, Norman M. (1974). Response effects in surveys. Chicago: Aldine Publishing Company.

Sudman, Seymour y Bradburn, Norman M. (1982). Asking questions. Nueva York: Wiley.

Tourangeau, Roger (1984). «Cognitive sciences and survey methods». En: Jabine, T.; Straf, M.; Tanur, J. y Tourangeau. R. (eds.). Cognitive aspects of survey methodology. Washington DC: National Academic Press.

Tourangeau, Roger y Rasinski, Kennetj (1998). «Cognitive Processes Underlying Context Effects in Attitude Measurement». Psychological Bulletin, 103, 299-314. https://doi.org/10.1037/0033-2909.103.3.299

Visauta, Restituto (1989). Técnicas de investigación social. Barcelona: PPU.

Zhang, Chan y Conrad, Frederic (2014). «Speeding in Web Surveys: The tendency to answer fast and its association with straighlining». Survey Research Methods, 8 (2), 127-135. https://doi.org/10.18148/srm/2014.v8i2.5453

Author Biography

Vidal Díaz de Rada, Universidad Pública de Navarra

Licenciado y doctor en Sociología, licenciado en Investigación y Técnicas de Mercado y catedrático en el Departamento de Sociología de la Universidad Pública de Navarra. Autor de más de un centenar de artículos en revistas científicas y de 25 libros, la mayoría sobre sociología del consumo y metodología.

Entre sus últimas publicaciones destacan: Técnicas multivariantes de interdependencia (RaMa, 2018), Internet como modo de administración de encuestas (CIS, 2019), «¿Influye en la respuesta el orden de administración de escalas valorativas 0-10?» (REIS, 2019) y «Calidad de respuesta y modo de administración del cuestionario» (RIS, 2019).