Factorial Survey: Benefits, Applications and Practical Guidelines

The article focuses on the application of the factorial survey design to social- psychological studies. The factorial survey is an experimental technique that allows testing hypotheses about causal relations. The key advantages of factorial survey are (1) higher external validity compared to classical laboratory experiments, (2) lower susceptibility to social desirability effects compared to classical survey approach, and (3) its ability to detect implicit determinants of social perception. Nevertheless, facto- rial survey design is rarely used in psychology. This work gives an introduction to the factorial survey design, describes its benefits and areas of application. Moreover, it gives practical guidelines on how to plan, conduct, and analyze the results of a factorial survey study, accompanied by examples from our own research. The research was supported within the framework of the Program for Basic Research of National Research University Higher School of Economics (HSE) in 2014- 2015.

1. Devyatko I.A. Prichinnost’ v obydennom soznanii i v sotsiologicheskom ob»yasnenii: kontury novogo issledovatel’skogo podkhoda [Causality in Everyday Knowledge and in Sociological Research: an Outline of New Exploratory Approach]. Sotsiologiya: 4M [Soci- ology, 4M], 2007, no. 25, pp. 5—21.
2. Aguinis H., Bradley K.J. Best Practice Recommendations for Designing and Imple- menting Experimental Vignette Methodology Studies // Organizational Research Meth- ods. 2014. Vol. 17(4). P. 351—371. doi:10.1177/1094428114547952
3. Alexander C.S., Becker H.J. The Use of Vignettes in Survey Research // Public Opin- ion Quarterly. 1978. № 42. 93—104. https://doi.org/10.1086/268432
4. Armacost R.L., Hosseini J.C., Morris S. A. Rehbein K.A. An Empirical Comparison of Direct Questioning, Scenario, and Randomized Response Methods for Obtaining Sensitive Business Information // Decision Science.1991. 22: 1073-1090. doi:10.1111/j.1540-5915.1991.tb01907.x
5. Atzmüller C., & Steiner P.M. Experimental vignette studies in survey research. Meth- odology // European Journal of Research Methods for the Behavioral and Social Sci- ences. 2010. № 6. P. 128—138. doi:10.1027/1614-2241/a000014
6. Auspurg K., Hinz T., & Liebig S. Komplexität von Vignetten, Lerneffekte und Plausi- bilität im Faktoriellen Survey [Complexity, learning effects and plausibility of vignettes in the factorial survey design] // Methoden — Daten — Analysen. 2009. № 1. P. 59—96.
7. Auspurg K., Sauer C. Factorial Survey Designs. GESIS Summer School in Survey Methodology, presentations. 2014.
8. Badgett M.V.L., Folbre N. Job gendering: occupational choice and the marriage mar- ket // Industrial Relations. 2003. № 42. P. 270—298. doi:10.1111/1468-232X.00290
9. Baker P.M. Ageism, sex, and age: a factorial survey approach // Canadian Journal on Aging. 1983. № 2. P. 177—184.
10. Boots D.P., Cochran J.K., Heide K.M. Capital punishment preferences for special of- fender populations // Journal of Criminal Justice.2003. № 31. P. 553—565. doi:10.1016/j. jcrimjus.2003.08.003
11. Burstin K., Doughtie E.B., & Raphaeli A. Contrastive vignette technique: An indirect methodology designed to address reactive social attitude measurement // Journal of Ap- plied Social Psychology. 1980. № 10. P. 147—165. doi:10.1111/j.1559-1816.1980.tb00699.x
12. Cavanaugh G.F., & Fritzsche D.J. Using vignettes in business ethics research. In
13. L.E. Preston (Ed.) // Research in corporate social performance and policy. London: Jai Press. 1985. Vol. 7. P. 279—293.
14. de Wolf I., van der Velden R. Selection processes for three types of academic jobs: an experiment among dutch employers of social sciences graduates // European Sociological Review. 2001. № 17 (3). P. 317—330. doi:10.1093/esr/17.3.317
15. Degenholtz H.B., Kane R.A., Kane R.L., Finch M.D. Long-term care case managers’ out-of-home placement decisions: an application of hierarchical logistic regression // Re- search on Aging. 1999. № 21. P. 240—274. doi:10.1177/0164027599212005
16. Dülmer H. (2007):  Experimental  Plans  in  Factorial  Surveys:  Random  or Quota Design? // Sociological Methods & Research. 2007. № 35. P. 382—409. doi;10.1177/0049124106292367
17. Elis L.A., Simpson S.S. Informal sanction threats and corporate crime: additive versus multiplicative models // Journal of Research in Crime and Delinquency. 1995. № 32. P. 399—424. doi:10.1177/0022427895032004002
18. Finch J. The vignette technique in survey research // Sociology. 1987. № 21. P. 105— 114. doi:10.1177/0038038587021001008
19. Ganong L., Coleman M. Measuring intergenerational obligations // Journal of Mar- riage and Family. 2005. № 67 (4). P. 1003—1011. doi:10.1111/j.1741-3737.2005.00190.x
20. Ganong L., Coleman M. Multiple segment factorial vignette designs // Journal of Mar- riage and Family. 2006. № 68 (2). P. 455—468. doi:10.1111/j.1741-3737.2006.00264.x
21. Hox J.J, Kreft I. G.G., Hermkens P.L.J. The Analysis of Factorial Surveys // Sociological Methods & Research. 1991. № 19 (4). P. 493—510. doi:10.1177/0049124191019004003
22. Hughes R., &  Huby M. The application of vignettes in social and nursing  re- search // Journal of Advanced Nursing. 2002. № 37. P. 382—386. doi:10.1046/j.1365- 2648.2002.02100.x
23. Hyman M.R., & Steiner S.D The vignette method in business research: Current uses, limitations and recommendations // In E. W. Stuart, D. J. Ortinau, & E. M. Moore (Eds.), Marketing: Moving toward the 21st century. Rock Hill, SC: Winthrop University School of Business. 1996. P. 261—265.
24. Jasso G. Whom shall we welcome? Elite judgments of the criteria for the selection of im- migrants // American Sociological Review. 1988. № 53. P. 919—932. doi:10.2307/2095900
25. Jasso G. Factorial-Survey Methods for Studying Beliefs and Judgments. // Sociologi- cal Methods and Research. 2006. № 34. P. 334—423. doi:10.1177/0049124105283121
26. Jasso G., Opp K.-D. Probing the character of norms: a factorial survey analysis of the norms of political action // American Sociological Review. 1997. № 62. P. 947—964. doi:10.2307/2657349
27. Jasso G., Rossi P.H. Distributive justice and earned income // American Sociological Review. 1977. № 42. P. 639—651. doi:10.2307/2094561
28. Louviere J.J., Flynn T.N. and Carson R.T. Discrete Choice Experiments Are Not Con- joint Analysis // Journal of Choice Modelling. 2010. № 3(3). P. 57—72. doi:10.1016/ s1755-5345(13)70014-9
29. Ludwick R.,& Zeller R.A. The factorial survey: An experimental method to replicate real world problems // Nursing Research. 2001. № 50 (2). P. 129—133. doi:10.1097/00006199- 200103000-00009
30. Murphy K.R., Herr B.M., Lockhart M.C.,& Maguire E. Evaluating the performance of pa- per people // Journal of Applied Psychology. 1986. № 71. P. 654—661. doi:10.1037/0021- 9010.71.4.654
31. O’Toole A.W., O’Toole R., Webster S., Lucal B. Nurses’ recognition and reporting of child abuse: a factorial survey // Deviant Behavior. 1993. № 14. P. 341—363. doi:10.1080/01639625.1993.9967950
32. Opp K.-D. When do norms emerge by human design and when by the unintended consequences of human action? The example of the no-smoking norm // Rationality and Society. 2002. № 14. P. 131—158. doi:10.1177/1043463102014002001
33. Pierce C.A., & Aguinis H. Using virtual reality technology in organizational behavior research // Journal of Organizational Behavior. 1997. № 18. P. 407—410. doi:10.1002/ (sici)1099-1379(199709)18:5<407::aid-job869>3.0.co;2-p
34. PorterJ.D.M.Genderdifferencesinmanagers’conceptionsandperceptionsofcommitment to the organization // Sex Roles. 2001. № 45. P. 375—398. doi:10.1023/a:1014313732152
35. Rossi P.H. The Application of Latent Structure Analysis to the Study of Social Strati- fication. Ph.D. dissertation, Columbia University, United States. ProQuest Digital Dis- sertations database. 1951. (Publication No. AAT 0002855).
36. Rossi P.H., Sampson W.A., Bose C.E., Jasso  G.  Passel  J.  Measuring  household  so- cial standing // Social Science Research. 1974. № 3. P. 169—190. doi:10.1016/0049- 089x(74)90011-8
37. Rossi P. H., Weber-Burdin E. Sexual harassment on the campus // Social Science Re- search. 1983. № 12. P. 131—158. doi:10.1016/0049-089x(83)90003-0
38. Rossi P.H., Anderson A.B. The Factorial Survey Approach: An Introduction. // In: Rossi, P. H., Nock, S. L. (Eds.): Measuring Social Judgments. The Factorial Survey Ap- proach. Beverly Hills. 1982. P. 15—67.
39. Sampson W.A., Rossi P.H. Race and family social standing // American Sociological Review. 1975. № 40. P. 201—214. doi:10.2307/2094345
40. Sauer C., Auspurg K., Hinz T. and Liebig S. The Application of Factorial Surveys in General Populations Samples: The Effects of Respondent Age and Education on Response Times and Response Consistency // Survey Research Methods. 2011. № 5. P. 89—102. doi:10.18148/srm/2011.v5i3.4625
41. Shlay A.B., DiGregorio D.A. Same city, different worlds: examining gender- and work- based differences in perceptions of neighborhood desirability // Urban Affairs Review. 1985. № 21. P. 66—86. doi:10.1177/004208168502100107
42. Short R., Magaña L. Political rhetoric, immigration attitudes, and contemporary prej- udice: a Mexican American dilemma // The Journal of Social Psychology. 2002. № 142. P. 701—712. doi: 10.1080/00224540209603930
43. Shulte A. Consensus versus disagreement in disease-related stigma: a comparison of reactions to aids and cancer patients // Sociological Perspectives. 2002. № 45. P. 81—104. doi:10.1525/sop.2002.45.1.81
44. Sniderman P.M., Grob D.B. Innovations in experimental design in attitude surveys // Annual Review of Sociology. 1996. № 22. P. 377—399. doi:10.1146/annurev.soc.22.1.377
45. Stjohn C., Healdmoore T. Fear of black strangers // Social Science Research. 1995.
46. № 24. P. 262—280. doi:10.1006/ssre.1995.1010
47. Thurman Q.C., Lam J.A., Rossi P.H. Sorting out the cuckoo’s nest: a factorial survey approach to the study of popular conceptions of mental illness // The Sociological Quar- terly. 1988. № 29. P. 565—588. doi:10.1111/j.1533-8525.1988.tb01435.x
48. Wallander L. 25 years of factorial surveys in sociology: a review // In: Social Science Research. 2009. № 38. P. 505—520. doi:10.1016/j.ssresearch.2009.03.004
49. Ward Jr. R.E., Lewis J.M., Benson D.E. Sports officials’ judgments of spectator behav- ior: a factorial survey // Michigan Sociological Review. 2002. № 16. P. 147—170.
50. Wason K.D., Polonsky M.J., & Hyman M.R. Designing vignette studies in market- ing // Australasian Marketing Journal. 2002. № 10. P. 41—58. doi:10.1016/s1441- 3582(02)70157-2
51. Will J.A. The dimensions of poverty: public perceptions of the deserving poor // Social Science Research. 1993. № 22. P. 312—332. doi:10.1006/ssre.1993.1016