Cross-national data: From questionnaire development to analysis

Description

This course covers all aspects of working with multi-national, multi-lingual, and multi-cultural (3MC) data. It will introduce the students to the concepts of equivalence and different types of bias that can threaten the comparability of 3MC data.

Following a total survey error perspective, it will discuss the steps entailed in collecting and analyzing 3MC data. Participants will learn about specific 3MC challenges (e.g., bias) and how to address them before data collection. The course will introduce 3MC questionnaire design, translation, and testing but also discuss sampling approaches, optimal mode choices, and data collection challenges. Participants will learn how to assess the comparability of 3MC data with measurement invariance testing and how to deal with lack of comparability. The course also includes consultation time with 3MC experts.

Following a 3MC Total Survey Error framework, this course addresses major stages of 3MC data collection and analysis. It will discuss specific challenges of coverage, sampling, and nonresponse and how to reduce measurement error in a 3MC context. Topics include state of the art 3MC questionnaire design, translation and adaptation approaches of the source questionnaire, and pretesting approaches (e.g., cognitive interviewing, web probing, behaviour coding) in a 3MC context. The course also provides guidance regarding optimal mode choices and 3MC data collection challenges as well as different data harmonization approaches of 3MC data.

Once the data is collected, it is necessary to assess whether the data is indeed comparable across countries, cultures or languages. Therefore, this course dedicates a full day on the assessment of comparability with measurement invariance tests and introduces different approaches (strict, partial, and approximate). On this day, there is also a computer lab where students learn to run measurement invariance tests with MPLUS or R.

However, a common finding of measurement invariance tests is that data are not comparable. Therefore, this course also discusses different strategies how to deal with missing comparability. In addition, it will touch on relevant considerations of 3MC survey quality and ethical considerations. The course also provides the opportunity to receive individual consultations with 3MC experts regarding own research projects.

Current developments will be reviewed and research literature will be discussed. After the course, participants are ready to apply the learned towards their own 3MC surveys, are able to critically assess existing 3MC surveys and survey documentation.

Participants are requested to bring their own laptop computer. Software will be available online.

A good pre-course is our Summer School course “Survey Research: Design, Implementation and Data Processing” (S15,  12-16 August) 

S72 Day to Day Program 2024.pdf

Target audience

The course is intended for advanced students and professionals in such fields as social and behavioral research, marketing, business, health sciences, and official statistics. The course aims at researchers who intend to design and analyze their own multi-national, multi-cultural, or multi-lingual (3MC) survey, but also at researchers who analyze secondary 3MC data such as the European Social Survey (ESS) or the International Social Survey Program (ISSP).

This course assumes general knowledge of survey methodology and statistics. A basic knowledge of methods and statistics (e.g. descriptive statistics) is required. Some familiarity with confirmatory factor analysis would be helpful but is not mandatory. In the course MPLUS will be used, but specific knowledge of MPLUS is not required.

Aim of the course

More and more cross-national surveys are being conducted. Due to data collection efforts of large cross-national surveys such as the European Value Survey or the International Social Survey Programme, there are large opportunities for researchers and students to work with cross-national data. However, 3MC data have an additional “layer of complexity” in comparison to single country/language studies. This course is intended as an introduction to 3MC data collection and analysis at the Master/graduate level for anyone who plans to collect 3MC data or wants to work with already collected 3MC data.

Study load

The course consists of formal lectures, practical exercises(with feedback) that apply the tools presented in the lectures, and a computer lab.

A typical course day starts at 9.00 and ends at 17.00 with breaks for coffee, lunch and tea. At the last day (Friday) after the morning program, there is the opportunity for individual consultation. Students who want to use this opportunity are expected to prepare for this in advance.

You will receive a certificate upon course completion. Please be aware that this course does not include graded activities, and therefore we cannot provide a transcript of grades.

Tags

• M&S
• 3MC
• comparability
• Multi-national
• multi-lingual
• multi-cultural
• survey methods
• translation
• questionnaire design
• pretesting
• measurement invariance testing