1. Usability Standards
    Can respondents and interviewers, if they are used, complete the questionnaire easily and as they were intended to?)
  2. Expert Reviews
    Subject matter experts review the questions to assess whether their content is appropriate for measuing the intended concepts, or in which questionnaire design experts assess whether the questions meet the three standards for questions.
  3. Cognitive Interviewing
    Based on a technique called Protocol Analysis--where subjects think aloud as they work on the problems and their verbalizations are recorded.
  4. Behavior Coding
    Tape recording pretest interviews, then making systematic observations of how questions are read and answered, can also provide useful information about the question.
  5. Randomized experiments
    Experimentally compare different methods of data collection, different field procedures, or different versions of the questions.
  6. Split-ballot Experiments
    Another name for randomized experiments.
  7. Validity
    The extent to which the survey measure accurately reflects the intended construct.

    • Estimating Validity with data external to the survey
    • Estimating Validity with Multiple indicators of the same construct
  8. Record Check Study
    Comparing survey responses to existing records.
  9. Simple Response Variance
    The opposite of reliability. When a survey has high reliability for a population, then it has low simple response variance.
  10. Reinterview
    Repeated interviews to measure the same constructs.
  11. Index of Inconsistency
    Reliability as in high reliability for a population (or low simple response variance), then the index of inconsistency which equals (1-reliability)
  12. Gross Difference Rate
    The gross difference rate, which for a dichotomous variable is merely twice the simple response variance.
  13. Cronbach's alpha
    A widely used measure of reliability of multi-item indices.
  14. Content Standards
    Do the questions ask about the right things?
  15. Cognitive Standards
    Do respondents understand the questions consistently; do they have the information required to answer them; are they willing and able to formulate answers to the question
Card Set
Survey Methods, Groves, Chapter 8