Research Methods

  1. Intuition
    When a mental shortcut applies previous learning recalled from the unconscious mind
  2. What does the accuracy of intuition depend on?
    • the validity of the origial learning
    • the current situation being one where the prior learning is applicable
  3. Are statistical prediction rules more economical than clinical judgment?
  4. What are the 3 possible ways to make a decision (based on SPR)
    • Method #1: Relying on experience and training to make an intuitive predition
    • Method #2: Relying wholly on the SPR developed to be used in that situation
    • Method #3: Taking account of the output of the SPR but possibly modifying it based on professional experience and intuition
  5. What is a fact?
    When scientists conducting the same research procedure get the same results
  6. What is a theory?
    A convenient way to summarize facts so they can be grasped by our human minds
  7. What is objectivity?
    The degree to which qualified people agree on something
  8. What is nomothetic?
    That which can be generalized (basic principles
  9. What is idiographic?
    That which is unique to an individual
  10. What does science usually come down to?
    the personal experience of the scientist
  11. Who were logical positivists?
    • Revolted against idealism
    • Goal #1: distinguish between science and metaphysics (the problem of demarcation)
    • Goal #2: to recontruct all (scientific) knowledge from experiece or sense data (confirmation of theories)
  12. What is demarcation?
    the ability of the theory to be falsified or the scientist's ability or willingness to state in advance what would count as falsifying it
  13. What do Popper (Neopositivism) believe?
    • science is characterized by the fact that its theories are falsifiable
    • the method of science is to propose bold conjectures and try repeatably to find them false
  14. What is confirmation?
    Acceptance of a theory that is most falsifiable, but not falsifiable
  15. What is Kuhn's view of neopositivism?
    • Neopositivism is not supported by the actual history of science
    • Dependence of theory choice in science is on factors other than observation and logic
  16. Why are paradigms chosen?
    • It solves the problem that lead its competitor to crisis
    • It's more simplistic or has a more accurate empirica fit
  17. Once a paradigm is accepted...
    its basic laws and theories are not subjected to testing and used to solved the problems the paradigm encounters
  18. What is data partially determined by?
    the paradigm they support
  19. What is the difference between immature and mature sciences?
    • Mature science has reached the stage where one paradigm at a time dominates research
    • Immature sciences is where a number of paradigms compete and more time is taken in debate than research
  20. Kuhn suggests the history of science is best understood as...?
    A succession of paradigms
  21. What is a research program?
    A series of complex theories whose hard core remains the same while auxiliary hypotheses are successively modified, replaced, or amplified in order to account for problematic observations
  22. Lakatos suggests that the history of science is best understood in terms of...?
    In terms of competing research programs
  23. When is a research program progressive?
    • 1.) Each new version of the theory (hard core theory plus auxiliaries) preseves the unrefuted content of its predecessor
    • 2.) Each has excess empirical content over its predecessor; it predicts some novel, previously unexpected facts
    • 3.) Some of these predicted facts are corroborated

    • 1&2 = theory progressive
    • 1,2 &3 = theory and empirically progressive
  24. When is a research program degenerating?
    When the change from one version to the next accounts at most for the one anomaly for which the change was made, but does not allow for the prediction and discovery of any novel facts
  25. What is a negative heuristic?
    the plan to avoid falsification of the hard-core, to direct the tests of the theory against the auxiliary hypotheses and suitable modifications there
  26. What is a positive heuristic?
    The development of the auxiliary hypotheses proceeds according to a preconceived plan
  27. What is the difference betweeen immature and mature science (Lakatos)
    • Immature only has negative heuristics
    • Mature has both negative and positive heuristics
  28. Why is it rational to continue to work on a degenerating program?
    • A degenerating program can sometimes be turned around (made progressive again
    • Competition in the field generally leads to more overall progress
  29. Explain the idea of "hierarchy of science"
    • One level of science can get support from other levels of science
    • Interconnection constrains science
  30. How does the interconnection within the hierarchy of science constrain science?
    • Laws and theories of higher levels have to conform to lower-level laws
    • Lower levels also have to permit the phenomenon described by the levels above them
  31. What is reductionism?
    Dividing an entity into its base parts, and explaining its properties or behavior in terms of the base parts' properties or behavior
  32. One has to consider not only the parts of an entity, but also its ______ with its environment in order to understand it.
  33. The demise of reductionism can be attributed to....:
    • Emergence
    • Decoupling
    • Top-Down Causation
    • Supervenience
  34. What is emergence?
    The appearance of properties and processes that are only describable by concepts pertaining to a higher level of analysis, which are neither applicable at the lower level nor reducible to (translatable itno) concepts at the lower level.
  35. What is decoupling?
    The relative economy of levels in the hierarchy of sciences; each level of the hierarchy of sciences has been stabilized in terms of causation and theory, and the partitions between the sciences are fairly well understood
  36. Top-down causation
    interactions at the lower levels cannot be predicted by looking at the structure of these levels alone; higher-level variables, which cannot be reduced to lower-level properties or processes, have genuine causal impact. The higher-level system needs to be considered in giving a complete causal account
  37. Supervenience
    A naturalistic but non-reductionistic account of entities
  38. Reduction will be possible when....
    In the limiting case where B (base) constitutes S (supervenience) under all C's (circumstances) and S (supervenience) is NOT multiply realizable. An example here would be consciousness (S), which is possibly only when there is a certain neural activity
  39. Reduction will NOT be possible when...
    • There are multiple C's (circumstances) that make a difference to S (supervenience) and the C's (circumstances) cannot be defined in terms of B (base).
    • or when S is multiply realizable by an infinite set of B's
  40. What is the general purpose of science?
    To establish knowledge
  41. Scientific research attempts to...
    Simplify the complexity of nature and isolate a particular phenomenon for scrutiny
  42. What does methodology teach science?
    • Ways to think about the relations between variables
    • About causes and effects
    • About conclusions drawn from theory, research, and experience
  43. Scientific Hyptheses attempt to:
    explain, predict, and explore specific relations
  44. What is parsimony?
    • Accept the simplest version or account of the data among the alternatives that are available
    • Adopting the simplest of competing views of explaining a phenomenon
  45. What is a plausible rival hyptheses?
    • Other explanations, outside of the hypothesis, that may explain findings
    • The better the experiment, the fewer the alternatives
  46. What are Findings?
    the results that are obtained
  47. What are Conclusions?
    The explanation of the findings
  48. What are steps of ruling out rival hypotheses?
    • experimental controls
    • selection of subjects
    • various control groups
  49. What is replication?
    Accumuation of multiple investigations
  50. In most research the findings are...
  51. What does methodology have to do with?
  52. What is internal validity?
    The extent to which the intervention accounts for the effect
  53. What is history?
    Any event occurring in the experiment (unplanned events disrupting administration) or outside the experiment (hisotrical events) that may account for the results
  54. What is maturation?
    Processes within the subjects that change over time, such as growing older, strongr, wiser, tired, or bored
  55. What is testing?
    When taking a test has an effect on the subsequent performance or learning
  56. What is instrumentation?
    Changes in the measuring instrument or measurement procedures over time
  57. What is statistical regression?
    The tendency for extreme scores at Time 1 to revert toward the average at Time 2, particularly a problem when subjects are selected because of extreme scores at Time 1
  58. What is selection bias?
    the use of different methods for selecting subjects for experimental conditions causing systematic differences in the groups
  59. What is attrition?
    Participants dropping out of the study over time
  60. What can affect groups in a study?
    combination of history, maturation, and other threats
  61. What is diffusion or imitation of treatment?
    When the intervention given to one group is also provided accidentally to all or some subjects of another group
  62. What is the purpose of a control group with random group assignment?
    To rule out the threats to internal validity because both groups will share the effects of these influences
  63. What is external validity?
    The extent to which the results of an experiment can be generalized or extended outside of the experiment
  64. What are sample characteristics?
    • How representative of the population is the sample?
    • To what extent can the results be generalized to another sample that varies in age, race, ethnic background, education, or any other characteristic
  65. What are stimulus characteristics and settings?
    To what extent is the effect due to the particulars of the setting, experimenters, interviewers, or other factors in experimental arrangement
  66. What is reactivity of experimental arrangements?
    • How much did it matter if the subjects knew they were participating in a study
    • Would the results be the same if the subjects were not aware that they were being studied?
  67. What is multiple treatment interference?
    When one treatment is in the context of other treatments, how do you know which one caused the effect?
  68. What are novelty effects?
    When the effect depends on the fact that the intervention is administered under conditions that are particularly salient, infrequent, or otherwise novel
  69. What is an obtrusive measure?
    • When subjects are aware that their performance is being assessed
    • Concern to external validity b/c awareness that performance is being assessed can alter performance
  70. What is a reactive measure?
    When awareness of assessment leads persons to respond differently from how they normally would
  71. What is test sensitization?
    The administration of a test may in some way sensitize the subjects so that they are affected differently by the intervention
  72. What is time measurement?
    Would the result be the same if the measurement had been taken at another time, especially several months after treatment
  73. Which is more important: internal or external validity?
    Internal validity
  74. What is efficacy?
    treatment outcomes from controlled studies that are conducted under laboratory-type conditions
  75. What is effectiveness?
    treatment outcomes obtained in clinic settings where the usual control procedures are not implemented
  76. What is construct validity?
    Did the study test what the researcher meant to test?
  77. What are confounds?
    features within the experiment that can interfere with the interpretation
  78. What are attention and contact with clients?
    Differential attention across experimental and control groups may be the basis for group differences and threaten constrcut validity
  79. What is the placebo effect?
    An apparently effective treatment or intervention that has no active treatment properties that would be expected to produce change for the problem to which it is applied
  80. What are experimenter expectancies?
    Unintentional process where the expectancies, beliefs, and desires about the results on the part of the experimenter influence how the subjects perform
  81. What are the cues of the experimental situation?
    Seemingly trivial things associated with intervention that may contrubte to the resuls (i.e. info given to subjects prior to their arrival of the experiment
  82. What is statistical conclusion validity?
    How well can an investigation detect the effect, and how large is it?
  83. What is a null hypothesis?
    There is no statistical difference between the groups; we can reject the NH if we find a statisticaly significant difference, or accept this hypothesis if we do not
  84. What is a Type I Error?
    The probability of rejecting the NH when it is true--being too liberal in interpretation
  85. What is a Type II Error?
    The probability of accepting the NH when it is false--being too conservative in interpretation
  86. What is power?
    The probability of finding a real effect: 1 minus Type II Error
  87. What is low statistical power?
    • When power is weak
    • Likelihood of concluding that there are no differences between groups
  88. What is subject heterogeneity?
    • The greater the heterogeneity of subject characteristics, the less likelihood of detecting a difference between conditions
    • Either homogenize or control statistically using multivariate regression and correlation methods
  89. What are unreliable measures?
    Reliability puts the ceiling on the effect size
  90. What are multiple comparisons and error rates?
    The more statistical tests perfomed, the more likely a chance difference will be found even if there are no true differences between conditions
  91. When can interferences in a study be addressed best?
    During planning
  92. An increase in sensitivity of a test can result in...?
    Limited generality of findings
  93. Enhancement of generality of a test can result in...
    Decrease in the sensitivity of the test
  94. What is an artifact/bias?
    Extraneous influences that may threaten the validity of experiments
  95. What is the Loose Protocol Effect?
    When the investigator does not carefully specify the rationale, the scrip, and activities of the experimenter = introduces variability
  96. What are experimenter expectancy effects?
    The experimenter may influence how the subject responds through tone of voice, posture, facial expressions, or other cues
  97. What are demand characteristics?
    Cues in the experimental situation that may influence how subjects respond
  98. What are subject roles?
    Different ways of responding to the experimental cues of the experiment
  99. The good subject
    attempting to support what they think is the experimenter's hypothesis
  100. The negativistic subject
    Attempting to refute what they think is the experimenter's hypothesis
  101. The apprehensive subject
    being concerned that experimental performance will be used to evaluate their abilities, personal characteristics, or opportunities
  102. The faithful subject
    attempting to carefully follow the experimental instructions
  103. What are subject selection biases?
    the experimental influences that are attributable to the types of subjects who participated in the experiment
  104. What are samples of convenience?
    the selection and use of subjects merely because they're available, and not because they are appropriate for the study
  105. How can the loss of subjects affect virtually all aspects of experimental validity?
    • Changing random composition of the groups and group equivalence (internal validity)
    • Limiting the generality of findings to a special group who are persistent (external validity)
    • Raising the prospect that the treatment X special subject characteristics interaction accounts for the conclusions that the investigator would actually like to attribute to the intervenion (external and construct validity)
    • Reducing sample size and power (statistical conclusion validity)
  106. Where do research ideas usually come from?
    • experience
    • previous studies
    • theory
  107. What is correlation?
    An association between variables at a given point in time, no time sequence established
  108. What is a risk factor?
    a correlation in which a time sequence is established
  109. What is a cause?
    a change in one variable leads to a change at another variable
  110. What are moderators?
    Factors that influence the relationship between variables
  111. What are mediators?
    the mechanisms or processes through which a relationship works
  112. What is a theory?
    a conceptualization of the phenomenon of interest
  113. Why is theory good?
    • Brings order to areas where findings are diffuse
    • Explains causal relationships
    • Gives a manageable focus to research
    • Leads to a more effective application of the research to practical probems
    • Helps us understand a phenomenon
  114. What are operational definitions?
    Defining a construct based on the specific operations used in the experiment
  115. What are the advantages of multiple operations?
    Allows an investigator to place more confidence in capturing the construct of interest
  116. Name two experimenter variables
    • Independent
    • Dependant
    • IV --> DV
  117. Independent Variable
    one the researcher manipulates
  118. Dependent Variable
    one the researcher observes to see how it responds to the different IVs
  119. Name two observational variables?
    • Predictor
    • Criterion
    • PV --> CV
  120. Predictor variable
    • analogus to an IV
    • Manipulated by elements outside the researcher's control
  121. Criterion Variable
    • Analogus to DV
    • Observed changes theoretically interpreted to have been in response to changes in the predictor variabl
  122. Name 2 measurement variables
    • Latent
    • Observed
    • LV --> OV
  123. Latent Variable
    the construct of interest, typically not directly measureable
  124. Observed Variable
    the specific measures of the construct
  125. What are true experiments?
    • Random group assignment
    • control over the IV
    • control over internal validity
    • Strongest basis for drawing causal inferences
  126. What are true experiments conducted in the context of an intervention?
    Randomized Controlled Clinical Trials
  127. What is a quasi experiment?
    • Less control than true experiment
    • lack of random group assignment
    • can still yield valid causal inferences
  128. What is case control design?
    • the variable of interest is studied by selecting cases that vary in the characteristic or experience of interest
    • can provide important insights
  129. What is a cross-sectional study?
    • variable of interest is studied by selecting cases that vary in characteristics or experience of interest
    • make comparisons between groups at a given point in time
  130. What are longitudinal studies?
    Make comparisons over an extended period
  131. What is random selection?
    • There is an equal probability that subjects within the population can be selected
    • Enhances the generality of the results
  132. What is random assignment?
    The probability of each subject appearing in any of the groups is equal
  133. What are nuisance variables?
    • Characteristics in which the researcher is not interest but could influence the results
    • random assignment make this less of the problem
  134. What is matching?
    grouping subjects together based on their similarity on a particular characteristic
  135. Matching without randomizing leads to...
    differential regression to the mean
  136. What is a pretest - posttest design
    • Widely used
    • controls threats to internal validity
    • brings both the advantages and disadvantages of using a pretest
  137. What are the advantages of using a pretest?
    • Allows the matching of subj. on one of the variables assessed
    • Permits evaluation of matched variable in the results
    • Increases statistical power
    • Allows the examination of who changed and what proportion of subjects changed in a particular way
    • Allows evaluation of attrition
  138. What are the disadvantages of a pretest?
    There might be a testing effect
  139. What is a posttest only design?
    • No pretest is given
    • Less popular in clinical research because of smaller experimental groups
  140. What is a Solomon Four group design?
    Gets at whether a pretest influences the results, but takes twice the effort and cost
  141. What are factorial designs?
    • Allows the simulataneous investigation of two or more IVs in a single experiment
    • Two or more conditions are administered
  142. What are the characteristics of Factorial Design?
    • Allows the analysis of interactions (where the effect of one of the IVs or PVs depends on the level of one of the other IVs or PVs)
    • Good because you can assess the effects of separate variables in a single experiment,
    • Different variables can be studied with fewer subjects and observations
    • Can look at interaction effects
    • Number of groups multiplies quickly
  143. What is the pretest-posttest control group design?
    • depends directly upon the similarity of the experimental and control groups
    • threats to validity differ between groups
  144. what are Posttest only design cons?
    attributing group differences to the intervention is problematic
  145. Multiple Treatment Design
    • Each of the different treatments is presented to each subject
    • Usually done with 2 groups in evaluating the effects of medication
Card Set
Research Methods
research methods