What are control groups?
- Used to address threats to internal validity (history, maturation, selection, and testing)
- Ensure that one group in the design shares these influences with the intervention group but does not receive the intervention
What is a 'no-treatment' control group?
- It answers the basic question of what would hapen without treatment
- Assessed but receives no treatment
- Significan ethical problems
What is a 'waiting list' control group?
- Delayed treatment and then treatment is provided
- Promised treatment within a specified length of time and are scheduled for appointments
What are the pros and cons of the 'waiting list' control group?
Pros: Subjects are more willing to participate; allows careful evaluation of the treatment effects at different points in time
Cons: Long-term impact of history, maturation, and repeated testng cannot be evaluated;
What is a 'non-specific' treatment control group?
- Separates the treatment from the placebo effect
- Subjects attend treatment sessions and have personal contact with the therapist
- Cons: sets up control group to not trust psychotherapy in the future
What is an alternative to a 'non-specific' control group?
Have a common therapy factor control group where they are given a non-specific treatment of empathic understanding, positive regard, and congruence while the treatment group gets this plus the treatment technique
What is the standard treatment design?
When a new treatment is compared with the standard one that is provided
What are the pros and cons of the standard treatment design?
Pros: Most ethical issues are taken care of; Attrition is decreased; Placebo effect is controlled for; Therapists are more interested in participating
Cons: Standard treatment often becomes sloppy and inconsistent so the comparison may not be fair
What is a yoked control group?
the subjects are matched across groups by pairing subjects and the experimenter ensures that both subjects are equal on potentially confounding variable, such as initial severity of symptoms
What is a non-equivalent control group?
Not part of the original subject pool or not randomly assigned, but can help rule out alternative explanations of the data
What does selection of a type of control group depend on?
- What the experimenter is trying to do
- Whether one is trying to strengthen internal validity, external validity, or construct validity
- How much research has accumulated in the area and what has been established
What is a treatment package strategy?
- Whether a particular treatment or treatment package (treatment may be multifaceted and include several different components)
- Treatment vs. no treatment or waitng list control
What is a dismantling treatment strategy?
- Analyzing the components of a given treatment package
- Individual components are elimiated or isolated from the treatment
- Some clients may receive the entire treatment package while other clients receive a package minus one or more components
- Two or more treatment groups that vary in components of treatment provided
What is an advantage of the dismantling treatment strategy?
Can help identify the necessary and sufficient components of treatment
What is a constructive treatment strategy?
- A treatment package that has added components to enhance outcome
- Explores whether the treatment effect can be enhanced
- Ex: adding verbal psychotherapy to pharmacotherapy
- Two or more treatment groups that vary in components of treatment provided
What is a parametric treatment study?
- Altering specific aspects of treatment to determine how to maximize therapeutic change, not adding anything new
- Ex: increases in duration of treatment or variations in how material is presented
- Two or more treatment groups that difer in one or more facets of the treatment
What is a comparative-treatment strategy?
- Which treatment is better for a particular clinical problem
- Ex: psychoanalysis v. behavior therapy, CBT v. medication, family therapy v. individual therapy
- 2 or more different treatments for a given clinical problem
What is a treatment-moderator strategy?
- The effectiveness of treatments can vary as a function of client characteristics, therapist characteristics, and context in which the client functions
- Statistically, when there is an interaction treatment status and another IV
- Ex: studying the effects of a given treatment depending upon diagnositc comorbitiy
- Treatment as applied seperately to different types of cases, such as gender, race/ethnicity, diagnosis, substance abuse
What is the treatment-mediatory strategy?
- A study of the mechanisms of change in therapy: Through what processes does theray lead to, or cause, change? ; What transpires between the delivery of an intervention and the ultimate impact on clients
- Treatment groups in which client and therapist interactions are evaluated within sessions
What is an observational study?
- The investigator evaluates the variables of interest by observing groups that "natre" has manipulated, rather than the researcher manipulating the variable
- Can be descriptive, exploratory, or test theoretical models
What are the statistics for observational studies?
Correlations, but can move to implied causality by using covariates/hierarchical analyses (controlling for demographics or other confounds) and structural equations modeling (specifying a causal model and seeing if it fits the data).
What are case control studies?
investigations with groups that vary in outcome or characteristic of interest
What is a cross-sectional case control study?
Evaluates a range of characteristics that are currently evident
What is a retrospective case control study?
Evaluating a range of characteristics that are currently evident
How case control studies helpful?
- They are helpful in identifying risk factors
- Efficient in time and resources
- No attrition
What is a disadvantage of case control studies?
Causality is more difficult to argue for because of no literal time gap
What is a cohort study?
A group of individuals are followed over time and are assessed before the outcome has occurred
What is a single group cohort design?
- A group that has not yet experienced the outcome of interest is assessed on multiple occasions and followed over time.
- Later, subgroups are delineated as those who do or don't show the outcome of interest
- Predictors can then be identified
What is an accelerated multicohort longitudinal design?
- cohorts of different ages begin the study and are followed over time, and the time to finish the study is shortened
- Establishes that the predictor occurred before the outcome; however can take time and money to follow subjects
- attriction is a problem
What is program evaluation?
Carefully collecting info about a program or some aspect of a program in order to make necessary decisions about the program
Name different types of program evaluations
- Needs assessment
- Cost/benefit analysis
What are the advantages of program evaluation?
- Understand, verify or increase the impact of products or services on customers
- Improve delivery mechanisms to be more efficient and less costly
- Verify that you're doing what you think you're doing
- Facilitate management's really thinking about what their program is all about (goals, how it meets goals)
- Produce data or verify results
- If cuts have to be made, produce valid comparisons between programs to decide which is retained
- Fully examine and describe effective programs for duplication elsewhere
What should an organization have?
- The board should be in good working order
- The organization should be staffed and organized to conduct activities to work toward the mission of the organization
- There should be no current crisis
What should a program have?
- A strong impression of what your customers or clients actually need
- Some effective methods to meet those needs
What are key components of programs?
What is an input?
the various resources needed to run the program
What is a process?
How the program is carried out
What is an output?
The units of service
What is an outcome?
The impacts on the customers or on clients receiving services
What is a goal-based evaluation?
The extent to which programs are meeting predetermined goals or objectives
What is a process-based evaluation?
To fully understand how a program works -- how does it produce the results that it produces
What are outcome-based evaluations?
Is the organization really doing the right program activities to bring about the outcomes you believe to be needed by your clients?
What are the 4 levels of evaluation info that can be gathered from clients?
- Reactions and feelings (feelings are often poor indicators that your service made lasting impact)
- Learning (enhanced attitudes, perceptions or knowledge)
- Changes in skills (applied the learning to enhance behaviors)
- Effectiveness (improved performance because of enhanced behavior)
What are some ethical factors that participants should be aware of?
- What you're doing with them in the evaluation and how any info associated with them will be reported
- Terms of confidentiality
- The right to participate or not
What is meta-analysis (MA)?
a method to integrate multiple empirical studies into a single study of studies.
What info does a MA yield?
A summary of statistics of interest: descriptives, correlations, group differences, proportions, odds ratios, etc.
MA benefits science because of:
- our natural human limitations of holding info
- Individual studies can be contradictory
- small effect sizes can be difficult for an individual study to detect because of power problems
- Bringing small studies together increases statistical power
What does a good MA start with?
- A good database search
- lots of reading
What details of the MA studies should be noted?
- Contructs and how they are operationalized
- Statistics of interest, usually effect size
- Traditional correlations
- N; effect sizes are weighted with larger N's
- Methodology rigor (random selections, comparison groups, random assignment, and measurement validity)
- Other potential moderating or mediating variables
What are the steps of MA data analysis?
- Check the hypothesis that the effect sizes are homogeneous
- If the effect sizes are heterogeneous, what are the moderating/mediating influences
- If effect sizes are heterogeneous and not moderating/mediating influences, randome variances must be used when calculating overall effect size
What are some disadvantages of non-experimental case studies?
- Basic threats to internal validity aren't ruled out
- Rely on clinical judgment and interpretation
- Subject to bias
- External validity is always an issue
Why is autocorrelation an issue?
- Raw scores are calculated, not the error
- Least square statistics (MRC) appropriate
Describe a single case experiment?
- Instruments must have high reliability and validity
- Observations are collected continuously (e.g. OQ)
Describe a simple single case design
- There are 3 baseline periods of collecting observations before treatment
- 3 treatment periods of collecting observations
- If all of the treatment observations are below the baseline = statistically significant and p > .05
Disadvantage of simple single case design
- This doesn't work when there is a trend in the data or a slope
- Works best with a stairstep
What is Moral Principle 1?
- Respect for persons and their autonomy
- Participation is a matter of their autonomous choice
- Persons have diminished autonomy, they require special concern
What is Moral Principle 2?
- Beneficence and nonmaleficence
- Researcher should maximize the possible benefits and minimize the possible harms from the research
What is Moral Principle 3?
- Distributive justice is the proper distribution of benefits and burdens; procedural justice is the adequacy of procedures to ensure fairness
What is Moral Principle 4?
- Participation is based on explicit agreement about what the participant will experience and its consequences, and the researcher's obligations
What is Moral Principle 5?
- Fidelity and scientific integrity
- The researcher is committed to the discovery and promulgation of truth; scientific integrity (truthfulness) is not open to compromise
When you're planning research what risks should be assessed?
- Physical risk (injury or illness)
- Psychological risk (boredom, depression, anxiety)
- Social risk (loss of important social relationships)
- Economic risk (loss of opportunity for job)
- Legal risk (arrest, data subpoenaed)
When does coercion exist?
- If individuals cannot refuse to participate in research
- If refusal causes a substantial loss
- If individuals believe that participation is not truly voluntary
In what populations does coercion readily occur?
- Military personnel
- Patients employees
What is informed consent?
A process that includes a clear statement of the purposes, procedures, risks, and benefits of the research project, as well as obligations and commitments of both the participants and the researchers
Consent should be voluntary, informed, ongoing throughout the study
What is privacy?
A person's right in controlling other people's access to info about him or herself
What is confidentiality?
The right to maintain private info divulged in the course of a professional relationship with the researcher. Has to do with agreements with the person about what may be done with their data
What is a requirement of research with deception?
Individuals must be debriefed
What are the benefits of debriefing?
Can ease the negative effects of dceoption
What are 3 means of evaluating clinical significance of change?
- Comparison Method
- Subjective Evaluation
- Social Impact
What is the criteria for eaclauating treatment procedures?
- Acceptability of Treatment
What is a case study?
- An intensive study of the individual, however could also be a family, group, institution, state, country, etc.
- The information has traditionally been in narrative style and is richly detailed.
- Efforts are made to convey the complexity of the case.
- The information is often retrospective, where past influences are used to account
- for the current situation.
What important contributions have anecdotal research made to psychological research?
- They have been a source of ideas and
- hypotheses about human performance and development.
- They have served as the source for developing therapy techniques.
- They permit the study of rare phenomena.
- They are valuable in providing a counter-instance for situations that are considered universal.
- They have persuasive and motivational value.
What are some limitations of the anecdotal case study?
- They rely heavily on clinical judgment and interpretation, and are often based on client reports, which can be distorted and highly selective.
- None of the basic threats to internal validity are usually ruled out.
- Difficult to know whether the results generalize to other individuals or situations.
What inferences are made in a single case experiment?
inferences are made about the effects of an intervention by comparing different conditions presented to the same subject over time.
What is continuous assessment?
observations on multiple occasions over time before and during the period in which the intervention is administered.
What is baseline assessment?
Assessment for period before the implementation of the intervention
What is the stability of the performance?
how variable that performance is over time
What is "use of different phases?"
periods of time in which a condition is implemented and data are collected
What is an ABAB design?
2 separate phases alternated over time including baseline condition when no intervention is in effect and the intervention condition' phases repeated again to complete the four phases
What are the cons of ABAB designs?
Problematic for clinical work, because withdrawing treatment does not always show a change in behavior and continued performance of appropriate behavior is desired
What is a multiple baseline design?
- Includes multiple behaviors, individuals and situations
- Better for clinical work because it demonstrates the effect of an intervention without a return to baseline condition
- intervention is applied in a gradual or sequential way across different situations
What is a changing criterion design?
with a baseline phase after which an intervention is introduced, and the specific level of performance is chosen as a criterion for the client; when the performance meets or surpasses the criterion level the client is rewarded; when the performance consistently meets the criterion, the criterion is made more stringent.