-
How would you define evaluation
- a process of reflection whereby the value of certain actions in relation to projects programs or policies are assessed (did I meet objectives, did I accomplish goals, did I realize the mission)
- 2nd define: the comparison of an object of interest against a standard of acceptability. Follow the framework
-
why should it be done (evaluation)
to determine achievement of objectives, to improve program implementation, to provide accountability to stakeholders, to increase support for initiatives and inform policy decisions, to contribute to the scientific base
-
What is meant by a “standard of acceptability”?
Minimum levels of performance, effectiveness, or benefits used to judge the value. Think Gold Standard
-
What is Process evaluation
talking about us not participants. Measurements obtained during the implementation of program activities to control assure or improve the quality of performance or delivery
-
What is Formative evaluation
talking about us not participants. Any combination of measurements obtained and judgements made before or during the implementation of materials, methods, activities or programs to control, assure or improve the quality of performance or delivery
-
What is impact evaluation
immediate observable effects see in participants. Immediate observable effects of a program, leading to the intended outcomes of a program; intermediate outcomes
-
What is outcome evaluation
might not be able to see right after program ends (longterm goals). Ultimate goals or product of a program or treatment, measured by morbidity, mortality statistics in a population, vital measures, symptoms, signs, or physiological indicators
-
What is Summative evaluation
involves both impact and outcome. any combination of measurments obtained and judgments made that permit conclusions to be drawn about impact, outcome, or benefits of a program or method.
-
What are the six steps in the CDC’s evaluation framework
- Engaging Stakeholders
- describing the program
- focusing evaluation
- gathering credible data
- justifying the conclusion
- ensuring use and sharing lessons learned
-
What happens at step 1 in the CDC evaluation framework (engaging stakeholders)
Who: those involved in program operations, those served or affected by the program, the primary users of the evaluation results. Involve them in evaluation efforts, you need to understand their expectations, they decide your fate
-
What happens at step 2 in the CDC's evaluation framework (describing the program)
sets the frame of reference for all subsequent decisions in the evaluation process, should at least describe mission goals, and objectives, good use logic model
-
What happens at step 3 in the CDC's evaluation framework (Focusing evaluation design)
make sure interests of stakeholders are addressed, using time and resources efficiently
-
What happens at step 4 in the CDC's evaluation framework (Gathering credible data)
decide on measurment indicators (sources of evidence, quality and quantity of evidence, logistics of collecting evidence)
-
What happens at step 5 in the CDC's evaluation framework (Justifying the conclusion)
comparing the evidence against the standards of acceptability, judging the worth, merit or significance of the program, creating recommendation for actions based upon results
-
What happens in step 6 of the CDC's evaluation framework (Ensuring use and sharing lessons learned)
use and dissemination of the results, address the needs of each group of stakeholders
-
What are the four standards of evaluation
- Utility
- feasibility
- propriety
- accuracy
-
What does utility mean in the four standards of evaluation
the information needs of evaluation users are satisfied
-
What does feasibility mean in the four standards of evaluation
the evaluation is viable and pragmatic
-
What does propriety mean in the four standards of evaluation
the evaluation is ethical
-
What does accuracy mean in the four standards of evaluation
the evaluation produces findings that are considered correct (valid and reliable)
-
what are the advantages of hiring an internal evaluator
more familiar with organization and program, knows decision making style of organization, present to remind people of results, able to communicate results more frequently and clearly.
-
What are the disadvantages of hiring an internal evaluator
bias and/or conflict of interest
-
What are the advantages of hiring an external evaluator
more objective; fresh outlook, can ensure unbiased evaluation outcome, brings global knowledge, typically brings more breath and depth of technical expertise
-
What are the disadvantages of hiring an external evaluator
expensive, may lack knowledge of the program itself
-
What are the major differences between evaluation and research
-
What are the typical items included in a program budget
- Personnel: right people are key (internal external or both)
- Curriculum and other instructional resources: in house, outside sources, canned program or combo
- Space: where program held
- Equipment and supplies:
- Financial resources
-
What are the different types of financial resources
- Hard money
- soft money (Third party support, in-kind support, cost sharing/matching, gift, grant)
-
What is hard money
ongoing support (profit, sales fees)
-
What is soft money
doesn’t last forever. Third party support, in-kind support, cost sharing/matching, grants and gifts
-
What is third party support in reference to soft money
the employee gets services from Gold Gym but it is the employer who is picking up the costs.
-
What is In-Kind support in reference to soft money
volunteers, donated goods, organizational sponsorship, funds from another source that can be dedicated to your project (dean donated my time to project)
-
What is cost sharing/matching in reference to soft money
you share the cost of the funding of the program. You match the amount of funding you get
-
What is a gift in reference to soft money
given gratuitously voluntarily, donor does not control use, no direct benefit to the donor, no quid pro quo, no intellectual or copyright rights to donor
-
What is a grant in reference to soft money
accountability for spending, restrictions on spending, reporting requirements, defined obligations, start and end dates, direct benefit to donor, expected return of unused funds
-
What are direct costs
can be identified specifically with a particular project
-
What are indirect costs
- F&A, overhead costs, cost incurred for common or joint objectives that cannot be identified specifically with a particular project (rent, lights, heat, service contracts for copier fax machine computers) phone lines, accountant who works for agnecty and handles all accounts
- Most sponsors will cover indirect costs based on a percentage of direct costs
-
What do health educators “market
Information, ideas, goods (tangible items bike helmets), services, events, behaviors
-
What are the five functions of the marketing process
using market research to determine the needs and desires of present/prospective clients from priority population (covered in needs assessment)
developing a product that satisfies the needs and desires of the clients (covered in needs assessment)
developing informative and persuasive communication flows
Ensuring that the product is provided in an appropriate manner
Keeping the clients satisfied and loyal
-
What does developing informative and persuasive communication flows mean (as one of the five functions of the marketing process)
how will the planners get the message out, media habits, electronic print visual auditory
-
What does Ensuring that the product is provided in an appropriate manner mean (as one of the five functions of the marketing process)
(the 4 P’s product, price, place, promotion
-
What are the three keys to the marketing process
- understanding priority population
- knowing how/when to segment the priority population
- understand diffusion theory
-
What does understanding the priority population mean as one of the three keys to marketing process
How they see the world, how they spend their time, what is important to them
-
What does knowing how/when to segment the priority population mean as one of the three keys to the marketing process
Factors or variables- demographics, geographics, benefits sought, behavioral/lifestyle/psychograpics factors usually multiple factors used
-
What does understanding diffusion theory mean as one of the three keys to the marketing process
Diffusion of innovations is the process by which an innovation is communicated through channels over time among members of a social system
-
What is diffusion theory
People tend to follow the lead of others. Behavior is contagious, bring in Diffusion of innovation
-
What are the four categories of program adopters
Innovators, early adopters, early and late majority, laggards
-
What do the innovators do as one of the four categories of program adopters
they want to be first to do things, take chances
-
What do the early adopters do as one of the four categories of program adopters
- (change agents at take off) tend to be opinion leaders, seen as role models
- Early and late majority (wait for it to become a norm) early majority is greatly influenced by opinion leaders; once they adopt it the innovation is considered mainstream
-
What do the Laggards do as one of the four categories of program adopters
last to become involved, if at all suspicious of innovation limited communication networks
|
|