Evaluation Research - University of Arizona：评价研究-亚利桑那大学.ppt

《Evaluation Research - University of Arizona：评价研究-亚利桑那大学.ppt》由会员分享，可在线阅读，更多相关《Evaluation Research - University of Arizona：评价研究-亚利桑那大学.ppt（42页珍藏版）》请在三一办公上搜索。

1、Evaluation Research,Kodi D.HavinsAED 615Fall 2006Dr.Franklin,Evaluation Research,Definitions:“Evaluation is the systematic assessment of the worth or merit of some object”“Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object”(Trochim,200

2、6),Evaluation Research,Basically is used to provide feedback on an event,organization,program,policy,technology,person,activity,etc.All of these are called“object(s)”It assesses information about the“object”Used heavily in social sciences and government agenciesUsually outcomes influence decision-ma

3、king process,Research Designs,4 Major Types:Survey ResearchCase StudyField ExperimentSecondary Data Analysis(Garson,n.d.),Procedures,4 Strategies:Scientific-Experimental ModelsManagement-Oriented Systems ModelsQualitative/Anthropological ModelsParticipant-Oriented Models(Trochim,2006),1.Scientific-E

4、xperimental Models,Take values and methods from the sciences(especially social sciences)Prioritize on the desirability of impartiality,accuracy,objectivity,and validity of the information generated(Trochim,2006),1.Scientific-Experimental Models,Examples:Tradition of experimental and quasi-experiment

5、al designsObjectives-based research that comes from educationEconometrically-oriented perspectives including cost-effectiveness and cost-benefit analysisRecent articulation of theory-driven evaluation(Trochim,2006),2.Management-Oriented Systems,2 of the most common:Program Evaluation and Review Tech

6、nique(PERT)Critical Path Method(CPM)(Trochim,2006),2.Management-Oriented Systems,Examples:Framework(“Logframe”)Model Used widely by U.S.Agency for International Development(Trochim,2006),3.Qualitative/Anthropological Models,Emphasis on:The importance of observationThe need to retain the phenomenolog

7、ical quality of the evaluation contextThe value of subjective human interpretation in the evaluation process(Trochim,2006),3.Qualitative/Anthropological Models,Examples:The approaches known in evaluation as naturalistic or“Fourth Generation”EvaluationThe various qualitative schoolsCritical Theory an

8、d art criticism approachesThe“Grounded Theory”approaches of Glaser and Strauss(Trochim,2006),4.Participant-Oriented Models,Emphasize the central importance of the evaluation participants(Trochim,2006),4.Participant-Oriented Models,Examples:Client-Centered Research Stakeholder ApproachesConsumer-Orie

9、nted Research(Trochim,2006),So,how do you decide which one to use?,Things to keep in mind:,Great evaluators use ideas from each of the 4 strategies as they need themMethodologies needed will and should be variedThis should not be an easy processThere is no simple answer!(Trochim,2006),Types of Evalu

10、ations:,There are many different types of evaluations,Types of Evaluations:,2 Important Types:Formative EvaluationSummative Evaluation(Trochim,2006),1.Formative Evaluation,Purpose:To strengthen or improve the object being evaluated(Trochim,2006)In other words,to provide useful feedback to for the gr

11、eater good of the“object”,1.Formative Evaluation,Includes several Evaluation TypesNeeds AssessmentEvaluablitiy AssessmentStructured ConceptualizationImplementation Evaluation Process Evaluation(Trochim,2006),1.Formative Evaluation,Questions generally asked:What is the definition and scope of the pro

12、blem or issue,or whats the question?Where is the problem and how big or serious is it?How should the program or technology be delivered to address the problem?How well is the program or technology delivered?(Trochim,2006),2.Summative Evaluation,Purpose:To examine the effects or outcomes of some obje

13、ct(Trochim,2006)In other words,to review the results/conclusions of an“object”,2.Summative Evaluation,Can be subdivided into different categories:Outcome EvaluationsImpact EvaluationCost-Effectiveness and Cost-Benefit Analysis Secondary Analysis Meta-Analysis(Trochim,2006),2.Summative Evaluation,Que

14、stions generally asked:What type of evaluation is feasible?What was the effectiveness of the program or technology?What is the net impact of the program?(Trochim,2006),Limitations,Limitations of this type of research include the level of expertise and knowledge of the expert panels If the evaluation

15、 is only as good as the knowledge of the expert panel,Journal Article,A Review of Subject Matter Topics Researched in Agricultural and Extension Education,By:Rama B.Radhakrishna&Wenwei Xu,A Review of Subject Matter Topics Researched in Agricultural and Extension Education,Retrieved from:Journal of A

16、griculture Education http:/pubs.aged.tamu.edu/jae/search/default.asp,Purpose of Study,The purpose of the study was to“examine subject matter topics researched in agricultural education over a ten year period”,Objectives,The objectives of the study were to:Identify subject matter topics researched in

17、 agricultural extension education in the last decade(1986 1996)Categorize subject matter topics published in the Journal of Agricultural Education and proceedings of the National Agricultural Education Research Meeting over a ten year period(1986-1996),How the Evaluation Method was used,2 sources we

18、re used to gain data Journal of Agricultural EducationPapers Presented at the National Agricultural Education Research meetings(NAERM)from 1986-1996,How the Evaluation Method was used,Resulted in 402 journal articles and 451 papers,How the Evaluation Method was used,Each paper was given a code numbe

19、rEach was reviewed and categorized into relevant subject matter categories3 Criteria were used:Title of StudyCentral Theme and FocusFindings and conclusions,How the Evaluation Method was used,25 subject-matter topics were identifiedThe list of the 25 topics was given to a panel of experts for review

20、 and validation,How the Evaluation Method was used,Panel of ExpertsAsked to comment on:The appropriateness of categoriesAdd or identify categories they thought that were left out or delete categories they felt did not fitSuggest whether some categories could be combined,How the Evaluation Method was

21、 used,Revised list of subject-matter categories contained 30 subject matter topicsData was then summarized using frequencies and percentages,Some of the Findings,Top 5 Studies:75 Secondary Agriculture Programs Studies70 Styles/Theory and Cognition Studies43 Professionalism Studies42 Extension Studie

22、s38 Agriculture Mechanics/Safety Studies,Conclusions of Study,“Study provided information on subject matter topics investigated by agricultural and extension educators,which in turn provides perspectives about the research efforts of the agricultural and extension education profession.”,Summary of E

23、valuation Research Method,The evaluation method was used in this study to assess what topics were researched in the agricultural education field during a 10 year period.,Summary of Evaluation Research Method,The outcomes of this study will provide useful feedback to those in the profession,References,Radhkrishna,R.B.&Xu,W.(1997).A review of subject matter topics researched in agricultural and extension education.Journal of Agricultural Education.38(3),59-69.Trochim,W.M.(2006)Research methods knowledge base.Drake University.Retrieved from:http:/on 10/26/06.,Any Questions?,Thank You,