Courses:

Program Evaluation

Course Syllabus

CEU, Winter 2012

3 CEU credits (6 ECTS credits)

Lecturer: Gabor Kezdi

Room 412, Nador 11

Short outline. This 3-cerdit course is an introduction to the logic and methods of modern social program evaluation. Program evaluation derives from the idea that social programs should have demonstrable effects, and those effects should in some way outweigh the costs of the program. We concentrate on the most important and perhaps most difficult question: How to measure the effect of a program? This is an applied econometrics course, with equal emphasis on the applied and the econometrics part. We are going to cover some new econometric methods and go over many real-life applications (many to be presented by the students, see later). The main text is pure econometrics (Imbens and Wooldridge, 2009), and it will be supplemented by many journal articles, most of them applied.

Course level. The course is designed for MA students in Economics or Economic Policy who have successfully completed their core courses. The course is also open for MPP students who have demonstrated sufficient background in statistics and econometrics.

Prerequisites: Core Statistics and Econometrics courses in the Economics or the Economic Policy MA program. Please contact the lecturer if you have not completed the prerequisites but would like to join the course: permit to join the course is granted on a case-by-case basis.

The goals of the course. The main aim of this course is to provide the students with up-to-date tools of program evaluation. Students successfully completing this course should be able to understand an evaluation study in great detail and form a well-grounded judgment about its value. Complemented with appropriate technical background or help, they should be able to design an evaluation study best suited for the program and the practical constraints at hand.

The learning outcomes of the course. i) development of analytical skills and practical knowledge to a level where students can read critically any program evaluation research output (and evaluate its merit); ii) development of skills to a level where they can design and, with appropriate help, carry out such research projects; iii) development of technical skills which enable students to understand and most of the econometric tools used in program evaluations; iv) development of presentation skills with the aim of generating constructive discussion; v) development of social skills for critical but constructive discussion.

Detailed outline

(1) Intro. The policy context. Important concepts. Defining the object of interest. The identification problem. SUTVA.
Imbens and Wooldridge (2009).

(2) Randomized field experiments. Identification, examples and practical problems.
Imbens and Wooldridge (2009). PROGRESA in Mexico, the NSW in the U.S.

(3) Regression-discontinuity design.
Imbens and Wooldridge (2009), Hahn, Todd, Van der Klaauw (2001). Mandatory Job Search in the U.S., class size in Israel, unemployment benefits in Austria.

(4) Identification and estimation under unconfoundedness (ignorable treatment) 1. Regression, diff-in-diffs.
Imbens and Wooldridge (2009). LaLonde (1986). The NSW in the U.S.

(5) Identification and estimation under unconfoundedness (ignorable treatment) 2. Matching, propensity score methods.
Imbens and Wooldridge (2009). Dehejia and Wahba (2002). Teacher training in Israel.

(6) Combined methods.
Imbens and Wooldridge (2009). Familias en Accion in Colombia, Integrated education in Hungary.

(7) Equilibrium effects.
The Knowledge Lift of Sweden, The New Deal for the Young Unemployed in the U.K.

(8) Effect mechanisms.
Ham and LaLonde (1996), Lee (2008)

(9) Student presentations.

(10) Summary and review.

See the course website about the presentation schedule, problem sets, suggested presentation topics, and papers not included in the reader:

http://www.personal.ceu.hu/staff/Gabor_Kezdi/Program-Evaluation

Assessment

10% problem sets

30% presentation

10% classroom activity

50% exam

Students have to present a particular evaluation study. Student presentations can take 20 minutes at most (strictly enforced). They should answer the following questions.

What are the program’s goal, target group, intended effect and supposed mechanism?

What are the outcome variable, data and methodology used in the evaluation study? What is the result?

Evaluate the evaluation study. Do you believe the results? Was the evaluation study properly designed and the appropriate methods used under the given circumstances? Would you do something differently? What and how and why?

MAIN TEXT

Imbens, G. W. and J. M. Wooldridge (2009), “Recent Developments in the Econometrics of Program Evaluation.” Journal of Economic Literature, 47:1, 5–86

ADDITIONAL READINGS

general readings

Ashenfelter, Orley (1978): “Estimating the effect of training programs on earnings,” Review of Economics and Statistics, 6(1): 47-57.

LaLonde, Robert J. (1986): “Evaluating the econometric evaluations of training programs with experimental data.” American Economic Review, 76(4): 604-620.

Smith (2000) “A Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies.” Swiss Journal of Economics and Statistics. 136(3): 1-22.

Smith, Jeffrey. 2000. “Evaluating Active Labour Market Policies: Lessons from North America.”

Blundell, Richard and Monica Costa Dias (2002): “Alternative approaches to evaluation in empirical microeconomics,” cemmap Working Paper CWP10/02.

Heckman, James and Jeffrey Smith. 2004. “The Determinants of Participation in a Social Program: Evidence from a Prototypical Job Training Program.” Journal of Labor Economics. 22(4): 243-298.

Deaton (2009): Instruments of Development. Randomization in the tropics, and the search for the elusive keys to economic development

randomized experiments