International evaluation of NSFC’s performance by evidence-based design

Abstract

Abstract: The Natural Science Fund of China (NSFC) has kicked off an international evaluation program since 2008 in order to assess its global funding and management performance establishing in 1986, which is faced with grave challenges. An evidence-based evaluation design approach is proposed to deal with the challenges for designing the overall international evaluation system of NSFC. The essentials of the international evaluation system are identified, appropriate evidences are searched to support them, and the evaluation essential-evidence matrix, which is applied to design the international evaluation system of NSFC, is constructed. The study reveals that the evidence-based evaluation design approach provides the sufficient support for designing the international evaluation system of NSFC, and could be widely used in other science and technology performance evaluation arenas.

Key words: NSFC, funding and management performance, international evaluation, evidence-based evaluation design

CLC Number:

N945.16

Wu Jiannan, Ma Liang, Zheng Yonghe. International evaluation of NSFC’s performance by evidence-based design[J]. Science Research Management, 2012, (6): 137-145.

References

[1] Chen Chen(陈晨). Chen Yi-yu: The Nation must ensure all disciplines of basic research being supported(陈宜瑜:国家必须保障所有学科的基础研究都能得到支持). Science Times(科学时报), 2008-6-16.
[2] Wu Jian-nan(吴建南), Ma Liang(马亮), Zheng Yong-he(郑永和). The Framework, Issues, and Methods of International Evaluation of National Science Funding Institutions: A Cross-national Comparative Multi-Case Study(科学基金国际评估的框架、内容与方法—基于多案例的跨国比较研究)[J]. Studies in Science of Science(科学学研究), 2010, 28(5): 704-712.
[3] Zhao Xue-wen(赵学文), Gong Xu(龚旭). Theory and Practice of Science Research Performance Evaluation(科学研究绩效评估的理论与实践)[M]. Beijing(北京): Higher Education Publisher(高等教育出版社), 2007.
[4] Wu Jian-nan(吴建南). International Practice of Performance Evaluation of S&T Program (Foundation): A Report from G-8 Research Assessment 2008 Workshop(科技计划(基金)绩效评估的国际实践—参加八国集团研究评估2008年会议的报告)[J]. Science Foundation in China(中国科学基金), 2009, 23 (4): 240-243.
[5] Han Jian-guo(韩建国), Chen Le-sheng(陈乐生), Zhu Dong-hua(朱东华). International Evaluation Practices of Science Foundation — Evaluation of Sino-German Center for Research Promotion(科学基金国际评估的实践—中德科学中心评估工作)[J]. Science Foundation in China(中国科学基金), 2009, 23 (3): 173-176.
[6] Zhang Jie(张杰). Thoughts on Mid and Long term International Evaluation of Colleagues and Departments(院系中长期国际评估的思考). Guangming Daily(光明日报), 2007-10-10.
[7] Zhang Xue-quan(张学全). The First International Evaluation of Institutes in China(中国科研所将首次进行国际评估). CCTV International(央视国际), 2004-1-15.
[8] Wu Jian-nan(吴建南). Public Sector Performance Evaluation: Theory and Practice(公共部门绩效评估:理论与实践)[J]. Science Foundation in China(中国科学基金), 2009, 23 (3): 149-154.
[9] Coplin WD, Merget AE, Bourdeaux C. The Professional Researcher as Change Agent in the Government Performance Movement[J]. Public Administration Review, 2002, 62 (6): 699-711.
[10] Wu Jian-nan(吴建南), Yan Bo(阎波). The Path Choices of Local Government Performance Evaluation System—An Analysis of Fujian Province(地方政府绩效评估体系的路径选择—福建的分析)[J]. Chinese Public Administration(中国行政管理), 2008, (2): 25-29.
[11] Forss K, Bandstein S. Evidence-based Evaluation of Development Cooperation: Possible? Feasible? Desirable?[J]. IDS Bulletin, 2008, 39 (1): 82-89.
[12] Solesbury W. Evidence Based Policy: Whence it Came and Where it's Going. London: Centre for Evidence Based Policy and Practice, 2001.
[13] Sackett DL, Rosenberg WMC, Gray JAM, et al. Evidence based medicine: what it is and what it isn't[J]. British Medical Journal, 1996, 312 (7023): 71-72.
[14] Sanderson I. Evaluation, Policy Learning and Evidence-Based Policy Making[J]. Public Administration, 2002, 80 (1): 1-22.
[15] Pfeffer J, Sutton RI. Evidence-Based Management[J]. Harvard Business Review, 2006, 84 (1): 63-74.
[16] Roessner D. Quantitative and qualitative methods and measures in the evaluation of research[J]. Research Evaluation, 2000, 9 (2): 125-132.
[17] Stufflebeam D. Program Evaluations Metaevaluation Checklist. Michigan: The Evaluation Center, Western Michigan University, 1999.
[18] Dewar B, Sharp C. Using evidence: how action learning can support individual and organisational learning through action research[J]. Educational Action Research, 2006, 14: 219-237.
[19] Hartley J, Benington J. Copy and Paste, or Graft and Transplant? Knowledge Sharing Through Inter-Organizational Networks[J]. Public Money & Management, 2006, 26 (2): 101-108.
[20] Fincham R, Clark T. Introduction: Can We Bridge the Rigour-Relevance Gap?[J]. Journal of Management Studies, 2009, 46 (3): 510-515.
[21] Neely A, Mills J, Platts K, et al. Performance measurement system design: developing and testing a process-based approach[J]. International Journal of Operations & Production Management, 2000, 20 (9/10): 1119-1145.
[22] Behn RD. Why Measure Performance? Different Purposes Require Different Measures[J]. Public Administration Review, 2003, 63 (5): 586-606.
[23] McDavid JC, Hawthorn LRL. Key Concepts and Issues in Program Evaluation and Performance Measurement. Program Evaluation and Performance Measurement: An Introduction to Practice. Sage Publications, Inc, 2006: 1-38.
[24] Arnold E, Balázs K. Methods in the evaluation of publicly funded basic research. London: Technopolis, 1998.
[25] Wu Jian-nan(吴建南), Ma Liang(马亮), Zheng Yong-he(郑永和). How do International Science Foundations Report Their Performance?: A Narrative Analysis on the Performance Report of Japan Society for the Promotion of Science(科学基金国际评估如何报告绩效:关于日本学术振兴会绩效报告的叙事分析)[J]. Science of Science and S&T Management(科学学与科学技术管理), 2009(12): 55-59.
[26] Nagarajan N, Vanheukelen M. Evaluating EU Expenditure Programmes: a Guide[M]. Luxembourg: Office for Official Publications of the European Communities, 1997.
[27] Horn J. A Checklist for Developing and Evaluating Evaluation Budgets. The Evaluation Center, Western Michigan University, 2001.
[28] Ma Liang(马亮), Yang Yu-qian(杨宇谦). Promoting International Cooperation and Enhancing NSFC Performance Evaluation—A Report on the International Conference of S&T Performance Management and Research Methods(加强国内外合作交流推进科学基金绩效评估—"科技绩效管理与研究方法国际会议"综述)[J]. Science Foundation in China(中国科学基金), 2009, 23 (6): 359-362.
[29] Nutley S, Walter I, Davies HTO. From Knowing to Doing: A Framework for Understanding the Evidence-into-Practice Agenda[J]. Evaluation, 2003, 9 (2): 125-148.