Abstract: The Comprehensive Innovation Reform Experiment (hereinafter referred to as the Experiment) is a key measure to implement the National Innovation Development Strategy at regional level, and an innovation policy shift with Chinese characteristics which representing the future trends of innovation policies. Conducting proper policy evaluation of the Experiment is of great significance for improving policy-making and implementation processes and guaranteeing that the Experiment is implemented as expected. However, due to policy system complexity and policy document accessibility issues, the Experiment is facing challenges like that theoretical research lagging behind policy practices and policy practices lacking support from theoretical research. Besides, traditional evaluation theories and practices which focus on pre-policy evaluation and post-policy evaluation are unable to meet the demands of policy evaluation of the Experiment, which requires timely feedback to monitoring the reform process and its impact, and correct the experiment mechanism accordingly. Therefore, there is an urgent need of developing an evaluation method targeting this complex innovation policy system. Based on policy characteristics and evaluation requirement analysis of the Experiment, this article develops anindirect policy evaluation method by introducing the emerging big-data analysis tools. By collecting public opinion information on the Experiment from its stakeholders, mainly the media and the public, identifying the rules and characteristics of information communication and analyzing opinion focuses, this article comes up with fact-based evaluation evidences to decision makers. This paper collects 264,918 data sets from websites, Wechat, Weibo, forums, newspapers, blogs, videos and some other communication channels. By using data mining, network association analysis, clustering analysis and other methods, this paper systematically analyses hot topics of media and the public opinions related to the Experiment, from perspectives of time, space, content and other dimensions, to grasp the characteristics and problems of the Experiment.By means of big data analysis, this paper comes up with some interesting findings about the public opinions and its communication rules as well as policy implications about the Comprehensive Innovation Reform Experiment. Firstly, official media are the main force of information dissemination in the Experiment. In the early stage of theExperiment, the state-level news media paid close attention to it and played a key role in the information dissemination of the Experiment. Local media at all levels are also very active, especially official media in the eight pilot areas, actively reporting on the reform process and reform initiatives. However, in general, the form of policy information dissemination is relatively single, and many media are just forward relevant news and reports by means of ‘template’, lacking in-depth focused analysis. Moreover, with the advance of the reform experiment, the attention of the mainstream media declined and insufficient attention were paid to the Experiment.Secondly, there are great differences inpublic attention among 8 pilot regions. Beijing-Tianjin-Hebei region, especially Beijing, has a very high hotspot of online reports, which is closely related to its role as national political center and large amount of active netizens in Beijing. As to the data of social media platforms such as Weibo and Wechat, the participation of netizens in Sichuan is the highest. In contrast, other reform pilot regions pay less attention. Although Zhejiang and Jiangsu are not official experimental regions, due to their economic development status and active reform atmosphere, netizens in these two provinces also have a high degree of concern for the Experiment.Thirdly,public opinions focus on people’s livelihood policies and measures in the Experiment. The Experiment involves almost all aspects of innovation and development policies. Many reform initiatives, such as transforming government authorities and functions to the public, are directly related to the vital interests of the public. Social media platforms provide channels for timely feedback of public opinions and suggestions on the Experiment, and for the interaction between policy subjects and policy objects. The analysis found that netizens generally supported and affirmed the Experiment, hoped to have a deep understanding of the relevant measures through the Internet, and some netizens expressed their expectations of bringing better livelihood from reform.Fourthly, the public opinion hotspots fit well with the reform orientation of the eight pilot regions raised by the central government, which suggests that network communication plays a vital role in guiding public opinions. The central government documents put forward different positioning requirements for the eight pilot regions, such as the Beijing-Tianjin-Hebei region focusing on cross regional coordinated development, Shanghai focusing on the construction of a science, technology and innovation center with global influence, and Guangdong Province focusing on deepening innovation cooperation among Guangdong, Hong Kong and Macao. According to the results of large-scale data analysis, the relevant news, reports and media discussion contents of public opinion in eight regions are closely related to their reform orientation, accurately publicizing and reporting the key points of their Reform Experiments in various regions, and playing an active role in guiding public opinion.At last but not at least, this paper puts forwards some recommendations tostrengthen propaganda and public opinion guidance, and to expand the policy impact of the Experiments in the future based on big data analysis. First, the relevant departments should strengthen cooperation with the media, continue to pay attention to and report on the reform process, and expand the policy impact of comprehensive innovation. Secondly, it is suggested to organize experts and scholars to promote and analysis key reform policies and measures, so as to enhance the public awareness of the Reform and expand the audience and influence of pilot policies. Thirdly, it is suggested that we should make good use of self-Media channels, participate in discussions through various forms, timely publish the progress and trends of reform, collect the public’s reform needs and concerns, and make the reform experiment not only a government endeavor, but also widely involvement and participation of the public, so as to create a favorable atmosphere of public opinion for the reform.The big data analysis method, which has a large sample size and is not affected by the government officials and the questionnaire respondents, can represent the public’s views on the Experiment relatively independently and objectively. However, due to the influence of analysis methods, data disclosure channels and data quality, it is impossible to accurately identify the policy demands of the public and to evaluate the policy implementation status by this method alone. It can only reflect the characteristics of information dissemination and the focuses of public attention from a macro trend perspective. Considering the difficulty of access to official policy documents and direct investigation to the policy audiences, this method can help to indirectly understand the public’s opinions and concerns, and to help policy makers optimize and improve the Experiment to some extent.

Key words: big-data analysis, comprehensive innovation reform experiment, policy impact evaluation