Science Research Management

Previous Articles     Next Articles

 A research on evaluation of disciplines in universities based on ESI data

 Zhang Wanshu1, Wang Xuefeng1, Yu Xuan2   

  1.  1. School of Management and Economics, Beijing Institute of Technology, Beijing 100081, China;
     2. Planning Bureau, National Natural Science Foundation of China, Beijing 100085, China
  • Online:2020-02-20 Published:2020-03-10
  • Supported by:
     

Abstract:  Discipline construction is an effective way to improve the academic level of universities. Essential Science Indicator (ESI) ranking has become one of the most important evaluation indicators in the world, which widely be used to evaluate the international academic level and influence of universities. ESI ranking also be widely used in China, however, there is a tendency of "quantity only" in discipline evaluation in Chinese universities at present and ESI ranking. It should be pointed out that ESI Category combine a wide range of other sub categories, but the ESI ranking cannot represent the same raking in these sub categories. Most scholars based on the the amount of articles, total article times cited and other amount indicators in discipline evaluation, whose suggestions for discipline construction in universities stay at a macro level. Also the same amount of the same discipline in different universities or regions does not mean the same development status of these universities or regions. The development of its sub discipline may be different. These kind of evaluation method may obscure the key problems in the development of discipline construction in universities. Based on the former analysis, a new discipline evaluation method is conducted, through the perspective of the composition of disciplines, hoping to provide suggestions for discipline construction in universities and to explore a new perspective of discipline evaluation.

In order to connect the relationship between the discipline and its sub disciplines, firstly, establish the correspondence between ESI category and Web of Science Category through the correspondence between ESI category and literatures, and between literatures and Web of Science Category. Web of Science Category have been seen as the sub discipline of ESI in case study.Referring to the summary of Stirling′s article to diversity, diversity can be evaluated by two dimensions for variety and balance. Diversity can represent the development of a system. Also, discipline diversity can represent the development of a discipline: a well-developed discipline first needs to develop in various sub disciplines, and at the same time, these sub disciplines need to develop well at the same time. While discipline variety and discipline balance can respectively evaluate the development of discipline.The coverage fraction of discipline in sub discipline and the one substation Gini index have been chosen to respectively measure the discipline variety and discipline balance.
In the case study, C9 League in China, Excellence 9 League in China and the top three universities in ESI ranking in January 2019 amounted 20 universities as research object. 13,676 ESI high cited papers from 20 universities which are derived from 1,433 journals were obtained to evaluate the development status of 22 ESI categories. The high cited papers collection time is in January 2019 and collected from Web of Science database. The journal list provided by ESI database is used to establish the correspondence between ESI category and Web of Science Category. Discipline variety and discipline balance is computed by the number of high cited papers in Web of Science Categories to evaluate the development of ESI categories in 20 universities. Furthermore, to combine two indicators of discipline diversity, three typical disciplines are evaluated in depth through Skyline Theory. Chemistry and Materials Science are the advantage disciplines in Chinese universities at present, while Computer Science is the basic discipline needed for the developing of science and technology facing the information age and big data age. Therefore, these three disciplines have been chosen as the typical disciplines. Skyline Theory is used to combine the two dimension of variety and balance for discipline diversity. The performance of two indicators in twenty universities is showed in the skyline figure and the skyline dot set representing the best performance in typical discipline is also showed. Also the impact of the amount of high cited papers to the two indicators is considered. Finally, in order to further explore the distribution of Web of Science Category in three typical disciplines and verify the rationality of the evaluation of discipline diversity and discipline balance, the cumulative figure of top ten advantage Web of Science Category in typical disciplines have been drawn.
The research shows that, firstly, the differences between discipline variety and discipline balance is showed in different ESI categories in twenty universities, the twenty universities shows better performance in Chemistry, Materials Science in discipline variety and discipline balance, and little differences between universities. The above could be regarded as advantage disciplines in China; In Social Science, Economics and Business, twenty universities show poor performance and little difference in discipline variety, but great differences in discipline balance. The above subjects can be regarded as weak disciplines in China. Secondly, among the three typical subjects, the development of Materials Science in twenty universities is relatively balanced, while there are differences in Chemistry and obvious differences in Computer Science. This conclusion can also be confirmed through the analysis of the cumulative figure of Web of Science Category. In general, a discipline evaluation method based on the perspective of discipline diversity is conducted by discipline variety and discipline balance. This evaluation method can be extended to the limited Web of Science Category range and specific Web of Science Category weight. Through case study, this method can effectively evaluate the status of discipline development in universities, help to discover the problems in discipline development in universities, provide new ideas for effective discipline evaluation, and provide decision support for building a balanced and coordinated discipline ecology. There are following deficiencies: firstly, the membership degree of Web of Science Category in different ESI Categories is ignored; Moreover, the author in different order has different contribution. And the impact of order of authorship in high cited paper is ignored; Then, only high cited papers is considered.The above three points may have influence on the conclusion and the supplement of theses factors needs to be further discussed in the follow-up research.

 

Key words:  discipline evaluation, discipline variety, discipline balance, Essential Science Indicator (ESI)

CLC Number: