Due to the strong professionalism of future strategic research and general new technologies research, technology foresight is usually implemented based on Delphi method which is relying on expert evaluation, and combining with scenario analysis, literature measurement, patent analysis and other methods. The procedures of technology foresight based on Delphi method mainly include questionnaire design, candidate technology list selection, expert survey and questionnaire statistics etc. Technology foresight adopts two rounds of Delphi survey to evaluate and select the list of alternative technologies.
    Technology foresight is a time-consuming and labor-intensive decision-making tool. In order to improve the scientific nature and effectiveness of technology foresight, a systematic assessment is necessary after each round of technology foresight. In the past, the assessment research was mainly for the technologies that have reached the predicted realization time, and the research period is very long. For the continuous and rolling technology foresight activities, it is difficult to timely evaluate the results of the previous round of technology foresight in this way, and to propose optimization suggestions for the next round of technology foresight. Therefore, to timely evaluate the methods, data and results of the previous round of technology foresight, the feasible method is based on comparative analysis and cross-validation etc.
    In the expert survey process of Delphi, the consulting experts need to score the familiarity, the core of the technology, the driving force, the importance to the economy and society, and the predicted realization time etc. of the alternative technical items. The familiarity of the consulting experts is normally divided into four levels: "very familiar", "familiar", "less familiar" and "unfamiliar". For the statistical analysis of survey data, according to the familiarity of the consulting experts, different weights are assigned, and experts with higher familiarity will give bigger weight and the opinions of unfamiliar experts are ignored. Therefore, the self-assessed familiarity of expert is an important factor to the results of technology foresight.
    "Technology Foresight on China′s Engineering Science and Technology to 2035" is a technology foresight activity jointly organized by Chinese Academy of Engineering and National Natural Science Foundation of China for engineering technology in China. To evaluate the results of the technology foresight, optimize the following technology foresight activities, and support the strategic research of China, this paper uses the evaluating data of the consulting experts in " Technology Foresight on China′s Engineering Science and Technology to 2035", through statistical analysis, complex networks and significance testing etc. methods, the differences among the different familiarity consulting experts are analyzed, the influences of different familiarity experts in technology foresight based on Delphi are evaluated, and the number of the consulting expert, the ratios and weights of the different familiarity experts are optimized.
     First, the distribution analysis of different familiarity consulting experts in different fields of "Technology Foresight on China′s Engineering Science and Technology to 2035" is implemented. In each field of "Technology Foresight on China′s Engineering Science and Technology to 2035", the number of technologies, the number of participating experts, the number of questionnaires, and the number and proportion of "very familiar", "familiar" and "less familiar" experts in each field are counted, the distribution characteristics of different familiarity consulting experts in each field are analyzed.
    Secondly, based on the expert evaluation data of "Technology Foresight on China′s Engineering Science and Technology to 2035", the network relationship and opinion differences of different familiarity consulting experts are studied by using complex network and significance test methods. The complex network is applied to study the evaluation relationship between the different familiarity consulting experts and technologies, to find the main types of consulting experts, to analyze the network relationship between different familiarity consulting experts and the evaluated technologies, and to understand the behavioral differences of different familiarity consulting experts. For the five basic indicators of technology importance(technical core, driving force, importance to economic development, the role of promotion to social development, and the role of safeguarding national and national defense security), a significant difference test by using t-test was conducted to measure whether there were significant differences in the scores to the indicators among different familiarity experts in various fields. Comparing the opinions of different familiarity experts by the pair-wise method, the comparisons are divided into three groups: "very familiar" experts vs. "familiar" experts, "very familiar" experts vs. "less familiar" experts, "familiar" experts vs. "less familiar" experts.
    Thirdly, based on statistical analysis model and the selected key technologies, the influences of different familiarity experts in technical foresight are assessed. Based on statistical analysis and selection methods, according to the scores of different familiarity experts and all the three familiarity experts, the top 10, top 15 and top 20 technologies in each field are selected, and then are compared with the key technologies identified by panel experts in each field. The number of key technologies included in the top 10, top 15 and top 20 technologies are counted, and the impacts of different familiarity experts in technology foresight are evaluated.
    Finally, based on the evaluating data of the consulting experts of key technologies, the reasonable number of experts and the reasonable proportions and weights of the different familiarity experts are determined. Through the statistical analysis of the consulting data of key technologies, the 95% confidence interval of the number of experts and the ratios of different familiarity experts are determined. Meanwhile, the weights of different familiarity experts are optimized by Brute Force Grid Search method.
Through the research of this paper, we can find out:(1) The evaluation method based on complex network and significance test can effectively support the timely evaluation of different familiarity expert opinions in technology foresight.(2) The results verify that the self-assessed "very familiar" experts in Delphi survey process are relatively optimistic for the evaluation of technical items, and the "less familiar" experts are relatively conservative in the prediction of technical items. It is also found that, without the influence of the number of experts, the opinions of the "familiar" experts are relatively more accurate.(3) Finally, through statistical test and optimization algorithm, the number of consulting experts in the foresight technology, the ratios and weights of different familiarity consulting experts are optimized, which offers a reference for the subsequent technology foresight activities.