公正合理的科研项目立项评估与选择是国家自然科学基金管理活动的关键环节。综合考虑科研项目的评估指标体系和选择流程，提出了利用历史评估准确性度量专家提供信息可靠性的方法，进而提出了一种系统性的基于证据推理规则的科学基金项目评估决策模型。该模型使用证据推理合成规则对多专家多指标评估信息进行集结。在集结过程中：充分考虑评估指标的权重以及评估等级的多样性；鉴于参与项目评估的专家具有不同的知识背景和经验，提出利用历史评估结果的准确性衡量专家提供的评价信息可靠性的方法；应用信度分布表征项目的整体评价结果，包含了更丰富的信息。国家自然科学基金项目评估的实例分析证明了该评估决策模型的有效性。
Research project evaluation and selection is one of the core research management activities in the National Science Foundation of China (NSFC). Through taking into consideration evaluation criteria and selection procedure, a systematic method of implementing the evidential reasoning rule is presented to aggregate evaluation information on multiple criteria by multiple experts. The proposed method has the following strengths:1) combine evaluation information with different weights and reliabilities effectively, 2) employ a confusion matrix for generating experts' evaluation reliabilities, as experts may have different knowledge background and expertise, and 3) use a belief distribution to represent the overall performance profile of research projects in a more informative way. The belief distribution can be further transformed as utilities which are used to support the decision making of project selection. A case study on the project evaluation and selection of the National Science Foundation of China (NSFC) demonstrates the applicability and effectiveness of the proposed method.
[1] 陈媛,樊治平.基于研究领域匹配度的科研项目评审指派方法[J].中国管理科学,2011,19(2):169-173.
[2] 张洪涛,朱卫东,王慧,等.多维框架证据推理的科研项目立项评估方法[J].科研管理,2013,34(6):122-128.
[3] 杨列勋.R&D项目评估研究综述[J].管理工程学报,2002,16(2):60-65.
[4] Poh K L,Ang B W,Bai F.A comparative analysis of R&D project evaluation methods[J].R&D Management,2001,31(1):63-75.
[5] Amiri M P.Project selection for oil-fields development by using the AHP and fuzzy TOPSIS methods[J].Expert Systems with Applications,2010,37(9):6218-6224.
[6] Feng Bo,Ma Jian,Fan Zhiping.An integrated method for collaborative R&D project selection:Supporting innovative research teams[J].Expert Systems with Applications,2011,38(5):5532-5543.
[7] Jung U,Seo D W.An ANP approach for R&D project evaluation based on interdependencies between research objectives and evaluation criteria[J].Decision Support Systems,2010,49(3):335-342.
[8] 王凭慧.科学研究项目评估方法综述[J].科研管理,1999,20(3):18-24.
[9] 陈媛,樊治平.科技类评审中项目选择的两阶段综合集成方法[J].中国管理科学,2010,18(2):127-133.
[10] Khalili-Damghani K,Sadi-Nezhad S,Tavana M.Solving multi-period project selection problems with fuzzy goal programming based on TOPSIS and a fuzzy preference relation[J].Information Sciences,2013,252:42-61.
[11] Huang Chicheng,Chu Pinyu,Chiang Y H.A fuzzy AHP application in government-sponsored R&D project selection[J].Omega,2008,36(6):1038-1052.
[12] Heidenberger K,Stummer C.Research and development project selection and resource allocation:A review of quantitative modelling approaches[J].International Journal of Management Reviews,1999,1(2):197-224.
[13] Thamhain H J.Assessing the effectiveness of quantitative and qualitative methods for R&D project proposal evaluations[J].Engineering Management Journal,2014,26(3):3-12.
[14] Silva T,Jian Ma,Chen Yang.Process analytics approach for R&D project selection[J].ACM Transactions on Management Information Systems (TMIS),2015,5(4):21.
[15] Yang Jianbo,Xu Dongling.Evidential reasoning rule for evidence combination[J].Artificial Intelligence,2013,205:1-29.
[16] Chen Yue,Chen Yuwang,Xu Xiaobin,et al.A data-driven approximate causal inference model using the evidential reasoning rule[J].Knowledge-Based Systems,2015,88:264-272.
[17] Yang Jianbo,Xu Dongling.On the evidential reasoning algorithm for multiple attribute decision analysis under uncertainty[J].IEEE Transactions on Systems,Man and Cybernetics,Part A:Systems and Humans,2002,32(3):289-304.
[18] 杜元伟,杨娜.大数据环境下双层分布式融合决策方法[J].中国管理科学,2016,24(5):127-138.
[19] Xu Dongling.An introduction and survey of the evidential reasoning approach for multiple criteria decision analysis[J].Annals of Operations Research,2012,195(1):163-187.
[20] Yang Jianbo,Xu Dongling.A study on generalising Bayesian inference to evidential reasoning[M].Belief Functions:Theory and applications.Switzertand:Springer International Publishing,2014:180-189.
[21] 朱卫东,张洪涛,张晨,等.基于两维语义的科学基金立项评估方法[J].系统工程理论与实践,2012,32(12):2697-2703.
[22] Shafer G.A mathematical theory of evidence[M].Princeton:Princeton University Press,1976:74-78.
[23] Yang Jianbo.Rule and utility based evidential reasoning approach for multiattribute decision analysis under uncertainties[J].European Journal of Operational Research,2001,131(1):31-61.
[24] Provost F,Kohavi R.Guest editors' introduction:On applied research in machine learning[J].Machine Learning,1998,30(2):127-132.
[25] 陈晓田.国家自然科学基金与我国管理科学(1986-2008)[M].北京:科学出版社,2009.
[26] 王雄,吴庆田.基于模糊语言的科研基金项目立项评估研究[J].科技进步与对策,2007,24(9):61-63.
[27] 肖人毅,王长锐.科研基金项目立项评估方法的研究与改进[J].系统工程理论与实践,2004,5:66-71.
[28] 王东鹏,朱卫东,陈波,等.认知视角的科学基金项目专家评议意见分析-基于ER规则的研究[J].科学学与科学技术管理,2015,36(04):22-35.
