Aggregation of Expert Ranking Opinions Using Multi-Criteria Decision-Making Methods

Dusko Tesic; Dragan Pamučar; Gülay Demir; Adis Puška; Darko Božanić

doi:10.31181/sor202780

Authors

Dusko Tesic Military Academy, University of Defence in Belgrade, Belgrade, Serbia Author https://orcid.org/0000-0002-5277-3270
Dragan Pamučar Department of Applied Mathematical Science, College of Science and Technology, Korea University, Sejong 30019, Republic of Korea Author https://orcid.org/0009-0004-2488-9403
Gülay Demir High School of Health Vocational Services, Sivas Cumhuriyet University, Sivas, Türkiye Author https://orcid.org/0000-0002-3916-7639
Adis Puška Department of Public Safety, Government of the Brčko District of Bosnia and Herzegovina, Brčko, Bosnia and Herzegovina Author https://orcid.org/0000-0003-3274-0188
Darko Božanić Military Academy, University of Defence in Belgrade, Belgrade, Serbia Author https://orcid.org/0000-0002-9657-0889

DOI:

https://doi.org/10.31181/sor202780

Keywords:

MCDM, Group Decision-Making, Aggregation, Expert, MARCOS, COPRAS, ARAS, CoCoSo, EWAA, HWAA, OWA, OWG

Abstract

This paper investigates the application of multi-criteria decision-making methods (MCDM) for the aggregation of expert opinions related to ranking, considering the weight coefficients of the experts' competencies. Measurement Alternatives and Ranking according to Compromise Solution (MARCOS), Complex Proportional Assessment (COPRAS), Additive Ratio Assessment (ARAS), and Combined Compromise Solution (CoCoSo) methods, which are traditionally used to select the optimal alternative in MCDM, are applied in this study for the purpose of aggregating the ranks given by experts. The validity of the results was confirmed by comparison with the Einstein weighted arithmetic average (EWAA), Hamacher weighted arithmetic aggregation (HWAA), Ordered Weighted Averaging (OWA), and Ordered Weighted Geometric (OWG) operators, where identical ranks were obtained. The conducted sensitivity analysis based on a Monte Carlo simulation confirms that classical MCDM methods provide stable and reliable results under conditions of perturbations of expert weights and ranks, while aggregation operators show significantly lower robustness and consistency. The aforementioned shows that MCDM methods can be a reliable and practical tool for group decision-making, i.e., the aggregation of expert opinions, with the advantage of generating final ranks without additional post-analysis.

Downloads

Download data is not yet available.

References

Ćosić, P. (2008). Dictionary of synonyms. Cornet.

Stevanović, M., & Jonke, L. (1967). Dictionary of the Serbo-Croatian literary language. Matica srpska.

Oxford Advanced Learner's Dictionary. (2026). Decision. https://www.oxfordlearnersdictionaries.com/definition/english/decision?q=decision

The Britannica Dictionary. (2026). Decision. https://www.britannica.com/dictionary/decision

Torres-Pomales, W. (2015). Considering risk and resilience in decision-making. Langley Research Center.

Yates, J. F. (2003). Decision management. Jossey-Bass.

Pavličić, D. (2007). Decision theory. Faculty of Economics, University of Belgrade.

Taherdoost, H., & Madanchian, M. (2023). Multi-criteria decision making (MCDM) methods and concepts. Encyclopedia, 3(1), 77–87. https://doi.org/10.3390/encyclopedia3010006

Peniwati, K. (2007). Criteria for evaluating group decision-making methods. Mathematical and Computer Modelling, 46(7–8), 935–947. https://doi.org/10.1016/j.mcm.2007.03.005

Tsyganok, V. V., Kadenko, S. V., & Andriichuk, O. V. (2012). Significance of expert competence consideration in group decision making using AHP. International Journal of Production Research, 50(17), 4785–4792. https://doi.org/10.1080/00207543.2012.657967

Tešić, D., & Božanić, D. (2024). Model for determining competences of experts in the field of military science. Vojno Delo, 76(1), 1–22. https://doi.org/10.5937/vojdelo2401001t

Pamucar, D., Deveci, M., Gokasar, I., Işık, M., & Zizovic, M. (2021). Circular economy concepts in urban mobility alternatives using integrated DIBR method and fuzzy Dombi CoCoSo model. Journal of Cleaner Production, 323, Article 129096. https://doi.org/10.1016/j.jclepro.2021.129096

Božanić, D., & Pamucar, D. (2023). Overview of the method defining interrelationships between ranked criteria II and its application in multi-criteria decision-making. In Lecture Notes in Electrical Engineering (Vol. 984, pp. 863–873). Springer Nature Singapore. https://doi.org/10.1007/978-981-19-8493-8_64

Pamučar, D., Stević, Ž., & Sremac, S. (2018). A new model for determining weight coefficients of criteria in MCDM models: Full consistency method (FUCOM). Symmetry, 10(9), Article 393. https://doi.org/10.3390/sym10090393

Emerson, P. (2013). The original Borda count and partial voting. Social Choice and Welfare, 40(2), 353–358. https://doi.org/10.1007/s00355-011-0603-9

Lin, S. (2010). Rank aggregation methods. WIREs Computational Statistics, 2(5), 555–570. https://doi.org/10.1002/wics.111

Tešić, D., Božanić, D., Radovanović, M., & Petrovski, A. (2023). Optimising assault boat selection for military operations: An application of the DIBR II-BM-CoCoSo MCDM model. Journal of Intelligent Management Decision, 2(4), 160–171. https://doi.org/10.56578/jimd020401

Deveci, M., Pamucar, D., Gokasar, I., Köppen, M., Gupta, B. B., & Daim, T. (2023). Evaluation of Metaverse traffic safety implementations using fuzzy Einstein based logarithmic methodology of additive weights and TOPSIS method. Technological Forecasting and Social Change, 194, Article 122681. https://doi.org/10.1016/j.techfore.2023.122681

Jin, Y., Wu, H., Merigó, J. M., & Peng, B. (2019). Generalized Hamacher aggregation operators for intuitionistic uncertain linguistic sets: Multiple attribute group decision making methods. Information, 10(6), Article 206. https://doi.org/10.3390/info10060206

Yanhong, L., Ambrin, R., Ibrar, M., & Ali Khan, M. (2022). Hamacher weighted aggregation operators based on picture cubic fuzzy sets and their application to group decision-making problems. Security and Communication Networks, 2022, Article 1651017. https://doi.org/10.1155/2022/1651017

Pamučar, D. (2025). Multiattributive decision making [Unpublished book manuscript].

Kreinovich, V. (2022). Ordered weighted averaging (OWA), decision making under uncertainty, and deep learning: How is this all related? Information, 13(2), Article 82. https://doi.org/10.3390/info13020082

Chiclana, F., Herrera, F., & Herrera-Viedma, E. (2002). The ordered weighted geometric operator: Properties and application in MCDM problems. In Studies in Fuzziness and Soft Computing (Vol. 141, pp. 173–183). Springer. https://doi.org/10.1007/978-3-7908-1796-6_14

Vatian, A., Gusarova, N., & Tomilov, I. (2025). Feature ranking on small samples: A Bayes-based approach. Entropy, 27(8), Article 773. https://doi.org/10.3390/e27080773

Köppel, M., Segner, A., Wagener, M., Pensel, L., Karwath, A., & Kramer, S. (2025). Pairwise learning to rank by neural networks revisited: Reconstruction, theoretical analysis and practical performance. Machine Learning, 114(4), Article 112. https://doi.org/10.1007/s10994-024-06644-6

Stević, Ž., Pamučar, D., Puška, A., & Chatterjee, P. (2020). Sustainable supplier selection in healthcare industries using a new MCDM method: Measurement of alternatives and ranking according to compromise solution (MARCOS). Computers & Industrial Engineering, 140, Article 106231. https://doi.org/10.1016/j.cie.2019.106231

Radovanović, M., Jovčić, S., Petrovski, A., & Cirkin, E. (2025). Evaluation of university professors using the spherical fuzzy AHP and grey MARCOS multi-criteria decision-making model: A case study. Spectrum of Decision Making and Applications, 2(1), 197-217. https://doi.org/10.31181/sdmap21202518

Zavadskas, E., Kaklauskas, A., & Šarka, V. (1994). The new method of multicriteria complex proportional assessment of projects. Technological and Economic Development of Economy, 1(3), 131–139.

Biswas, A., Gazi, K. H., Bhaduri, P., & Mondal, S. P. (2024). Neutrosophic fuzzy decision-making framework for site selection. Journal of Decision Analytics and Intelligent Computing, 4(1), 187–215. https://doi.org/10.31181/jdaic10004122024b

Zavadskas, E. K., & Turskis, Z. (2010). A new additive ratio assessment (ARAS) method in multicriteria decision-making. Technological and Economic Development of Economy, 16(2), 159–172. https://doi.org/10.3846/tede.2010.10

Saha, A., & Chatterjee, P. (2025). A Fermatean fuzzy decision-making model for manufacturing outsourcing vendor selection: An improved combined compromise solution method. Spectrum of Mechanical Engineering and Operational Research, 2(1), 231-247. https://doi.org/10.31181/smeor21202544

Yazdani, M., Zarate, P., Zavadskas, E. K., & Turskis, Z. (2019). A combined compromise solution (CoCoSo) method for multi-criteria decision-making problems. Management Decision, 57(9), 2501–2519. https://doi.org/10.1108/MD-05-2017-0458

Mukherjee, A. K., Gazi, K. H., Raisa, N., Momena, A. F., Mukherjee, S. B., Sobczak, A., Salahshour, S., Mondal, S. P., & Ghosh, A. (2025). Review of alternative ranking methods in multi-criteria decision analysis based on WASPAS and CoCoSo methodologies. Yugoslav Journal of Operations Research. Advance online publication. https://doi.org/10.2298/YJOR241215037M

Tešić, D. (2025). A machine learning and multi-criteria decision-making framework for student grade prediction. *Smart Multi-Criteria Analytics and Reasoning Technologies, 1*(1), 22-32. https://doi.org/10.65069/smart1120253

Kaya, A., Gurler, H. E., Karyağdı, N. G., Özçalıcı, M., Akpınar, Y., Koca, N., & Pamucar, D. (2025). Decision-analytics-based enhancing auditor performance evaluation: Application of multi-criteria methodology with Monte Carlo simulation. Expert Systems with Applications, 291, Article 128561. https://doi.org/10.1016/j.eswa.2025.128561