Enhancing Decision Accuracy Through Human-in-the-Loop Decision Support Systems: A Human-AI Collaborative Framework
Keywords:
Human-in-the-Loop, Decision Support System, Decision Accuracy, Human-AI Collaboration, Artificial IntelligenceAbstract
The rapid advancement of Artificial Intelligence (AI) has accelerated the adoption of AI-powered Decision Support Systems (DSS) across healthcare, finance, manufacturing, government, and education. Although AI-based DSS enhance decision-making through efficient data analysis and predictive recommendations, fully automated systems remain susceptible to hallucinations, algorithmic bias, misclassification, and limited contextual understanding, which may reduce decision accuracy in high-risk situations. Consequently, Human-in-the-Loop (HITL) has emerged as an important approach for incorporating human oversight into AI-assisted decision-making. However, empirical evidence regarding the effectiveness of HITL in improving decision accuracy remains limited. Therefore, this study investigates the influence of Human-in-the-Loop Decision Support Systems on decision accuracy. A mixed-methods research design was employed, combining quantitative and qualitative approaches. Participants were users performing analytical decision-making tasks using Decision Support Systems. An experimental framework compared AI-only and collaborative Human-AI decision-making using representative datasets. Data were analyzed through descriptive statistics, reliability and validity testing, regression analysis, and hypothesis testing. The results indicate that Human-in-the-Loop Decision Support Systems significantly improve decision accuracy by integrating human expertise with AI capabilities. The proposed Human-AI collaborative framework enhances transparency, reliability, and user trust while providing practical guidance for developing trustworthy Decision Support Systems that balance AI automation with meaningful human participation.
Downloads
References
Aithal, A., & Aithal, P. S. (2020). Development and validation of survey questionnaire & experimental data–a systematical review-based statistical approach. International Journal of Management, Technology, and Social Sciences (IJMTS), 5(2), 233–251.
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019). Beyond accuracy: The role of mental models in human-AI team performance. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 7, 2–11.
Behmann, F., & Wu, K. (2015). Collaborative internet of things (C-IoT): For future smart connected life and business. John Wiley & Sons.
Biswas, S., Corti, L., Buijsman, S., & Yang, J. (2022). Chime: Causal human-in-the-loop model explanations. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 10, 27–39.
Bonczek, R. H., Holsapple, C. W., & Whinston, A. B. (2014). Foundations of decision support systems. Academic Press.
Chiclana, F., García, J. M. T., del Moral, M. J., & Herrera-Viedma, E. (2013). A statistical comparative study of different similarity measures of consensus in group decision making. Information Sciences, 221, 110–123.
Cui, Y., Koppol, P., Admoni, H., Niekum, S., Simmons, R., Steinfeld, A., & Fitzgerald, T. (2021). Understanding the relationship between interactions and outcomes in human-in-the-loop machine learning. International Joint Conference on Artificial Intelligence.
Du, W., Kim, Z. M., Raheja, V., Kumar, D., & Kang, D. (2022). Read, revise, repeat: A system demonstration for human-in-the-loop iterative text revision. Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2writing 2022), 96–108.
Duan, Y., Edwards, J. S., & Dwivedi, Y. K. (2019). Artificial intelligence for decision making in the era of Big Data–evolution, challenges and research agenda. International Journal of Information Management, 48, 63–71.
González-Cutre, D., Sicilia, Á., Sierra, A. C., Ferriz, R., & Hagger, M. S. (2016). Understanding the need for novelty from the perspective of self-determination theory. Personality and Individual Differences, 102, 159–169.
Hagos, D. H., & Rawat, D. B. (2022). Recent advances in artificial intelligence and tactical autonomy: Current status, challenges, and perspectives. Sensors, 22(24), 9916.
Hamrouni, B., Bourouis, A., Korichi, A., & Brahmi, M. (2021). Explainable ontology-based intelligent decision support system for business model design and sustainability. Sustainability, 13(17), 9819.
Hebbar, K. S. (2022). Machine learning-assisted service boundary detection for modularizing legacy systems. International Journal of Applied Engineering & Technology, 4(2), 401–414.
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018). Metrics for explainable AI: Challenges and prospects. ArXiv Preprint ArXiv:1812.04608.
Hwang, J., Lee, T., Lee, H., & Byun, S. (2022). A clinical decision support system for sleep staging tasks with explanations from artificial intelligence: user-centered design and evaluation study. Journal of Medical Internet Research, 24(1), e28659.
Jarrahi, M. H. (2018). Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making. Business Horizons, 61(4), 577–586.
Knop, M., Weber, S., Mueller, M., & Niehaves, B. (2022). Human factors and technological characteristics influencing the interaction of medical professionals with artificial intelligence–enabled clinical decision support systems: literature review. JMIR Human Factors, 9(1), e28639.
Lebovitz, S., Lifshitz-Assaf, H., & Levina, N. (2022). To engage or not to engage with AI for critical judgments: How professionals deal with opacity when using AI for medical diagnosis. Organization Science, 33(1), 126–148.
Lu, H., Ehwerhemuepha, L., & Rakovski, C. (2022). A comparative study on deep learning models for text classification of unstructured medical notes with various levels of class imbalance. BMC Medical Research Methodology, 22(1), 181.
Rahman, S., & Kandogan, E. (2022). Characterizing practices, limitations, and opportunities related to text information extraction workflows: A human-in-the-loop perspective. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 1–15.
Rastogi, C., Zhang, Y., Wei, D., Varshney, K. R., Dhurandhar, A., & Tomsett, R. (2022). Deciding fast and slow: The role of cognitive biases in ai-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 6(CSCW1), 1–22.
Schwab, K., & Davis, N. (2018). Shaping the future of the fourth industrial revolution: A guide to building a better world. Penguin UK.
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022). Towards a standard for identifying and managing bias in artificial intelligence. National Institute of Standards and Technology.
Shin, D. (2021). The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AI. International Journal of Human-Computer Studies, 146, 102551.
Tiron-Tudor, A., & Deliu, D. (2022). Reflections on the human-algorithm complex duality perspectives in the auditing process. Qualitative Research in Accounting & Management, 19(3), 255–285.
Turker, D. (2009). Measuring corporate social responsibility: A scale development study. Journal of Business Ethics, 85(4), 411–427.
Wang, Z., & Srinivasan, R. S. (2017). A review of artificial intelligence based building energy use prediction: Contrasting the capabilities of single and ensemble prediction models. Renewable and Sustainable Energy Reviews, 75, 796–808.
Wiethof, C., & Bittner, E. A. C. (2021). Hybrid Intelligence–Combining the Human in the Loop with the Computer in the Loop: A Systematic Literature Review.
Xu, W., Dainoff, M. J., Ge, L., & Gao, Z. (2021). From human-computer interaction to human-AI Interaction: new challenges and opportunities for enabling human-centered AI. ArXiv Preprint ArXiv:2105.05424, 5(10.1080), 10442022–10447318.
Young, A., Clark, P., & Rahman, N. (2022). AI-Driven Automation: Enhancing Workflow Efficiency through Human-AI Collaboration and Smart Decision Frameworks.
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020). Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 295–305.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Muhazzim Muhazzim, Tahmaseb Tahmaseb, Atiyyatullah Atiyyatullah
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
