Evaluating AI Literacy Scales in Higher Education: A Systematic Review of Patterns, Gaps, and Research Opportunities from a Socio-Technical Perspective
Keywords:
AI Literacy Scale, Higher Education, Socio-Technical TheoryAbstract
The widespread adoption of large language models (LLMs) in higher education has intensified the need to assess the extent to which AI literacy scales measure students’ readiness for an AI-driven learning environment. However, in the absence of a unified framework to guide construct selection, it remains unclear whether current instruments capture the full spectrum of AI competencies required for effective engagement. This study aims to examine the current literature on AI literacy scales to characterize their scope, identify prevailing constructs, and highlight gaps for future research. To achieve this, the review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology. The search strategy identified 39 AI literacy scales in peer-reviewed journal articles from ScienceDirect, SpringerLink, and IEEE Xplore databases, among other sources. A critical analysis of these scales revealed several gaps, spanning dimensionality, regional representation, assessment methods, theoretical grounding, and population diversity. The findings of this review underscore the importance of joint optimization in scale development to capture the human, technological, organizational, and contextual elements that shape AI literacy in higher education. This study makes two important contributions: (1) it provides a comprehensive account of the current composition of AI literacy scales, and (2) it identifies key gaps and emerging opportunities for refining future instruments.
References
Albikawi, Z. (2025). A Novel Scale to Measure Nursing Students ’ Fear of Artificial Intelligence : Development and Validation. The Open Nursing Journal, 1–11. https://doi.org/10.2174/0118744346376183250212033153
Almassaad, A., Alajlan, H., & Alebaikan, R. (2024). Student Perceptions of Generative Artificial Intelligence : Investigating Utilization, Benefits, and Challenges in Higher Education.
Almatrafi, O., Johri, A., & Lee, H. (2024). A systematic review of AI literacy conceptualization, constructs, and implementation and assessment efforts (2019–2023). Computers and Education Open, 6(January), 100173. https://doi.org/10.1016/j.caeo.2024.100173
Anders, B. (2023). The AI Literacy Imperative: Empowering Instructors & Students. Sovorel Publishing.
Barcelona, A., Rhoy, S., & Cruz, D. (2025). Development and validation of a scale measuring students ’ use of generative artificial intelligence tools. International Journal of Evaluation and Research in Education (IJERE), September. https://doi.org/10.11591/ijere.v14i5.34809
Biagini, G. (2024). Assessing the assessments: toward a multidimensional approach to AI literacy. Media Education, 15(1), 91–101. https://doi.org/10.36253/me-15831
Bond, M., Khosravi, H., De Laat, M., Bergdahl, N., Negrea, V., Oxley, E., Pham, P., Chong, S. W., & Siemens, G. (2024). A meta systematic review of artificial intelligence in higher education: a call for increased ethics, collaboration, and rigour. International Journal of Educational Technology in Higher Education, 21(1). https://doi.org/10.1186/s41239-023-00436-z
Bostrom, R. P., & Heinen, J. S. (1977). MIS Problems and Failures: A Socio-Technical Perspective, Part II: The Application of Socio-Technical Theory. MIS Quarterly, 1(4), 11. https://doi.org/10.2307/249019
Chai, C. S., Yu, D., & King, R. B. (2024). Development and Validation of the Artificial Intelligence Learning Intention Scale ( AILIS ) for University Students. SAGE Open, June, 1–16. https://doi.org/10.1177/21582440241242188
Chan, C. K. Y. (2025). Understanding AI guilt: the development, pilot-testing, and validation of an instrument for students. Education and Information Technologies. https://doi.org/10.1007/s10639-025-13629-y
Çobanoğullari, F., & Özbek, Ö. (2025). AI powered language learning: Developing the chatGPT usage scale for foreign language learners. Education and Information Technologies, 30, 12517–12534. https://doi.org/https://doi.org/10.1007/s10639-025-13342-w
DEC. (2024). Digital Education Council Global AI Student Survey 2024 AI or Not AI : What Students Want.
Dwivedi, Y. K., Kshetri, N., Hughes, L., Louise, E., Jeyaraj, A., Kumar, A., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Ahmad, M., Al-busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., … Wright, R. (2023). “ So what if ChatGPT wrote it ? ” Multidisciplinary perspectives on opportunities , challenges and implications of generative conversational AI for research , practice and policy ☆. International Journal of Information Management, 71(March). https://doi.org/10.1016/j.ijinfomgt.2023.102642
Emery, F. (1993). Characteristics of socio-technical systems. In The Social Engagement of Social Science, a Tavistock Anthology- The Socio-Technical Perspective (Vol. 2). University of Pennsylvania Press.
Gwagwa, A., Kazim, E., & Hilliard, A. (2022). The role of the African value of Ubuntu in global AI inclusion discourse : A normative ethics perspective. Patterns, 3(4), 100462. https://doi.org/10.1016/j.patter.2022.100462
Hornberger, M., Bewersdorff, A., Schiff, D. S., & Nerdel, C. (2025a). A multinational assessment of AI literacy among university students in Germany , the UK , and the US. Computers in Human Behavior: Artificial Humans, 4(October 2024), 100132. https://doi.org/10.1016/j.chbah.2025.100132
Hornberger, M., Bewersdorff, A., Schiff, D. S., & Nerdel, C. (2025b). Development and validation of a short AI literacy test ( AILIT-S ) for university students. Computers in Human Behavior: Artificial Humans, 5(June), 100176. https://doi.org/10.1016/j.chbah.2025.100176
Hwang, H. S., Zhu, L. C., & Cui, Q. (2023). Development and Validation of a Digital Literacy Scale in the Artificial Intelligence Era for College Students. KSII Transactions on Internet and Information Systems, 17(8), 2241–2259.
Jin, Y., Martinez-maldonado, R., Gašević, D., & Yan, L. (2025). GLAT : The generative AI literacy assessment test. Computers and Education: Artificial Intelligence, 9(June), 100436. https://doi.org/10.1016/j.caeai.2025.100436
Karaca, O., Çal, S. A., & Demir, K. (2021). Medical artificial intelligence readiness scale for medical students ( MAIRS-MS ) – development , validity and reliability study. BMC Medical Education, 1–9. https://doi.org/10.1186/s12909-021-02546-6
Koch, M. J., Carolus, A., Wienrich, C., & Latoschik, M. E. (2024). Meta AI literacy scale : Further validation and development of a short version. Heliyon, 10(21), e39686. https://doi.org/10.1016/j.heliyon.2024.e39686
Köhler, C., & Hartig, J. (2024). ChatGPT in higher education: Measurement instruments to assess student knowledge, usage, and attitude. Contemporary Educational Technology, 16(4), ep528. https://doi.org/10.30935/cedtech/15144
Kong, S. C., & Zhu, J. (2025). Developing and validating an artificial intelligence ethical awareness scale for secondary and university students : Cultivating ethical awareness through problem-solving with artificial intelligence tools. Computers and Education: Artificial Intelligence, 9(July), 100447. https://doi.org/10.1016/j.caeai.2025.100447
Kong, S. C., Zhu, J., & Yang, N. Y. (2025). Developing and validating a scale of empowerment in using artificial intelligence for problem-solving for senior secondary and university students. Computers and Education: Artificial Intelligence, 8(August 2024), 100359. https://doi.org/10.1016/j.caeai.2024.100359
Köse, N., Şimşek, E., & Can, M. (2025). Adaptation of Artificial Intelligence Attitude Scale ( AIAS-4 ) into Turkish : a validity and reliability study. Current Psychology, 44, 8096–8105. https://doi.org/10.1007/s12144-025-07418-6
Lintner, T. (2024). A systematic review of AI literacy scales. Npj Science of Learning, 9(1). https://doi.org/10.1038/s41539-024-00264-4
Liu, M., Jun, L., & Zhang, D. (2025). Enhancing student GAI literacy in digital multimodal composing through development and validation of a scale. Computers in Human Behavior, 166(December 2024), 108569. https://doi.org/10.1016/j.chb.2025.108569
Long, D., & Magerko, B. (2020). What is AI Literacy? Competencies and Design Considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16. https://doi.org/10.1145/3313831.3376727
Mahadewi, M. P., Aysya, A. A. A., Sofiyani, Z., & Fahmi, F. (2025). The Importance of Literacy on Artificial Intelligence for the Higher Education Students: A Systematic Literature Review. International Journal of Advances in Data and Information Systems, 6(1), 1–14. https://doi.org/10.59395/ijadis.v6i1.1350
Maina, A. M., & Kuria, J. (2024). Building an AI Future : Research and Policy Directions for Africa ’ s Higher Education. 2024 IST-Africa Conference (IST-Africa), 1–9. https://doi.org/10.23919/IST-Africa63983.2024.10569692
Makarius, E. E., Mukherjee, D., Fox, J. D., & Fox, A. K. (2020). Rising with the machines: A sociotechnical framework for bringing artificial intelligence into the organization. Journal of Business Research, 120(August), 262–273. https://doi.org/10.1016/j.jbusres.2020.07.045
Møgelvang, A., & Grassini, S. (2025). Validating the AI attitude scale ( AIAS-4 ) and exploring attitudinal differences in a large sample of Norwegian university students. Discover Education, 4. https://doi.org/10.1007/s44217-025-00657-6
Nemt-allah, M., Khalifa, W., Badawy, M., & Elbably, Y. (2024). Validating the ChatGPT Usage Scale : psychometric properties and factor structures among postgraduate students. BMC Psychology, 12(497). https://doi.org/10.1186/s40359-024-01983-4
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. https://doi.org/10.1016/j.caeai.2021.100041
Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-wilson, E., Mcdonald, S., … Prisma, M. (2021). PRISMA 2020 explanation and elaboration : updated guidance and exemplars for reporting systematic reviews. BMJ. https://doi.org/10.1136/bmj.n160
Rasul, T., Nair, S., Kalendra, D., Robin, M., & Oliveira, F. De. (2023). The role of ChatGPT in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning & Teaching, 6(1), 1–16.
Rupnik, D. (2025). Toward a Coherent AI Literacy Pathway in Technology Education : Bibliometric Synthesis and Cross-Sectional Assessment. Education Sciences. https://doi.org/https://doi.org/10.3390/educsci15111455
Sairitupa-sanchez, L. Z., Morales-garcía, S. B., & Morales-garcía, M. (2024). Development and validation of a scale for dependence on artificial intelligence in university students. Frontiers in Education, March, 1–7. https://doi.org/10.3389/feduc.2024.1323898
Santos, R., Villaceran, R., Rioflorido, J., & Paguiligan, D. (2024). Initial Development and Validation of a Questionnaire for Students ’ Artificial Intelligence Knowledge in Education. Cognizance Journal of Multidisciplinary Studies, 4(5), 32–41. https://doi.org/10.47760/cognizance.2024.v04i05.003
Sarkis-onofre, R., Catalá-lópez, F., Aromataris, E., & Lockwood, C. (2021). How to properly use the PRISMA Statement. Systematic Reviews, 13–15. https://doi.org/10.1186/s13643-021-01626-4.9.
Trist, E., & Bamforth, K. (1951a). The Evolution of socio-technical systems: a conceptual framework and action research program. In Conference on Organizational Design and Performance (Vol. 2, pp. 1–67). https://doi.org/0-7743-6286-3
Trist, E., & Bamforth, K. . (1951b). Some Social and Psychological Consequences of the Longwall Method of Coal-Getting. Human Relations, 4(1), 3–38.
Published
How to Cite
Issue
Section
Copyright (c) 2026 Esther Nderitu Imbamba, Paul Okanda, Gerald Chege
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
