An Investigation into the Utility of Large Language Models in Geotechnical Education and Problem Solving

被引:8
作者
Chen, Liuxin [1 ]
Tophel, Amir [1 ]
Hettiyadura, Umidu [2 ]
Kodikara, Jayantha [1 ]
机构
[1] Monash Univ, Dept Civil Engn, ARC Ind Transformat Res Hub ITRH SPARC Hub, Clayton Campus, Clayton, Vic 3800, Australia
[2] Monash Univ, Dept Civil Engn, Clayton Campus, Clayton, Vic 3800, Australia
来源
GEOTECHNICS | 2024年 / 4卷 / 02期
基金
澳大利亚研究理事会;
关键词
data science applications in education; human-computer interface; technology-enhanced learning; cooperative/collaborative learning; CHATGPT; GPT-4;
D O I
10.3390/geotechnics4020026
中图分类号
P5 [地质学];
学科分类号
070403 [天体物理学];
摘要
The study explores the capabilities of large language models (LLMs), particularly GPT-4, in understanding and solving geotechnical problems, a specialised area that has not been extensively examined in previous research. Employing a question bank obtained from a commonly used textbook in geotechnical engineering, the research assesses GPT-4's performance across various topics and cognitive complexity levels, utilising different prompting strategies like zero-shot learning, chain-of-thought (CoT) prompting, and custom instructional prompting. The study reveals that while GPT-4 demonstrates significant potential in addressing fundamental geotechnical concepts and problems, its effectiveness varies with specific topics, the complexity of the task, and the prompting strategies employed. The paper categorises errors encountered by GPT-4 into conceptual, grounding, calculation, and model inherent deficiencies related to the interpretation of visual information. Custom instructional prompts, specifically tailored to address GPT-4's shortcomings, significantly enhance its performance. The study reveals that GPT-4 achieved an overall problem-solving accuracy of 67% with custom instructional prompting, significantly higher than the 28.9% with zero-shot learning and 34% with CoT. However, the study underscores the importance of human oversight in interpreting and verifying GPT-4's outputs, especially in complex, higher-order cognitive tasks. The findings contribute to understanding the potential and limitations of current LLMs in specialised educational fields, providing insights for educators and researchers in integrating AI tools like GPT-4 into their teaching and problem-solving approaches. The study advocates for a balanced integration of AI in education to enrich educational delivery and experience while emphasising the indispensable role of human expertise alongside technological advancements.
引用
收藏
页码:470 / 498
页数:29
相关论文
共 41 条
[1]
Performance of ChatGPT and GPT-4 on Neurosurgery Written Board Examinations [J].
Ali, Rohaid ;
Tang, Oliver Y. ;
Connolly, Ian D. ;
Sullivan, Patricia L. Zadnik ;
Shin, John H. ;
Fridley, Jared S. ;
Asaad, Wael F. ;
Cielo, Deus ;
Oyelese, Adetokunbo A. ;
Doberstein, Curtis E. ;
Gokaslan, Ziya L. ;
Telfeian, Albert E. .
NEUROSURGERY, 2023, 93 (06) :1353-1365
[2]
Evaluating the Performance of ChatGPT in Ophthalmology [J].
Antaki, Fares ;
Touma, Samir ;
Milad, Daniel ;
El -Khoury, Jonathan ;
Duval, Renaud .
OPHTHALMOLOGY SCIENCE, 2023, 3 (04)
[3]
Transforming Education: A Comprehensive Review of Generative Artificial Intelligence in Educational Settings through Bibliometric and Content Analysis [J].
Bahroun, Zied ;
Anane, Chiraz ;
Ahmed, Vian ;
Zacca, Andrew .
SUSTAINABILITY, 2023, 15 (17)
[4]
A Survey of Current Machine Learning Approaches to Student Free-Text Evaluation for Intelligent Tutoring [J].
Bai, Xiaoyu ;
Stede, Manfred .
INTERNATIONAL JOURNAL OF ARTIFICIAL INTELLIGENCE IN EDUCATION, 2023, 33 (04) :992-1030
[5]
The Power of Generative AI: A Review of Requirements, Models, Input-Output Formats, Evaluation Metrics, and Challenges [J].
Bandi, Ajay ;
Adapa, Pydi Venkata Satya Ramesh ;
Kuchi, Yudu Eswar Vinay Pratap Kumar .
FUTURE INTERNET, 2023, 15 (08)
[6]
Bewersdorff A, 2023, Computers and Education Artificial Intelligence, V5, P100177, DOI [10.1016/j.caeai.2023.100177, 10.1016/j.caeai.2023.100177, DOI 10.1016/J.CAEAI.2023.100177]
[7]
ChatGPT in medical imaging higher education [J].
Currie, G. ;
Singh, C. ;
Nelson, T. ;
Nabasenja, C. ;
Al-Hayek, Y. ;
Spuur, K. .
RADIOGRAPHY, 2023, 29 (04) :792-799
[8]
Daniel W.W., 2018, Biostatistics: A Foundation for Analysis in the Health Sciences
[9]
"So what if ChatGPT wrote it?" Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy [J].
Dwivedi, Yogesh K. ;
Kshetri, Nir ;
Hughes, Laurie ;
Slade, Emma Louise ;
Jeyaraj, Anand ;
Kar, Arpan Kumar ;
Baabdullah, Abdullah M. ;
Koohang, Alex ;
Raghavan, Vishnupriya ;
Ahuja, Manju ;
Albanna, Hanaa ;
Albashrawi, Mousa Ahmad ;
Al-Busaidi, Adil S. ;
Balakrishnan, Janarthanan ;
Barlette, Yves ;
Basu, Sriparna ;
Bose, Indranil ;
Brooks, Laurence ;
Buhalis, Dimitrios ;
Carter, Lemuria ;
Chowdhury, Soumyadeb ;
Crick, Tom ;
Cunningham, Scott W. ;
Davies, Gareth H. ;
Davison, Robert M. ;
De, Rahul ;
Dennehy, Denis ;
Duan, Yanqing ;
Dubey, Rameshwar ;
Dwivedi, Rohita ;
Edwards, John S. ;
Flavian, Carlos ;
Gauld, Robin ;
Grover, Varun ;
Hu, Mei-Chih ;
Janssen, Marijn ;
Jones, Paul ;
Junglas, Iris ;
Khorana, Sangeeta ;
Kraus, Sascha ;
Larsen, Kai R. ;
Latreille, Paul ;
Laumer, Sven ;
Malik, F. Tegwen ;
Mardani, Abbas ;
Mariani, Marcello ;
Mithas, Sunil ;
Mogaji, Emmanuel ;
Nord, Jeretta Horn ;
O'Connor, Siobhan .
INTERNATIONAL JOURNAL OF INFORMATION MANAGEMENT, 2023, 71
[10]
Epstein Z, 2023, Arxiv, DOI [arXiv:2306.04141, DOI 10.48550/ARXIV.2306.04141]