Volltext-Downloads (blau) und Frontdoor-Views (grau)
  • search hit 1 of 157
Back to Result List

ChatGPT Code Detection: Techniques for Uncovering the Source of Code

  • In recent times, large language models (LLMs) have made significant strides in generating computer code, blurring the lines between code created by humans and code produced by artificial intelligence (AI). As these technologies evolve rapidly, it is crucial to explore how they influence code generation, especially given the risk of misuse in areas such as higher education. The present paper explores this issue by using advanced classification techniques to differentiate between code written by humans and code generated by ChatGPT, a type of LLM. We employ a new approach that combines powerful embedding features (black-box) with supervised learning algorithms including Deep Neural Networks, Random Forests, and Extreme Gradient Boosting to achieve this differentiation with an impressive accuracy of 98%. For the successful combinations, we also examine their model calibration, showing that some of the models are extremely well calibrated. Additionally, we present white-box features and an interpretable Bayes classifier to elucidate critical differences between the code sources, enhancing the explainability and transparency of our approach. Both approaches work well, but provide at most 85–88% accuracy. Tests on a small sample of untrained humans suggest that humans do not solve the task much better than random guessing. This study is crucial in understanding and mitigating the potential risks associated with using AI in code generation, particularly in the context of higher education, software development, and competitive programming.

Export metadata

Additional Services

Search Google Scholar

Statistics

frontdoor_oas
Metadaten
Author:Marc OedingenORCiD, Raphael C. EngelhardtORCiD, Robin DenzORCiD, Maximilian Hammer, Wolfgang KonenORCiD
URN:urn:nbn:de:hbz:832-epub4-27321
DOI:https://doi.org/10.3390/ai5030053
ISSN:2673-2688
Parent Title (English):AI
Publisher:MDPI
Editor:Kenji Suzuki, Demos T. Tsahalis, Antony Bryant, Roberto Montemanni, Min Chen, Paolo Bellavista, Jeanine Treffers-Daller
Document Type:Article
Language:English
Date of first Publication:2024/07/02
Date of Publication (online):2024/09/06
GND-Keyword:ChatGPT
Tag:AI; Code Detection; Large Language Models; Machine Learning
Volume:5
Issue:3
Page Number:29
Institutes:Informatik und Ingenieurwissenschaften (F10) / Fakultät 10 / Institut für Informatik
Dewey Decimal Classification:000 Allgemeines, Informatik, Informationswissenschaft
Open Access:Open Access
DeepGreen:DeepGreen
Licence (German):License LogoCreative Commons - CC BY - Namensnennung 4.0 International