Abstract
This paper explores the innovative use of Natural Language Processing (NLP) technology, specifically ChatGPT, for Automated Program Repair (APR) in an agile system context. The core of this study involved the construction of a conventional virtualised DevOps Pipeline System that integrated ChatGPT alongside critical software management systems such as Bitbucket and Jenkins. This integration created a smart framework that could perform automated code analysis and repair. Guided by Systems Thinking principles, the study synthesised knowledge from the fields of systems engineering, software engineering, and Artificial Intelligence (AI) to build an experimental case study where ChatGPT’s APR competencies were evaluated against the QuixBugs benchmark. Through extensive testing, the paper showed the efficacy of ChatGPT in APR, revealing notable performance improvements after its fine-tuning on a dataset from the QuixBugs code repository. The experimental tests carried out on the pipeline system showed that ChatGPT was fully capable of performing APR with a 59.78% pass score for the base model GPT-3.5-Turbo-0613, and a 97.46% score when it was trained and fine-tuned on the dataset of erroneous code. The findings of this research underscored the efficacy of ChatGPT in both syntactical and semantical code corrections and offered prudent insights regarding its application. The paper serves as a valuable resource for both practitioners and researchers responsible for developing and maintaining software infrastructures.
Keywords: APR, NLP, ChatGPT, DevOps Pipeline System