32144 Technology Research Preparation- Literature review
This subject provides postgraduate technology students with professional communication skills appropriate to their course. The subject begins with the techniques necessary to develop a literature review and professional ethics is taught in the research context. Students produce a project proposal, suitable for implementation in a research project.
My leterture review topic is What are the criteria for a successful debugging system?
1. Introduction
Artificial Intelligence (AI) has emerged as a disruptive force in response to bug problems, reshaping traditional debugging techniques with the introduction of AI-powered debugging systems. An innovative paradigm shift in software engineering can be seen in the development of AI-powered debugging systems. These systems revolutionise debugging with notable gains in effectiveness, accuracy, and productivity (Shafiq & Arshad, 2013). AI-powered debugging systems have become a promising way to speed up finding and fixing code problems in the rapidly changing world of software development (Surameery & Shakor, 2023). By looking at the fundamental standards that determine the effectiveness and usability of such systems, this report will answer the research question: "What are the criteria for a successful debugging system?" These criteria offer a thorough framework for evaluating the debugging tools powered by AI and cover a wide range of variables, including customisation, integration, accuracy, and speed. This study also thoroughly examines these criteria on how AI can speed up software development and enhance debugging procedures. The report aims to summarise the various concepts using other data analysis. It would be significant for programmers and developers. The goal of this report is to make clear the requirements for programmers who create AI-powered bug detection tools.
2.Literature review
Four criteria are crucial for defining a successful AI-power Debugging system: customisation, integration, accuracy and speed.
Customisation in AI-Powered Debugging
Customisation plays a pivotal role in enhancing the efficacy of AI-powered debugging systems. The ability to tailor these systems to specific development needs has been a recurring theme in recent research. Merow et al. (2023) highlight this aspect, pointing out that AI models like ChatGPT can be fine-tuned through training on specialised datasets, enhancing their understanding of code-bug relationships. Customisation extends to various aspects of AI debugging. Elmishali et al. (2019) introduce DeBGUer, a software tool designed for software fault prediction and automated diagnosis. It allows researchers to compare and build upon their prediction and diagnosis algorithms, indicating flexibility and adaptability (Elmishali et al., 2019). The importance of customisation in AI-powered debugging systems lies in their versatility. However, they might have limited customisation concerning integration with other development tools (Mathew, 2023, as cited in Haque & Li, 2023). In the dynamic landscape of software development, where projects often vary significantly in terms of programming languages, architectures, and complexities, the ability to fine-tune AI debugging tools to specific project requirements enhances their applicability and effectiveness.
Integration of AI-Powered Debugging
Seamless integration of AI-powered debugging systems into existing development workflows is essential for their effectiveness. Gopher, one of the AI-power debugging systems mentioned in Pradhan (2021), is designed to be highly compatible with existing machine learning frameworks and tools. Specifically, Gopher provides a Python API that can integrate with popular machine learning frameworks such as TensorFlow and PyTorch. The integration feature allows users to incorporate Gopher into their machine-learning workflows and pipelines easily. The research by Kumar et al. (2019) underscores this point by exploring Sankie, a system operating within the Azure DevOps environment. Sankie's deep integration into the development pipeline enables it to react quickly, streamlining debugging processes and contributing to more accurate outcomes (Kumar et al., 2019). Additionally, Shafiq and Arshad (2013) discuss an approach that transforms code into constraints for easy conversion, suggesting efficiency in the debugging process. Although this approach may require access to source code and failing test cases, it highlights the importance of seamless integration into the existing development pipeline in AI-powered debugging processes (Shafiq & Arshad, 2013). Integration is particularly relevant in modern software development practices characterised by rapid iterations and collaboration among diverse teams. AI debugging systems must seamlessly fit into these workflows to be effective as part of the development pipeline.
Accuracy of AI Debugging Systems
Accuracy is one of the cornerstone criteria for evaluating AI-powered debugging systems. In the quest to identify and rectify software bugs, the precision of these systems is paramount. DeepBugs, as discussed by Pradel and Sen (2017), showcases impressive accuracy in bug detection. With success rates ranging from 84.23% to 94.53%, DeepBugs demonstrates the potential for AI systems to excel in distinguishing between correct and erroneous code (Pradel & Sen, 2017). However, the accuracy of AI debugging systems is not solely dependent on their algorithms, but it is also contingent on the quality and diversity of the training data. As Merow et al. (2023) note, ChatGPT's accuracy in debugging relies heavily on the richness of its training data. Thus, while the potential for high accuracy is evident, achieving it in practice necessitates robust data collection and curation processes (Merow et al., 2023). The pursuit of accuracy is further exemplified by the research of Machado et al. (2018) on AI-assisted game debugging with Cicero. They reveal that accuracy still needs to be improved, as users frequently revisit their analyses and sometimes obtain incorrect results. Noever (2023) agrees with Machado et al. (2018) that the accuracy and reliability of automated debugging algorithms are still controversial, especially in complex tasks. The dynamic nature of software development introduces complexities that AI systems must navigate to provide accurate debugging suggestions consistently (Machado et al., 2018).
Speed in AI Debugging Systems
Speed is a critical dimension of AI-powered debugging systems, encompassing bug detection resolution efficiency and timeliness. In the context of speed, the contributions of AI-powered debugging systems are noteworthy. Surameery and Shakor (2023) emphasise how systems like ChatGPT can significantly expedite the debugging process. By providing automated suggestions and code corrections, they reduce the time developers spend on debugging tasks, ultimately accelerating the software development lifecycle (Shafiq & Shakor, 2023). Furthermore, Kumar et al. (2019) discuss the high-speed infrastructure of Azure DevOps, where Sankie operates. This infrastructure, characterised by high-performance build/release pipelines, aligns with the need for rapid real-time reactions to developers' actions and events.
On the other hand, ChatGPT could be quicker at providing high-level suggestions as it does not need as much specification and can draw on past interactions (Hughes, 2023, as cited in Haque & Li, 2023 ). The synergy between AI-powered debugging systems and high-speed development pipelines can revolutionise software development practices, minimising debugging bottlenecks (Kumar et al., 2019). However, it is worth noting that the speed of AI debugging systems may vary depending on the codebase's complexity and the bugs' nature. While they excel at providing high-level suggestions, their speed in pinpointing nuanced or deep-seated issues may differ.
Challenges and Future Directions in AI Debugging:
While AI-powered debugging systems hold immense promise, several challenges and future directions are worth considering. Achieving consistently high accuracy across various codebases and scenarios remains a research challenge. Robust data collection, diversity in training datasets, and refining machine learning algorithms are avenues for improvement. Moreover, integrating AI debugging tools into existing workflows needs further exploration. Seamless integration with various development environments and tools is essential for widespread adoption.
Additionally, addressing security and privacy concerns in debugging processes, especially when AI systems access source code, is paramount. In the pursuit of speed, optimising AI-powered debugging systems to handle large and complex codebases efficiently is a research area with substantial potential. Strategies for parallelising computations and optimising the search space for identifying bugs can significantly enhance the speed of these systems.
3.Conclusion
In conclusion, AI-powered debugging systems represent a significant advancement in software development, offering highly customisable solutions that are seamlessly integrated into existing workflows, accurate in bug detection, and efficient in bug resolution. Customisation ensures that these systems adapt to the unique needs of each project, enhancing their applicability. Integration fosters collaboration and streamlines processes, while accuracy is essential for reliable bug detection. Speed, on the other hand, accelerates the development lifecycle. While these criteria represent the pillars of AI-powered debugging systems, it is essential to recognise that the field is continually evolving. The challenges of achieving high accuracy and optimising speed are ongoing research endeavours, as demonstrated by the dynamic landscape of AI debugging. In summary, AI-powered debugging systems hold immense potential to transform software development practices, making them more efficient, accurate, and adaptable to diverse project requirements. Addressing the challenges and advancing the state of the art in customisation, integration, accuracy, and speed will be instrumental in realising this potential.
4. Reference list
Haque, M.A., & Li, S. (2023). The Potential Use of ChatGPT for Debugging and Bug Fixing. EAI Endorsed Transactions on AI and Robotics. https://pdfs.semanticscholar.org/e5f1/724d6e8533edbcdcfaf8d59fe51884681342.pdf
Kumar, R., Bansal, C., Maddila, C.S., Sharma, N., Martelock, S., & Bhargava, R. (2019). Building Sankie: An AI Platform for DevOps. 2019 IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE), pp. 48–53. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8823620&tag=1
Li, J., Zhang, Y., Lu, S., Gunawi, H.S., Gu, X., Huang, F., & Li, D. (2023). Performance Bug Analysis and Detection for Distributed Storage and Computing Systems. ACM Transactions on Storage, 19, 1 - 33.
Machado, T., Gopstein, D., Nealen, A., Nov, O., & Togelius, J. (2018). AI-Assisted Game Debugging with Cicero. 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8. https://ieeexplore.ieee.org/abstract/document/8477829?casa_token=17j_WBUYGasAAAAA:GLijgHNGZxkXNNYbRNsu0tunmlZU47msCSaKq_-ZCEzTKxvtjEbr56NFn5CKp3NOEkrSqqLm
Merow, C., Serra-Diaz, J. M., Enquist, B. J., & Wilson, A. M. (2023). AI chatbots can boost scientific coding. Nature Ecology & Evolution, 7(7), 960–962. https://doi.org/10.1038/s41559-023-02063-3
Nigar M. Shafiq Surameery, & Mohammed Y. Shakor. (2023). Use Chat GPT to Solve Programming Bugs. International Journal of Information Technology &Amp; Computer Engineering (IJITC) ISSN: 2455–5290, 3(01), 17–22. https://doi.org/10.55529/ijitc.31.17.22
Noever, D. (2023). Can Large Language Models Find And Fix Vulnerable Software? ArXiv, abs/2308.10345. https://arxiv.org/ftp/arxiv/papers/2308/2308.10345.pdf
Pradel, M., & Sen, K. (2017). Deep Learning to Find Bugs. https://software-lab.org/publications/DeepBugs_TR_Nov2017.pdf
Pradhan, R., Zhu, J., Glavic, B., & Salimi, B. (2021). Interpretable Data-Based Explanations for Fairness Debugging. Proceedings of the 2022 International Conference on Management of Data. https://doi.org/10.1145/3514221.3517886
Shafiq, H.A., & Arshad, Z. (2013). Automated Debugging and Bug Fixing Solutions: A Systematic Literature Review and Classification. http://www.diva-portal.org/smash/get/diva2:830404/FULLTEXT01.pdf
Tutor’s review:
“The purpose is touched upon but not clearly stated. Where is the description of the different sections in the report ? Your literature review is only 2 pages but it has a good amount of citations. You needed to reiterate the research question in the conclusion.
You should also have expanded on the significance in the introduction and placed it in the conclusion.
Pradel, M., & Sen, K. (2017). Deep Learning to Find Bugs. https://software- lab.org/publications/DeepBugs_TR_Nov2017.pdf is not a peer reviewed article but a technical report.”
Thomas Dolmark, 27 Sep 2023 at 17:35
My reflection:
Maybe because it is the first time writing literature review, I have some structure problems and stating issue in my document. I am getting used to writing with formal words and terms. Next time, I need to be more careful about the lecture I used.
