"Zero-shot Prompting for Code Complexity Prediction Using GitHub Copilot" accepted at NLBSE'23 (co-located with ICSE'23).

Feb 24, 2023. | By: Joanna C. S. Santos

Our short paper, “Zero-shot Prompting for Code Complexity Prediction Using GitHub Copilot”, got accepted at the 2nd InternationalWorkshop on Natural Language-based Software Engineering (NLBSE’23) co-located with ICSE 2023.

In this short paper, we describe a preliminary study that investigates whether GitHub Copilot can help predict the runtime complexity of a source code using zero-shot prompting. In our preliminary study, we found that GitHub Copilot can correctly predict the runtime complexity 45.44% times in the first suggestion and 56.38% times considering all suggestions. We also compared Copilot to other machine learning, neural network, and transformer-based approaches for code complexity prediction. We observed that Copilot outperformed other approaches for predicting code with linear complexity O(n).

BibTeX

@inproceedings{siddiq2023zero,
  author={Siddiq, Mohammed Latif and Samee, Abdus and Azgor, Sk Ruhul and Haider, Md. Asif and Sawraz, Shehabul Islam and Santos, Joanna C. S.},
  booktitle={2023 IEEE/ACM 2nd International Workshop on Natural Language-Based Software Engineering (NLBSE)}, 
  title={Zero-shot Prompting for Code Complexity Prediction Using GitHub Copilot}, 
  year={2023},
  volume={},
  number={},
  pages={56-59},
  doi={10.1109/NLBSE59153.2023.00018}
}