Predicting software effort using BERT-based word embeddings

Accurate software effort estimation is essential for effective project planning and resource allocation, particularly in Agile software development where evolving requirements challenge traditional methods. This study explores the potential of pre-trained BERT (Bidirectional Encoder Representations from Transformers) models, a state-of-the-art NLP technique, to improve estimation accuracy. We compare the performance of the BERT base and BERT large models in diverse project scenarios. The results show that BERT Base consistently outperforms BERT Large in cross-repository and project-based contexts, owing to its computational efficiency and adaptability. A combined CNN and BERT Base model further enhances story point prediction for new projects, achieving superior accuracy and robustness. These findings highlight the practical advantages of leveraging BERT Base in Agile environments, offering valuable insights for researchers, software developers, and project managers. Future work will focus on external validation using commercial datasets, alternative deep learning architectures, and improved fine-tuning strategies to further advance effort estimation practices.