A Comparative Study of Automated Approaches for Detecting Subjectivity and Unverifiability in Natural Language Software Requirements

Natural language software requirements are prone to subjectivity and unverifiability, which reduces clarity and testability. This study provides a controlled comparative evaluation of four automated detection approaches: rule-based lexicons, classical machine-learning models with TF-IDF features, a fine-tuned DistilBERT transformer, and a feature-level hybrid model that concatenates rule-based linguistic indicators with DistilBERT contextual embeddings on the same manually annotated dataset of 985 requirements, including 259 subjective and 726 objective items, and 165 unverifiable and 820 verifiable items. All models used identical preprocessing, an 80:20 stratified train-test split, and the same evaluation metrics. The rule-based approach achieved a precision of 0.67 and a recall of 0.04 for subjectivity and zero recall for unverifiability. Classical ML models reached F1-scores ranging from 0.47 to 0.58 (positive class). DistilBERT obtained weighted F1-scores of 0.74 (subjectivity) and 0.86 (unverifiability). The feature-level hybrid model improved subjectivity detection to a weighted F1-score of 0.79 while matching DistilBERT at 0.86 for unverifiability. These results demonstrate that combining explicit linguistic cues with contextual embeddings improves detection performance under class imbalance.