Evaluating the semantic clustering power of LLMs on ERP support texts

In this study, 6,076 IT request records obtained from the IT support software of a corporate company were analyzed in terms of clustering using text mining techniques. The aim is to improve process management by obtaining meaningful groups from unstructured text data. For text representation, the traditional TF-IDF method was used alongside deep learning-based BERT and GPT-2 models. The K-Means algorithm was applied to all three representations with different numbers of clusters; the performance of the clusters was evaluated using Silhouette and Davies-Bouldin scores. The results showed that GPT-2-based representations created more successful and discriminative clusters compared to classical methods. These findings highlight the potential of advanced language models in the automatic classification of corporate requests and the digitization of business processes.