Automatic text summarization condenses large amounts of text into concise representations, making information retrieval and comprehension more efficient. This research investigates the impact of language-model size on the quality of automatic summarization. We compare the performance of large models against smaller open-source alternatives.

Background research highlights the increasing prevalence of large language models and the significant role they play in Natural Language Processing (NLP) tasks. However, the relationship between model size and summarization quality remains an open question, this research gap is addressed in our study by comparing the summarization quality of large and small size models, understanding the trade-offs between model size, cost and performance along with answering the practical implication of using smaller models interms of cost, deployment and data privacy.

Our methodology employs an experimental design together with a correlational design as an alternative research design methodology. We utilize a benchmark dataset encompassing diverse news text types to ensure generalizability. 10 large and small LMs are evaluated on CNN-Daily Mail Dataset, using Recall-Oriented Understudy for Gisting Evaluation (ROUGE) scores (ROUGE-1, ROUGE-2, ROUGE-L) to further showcase the quality of summarization results at model level sample generated summaries from 8 models are presented. In the analysis factors beyond ROUGE scores, such as fluency, coherence, and factuality are discussed. The findings of this study with it’s limitations underlines that using small open-source models for abstractive news summarization offers advantages in terms of cost, maintenance, deployment, and potential data privacy. However, it's important to weigh these benefits against the potential trade-off in summarization quality and functionality.