USI - Email

Archive / INF Seminars / INF_2025_04_10_GiuseppeCrupi

USI - Email

	Cost-Efficient Software Automation with Cooperative Small Language Models

Chair: Akshatha Shenoy

Thursday

10.04

USI Campus EST, Room D0.02

16:30 - 17:30


	Giuseppe Crupi Università della Svizzera italiana

Abstract: The increasing reliance on large commercial language models (LLMs) such as ChatGPT and GitHub Copilot for software engineering tasks raises significant concerns regarding data privacy and the high cost of APIs. As a response, we are investigating a novel approach based on the collaboration of multiple small-scale LLMs as an alternative for automating software development tasks such as code generation. Preliminary experiments revealed a critical limitation: these smaller models consistently fail to reliably assess the correctness of software solutions, particularly with respect to functional requirements. This observation prompted a deeper investigation into the ability of LLMs to act as evaluators—or ‘judges’—of software-related outputs. Our findings indicate that, in zero-shot scenarios, none of the evaluated models demonstrate sufficient trustworthiness or accuracy in judgment tasks. While larger models exhibit relatively better performance, they still fall short of the reliability required for practical deployment. These results underscore the need for further research into model evaluation strategies and collaboration protocols, in order to build more sustainable and affordable AI-assisted software engineering recommender systems.

Biography: Giuseppe Crupi holds a Master’s degree in Physics from the University of Turin. Currently,he is pursuing a Ph.D. at the Università della Svizzera italiana, where he's a member of the SoftwarE Analytics Research Team (SEART) group, led by Prof. Gabriele Bavota. His research focuses on leveraging language models to support and enhance software development practices.

*************************

In February 2019, the Software Institute started its SI Seminar Series. Every Thursday afternoon, a researcher of the Institute will publicly give a short talk on a software engineering argument of their choice. Examples include, but are not limited to novel interesting papers, seminal papers, personal research overview, discussion of preliminary research ideas, tutorials, and small experiments.

On our YouTube playlist you can watch some of the past seminars. On the SI website you can find more details on the next seminar, the upcoming seminars, and an archive of the past speakers