Multi-Model LLM Orchestration in Python: A Case Study in Research Automation
Speaker
Mauro Pelucchi
Mauro Pelucchi is Senior Data Scientist and Big Data Engineer responsible for the design of the “Real-Time Labour Market Information System on Skill Requirements” for CEDEFOP (European Centre for the Development of Vocational Training). He currently works as Head of Global Data Science at Lightcast with the goal to develop innovative models, methods, and deployments of labour market data and other data to meet customer requirements and prototype new potential solutions. His main tasks are related to advanced machine learning modelling, labour market analyses, and the design of big data pipelines to process large datasets of online job vacancies. In collaboration with the University of Milano-Bicocca, he took part in many research projects related to the labour market intelligence systems. He collaborates with the University of Milano-Bicocca as a Lecturer for the Masters of Business Intelligence and Big Data Analytics and with the University of Bergamo as a Lecturer in Computer Engineering.
Abstract
How do you turn thousands of PDFs into actionable insights? This talk shows how we built a Python-based AI assistant using LLMs and RAG to automate literature reviews: covering architecture, trade-offs, and real lessons from production use in policy research.
Description
Literature reviews remain one of the most time-consuming and fragile steps in research workflows, often involving thousands of heterogeneous documents and rapidly evolving evidence. This talk presents the design and implementation of a Python-based AI virtual assistant that automates and accelerates literature reviews while preserving rigor, transparency, and traceability.
Built using LlamaIndex, large language models from OpenAI and Claude, and a modular retrieval-augmented generation (RAG) architecture, the system ingests academic and grey literature, semantically indexes long documents, and produces grounded, citation-aware syntheses tailored to researchers’ questions. The assistant has been deployed in policy-facing research for a European public agency, where it was used to assess the impact of artificial intelligence on labour markets across multiple sources and disciplines.
Beyond the use case, the talk focuses on engineering decisions and lessons learned: document preprocessing at scale, multi-model orchestration, prompt design, handling conflicting evidence, evaluation strategies, and human-in-the-loop validation. Attendees will gain practical insights into building production-ready LLM systems in Python that go beyond demos—showing how AI can meaningfully augment complex analytical workflows in real research settings.
How do you turn thousands of PDFs into clear, evidence-based insights? This talk explores a Python-built AI assistant for literature reviews, using LLMs, RAG, and Azure to help researchers and policymakers synthesize complex evidence—faster, transparently, and at scale.