From Data Collection to Partner Delivery: Shipping a Paraphrasing Skill on Small Language Models

Marat Saidov

I'll discuss building capabilities on top of on-device language models, with Rewrite as a case study – a publicly available paraphrasing skill that is widely used across Microsoft's products. I'll cover comprehensive data collection strategies, carefully designed adapters training and evaluation. I'll discuss the engineering challenges we faced: achieving target latency while maintaining quality, hardening the system against edge cases, and how to deliver the technology to partners and the lessons learnt.