Generative AI in Lithuanian language

Schedule

Room: Room 228

April 5

12:00–12:25

Abstract

Presentation about how we (few local NLP enthusiasts) trained Language Transformer to generate meaningful text in Lithuanian language. Everything was based on volunteer work with huge R&D flavor. During this presentation I will not only cover what kind of data we used to train this model and what results we got but also present other initiatives we drive in NLP field. Will try to do both technical and interactive presentation.

Prerequisites

Maybe familiar with NLP tasks and LLM models in general.

Description

At this moment it's difficult to propose exact format (outlet) of presentation. There are two main topics (parts) which I would like to present:

• Part 1. Sequence-to-sequence Transformer which is able to generate (locally) text in Lithuanian language. We trained this model in two ways: 1) from text generate headline of article. 2) from headline generate text. During presentation I will cover both modes of this model. This is not LLM this is transformer which learned language rules and is able to generate sequences in Lithuanian language.
• Part 2. Lithuanian characters predictor. In Lithuanian language we have few diacritic symbols (ąčęėįšųū) and often we are skipping them and autocorrect helps to fix this. We trained model which is able to restore diacritic symbols from text i.e., it corrects suris to sūris. It's not autocorrect tool it's something more light and sophisticated how to fix 'lisping' text to correct one. It's 99% accurate and can be used i.e., in chatbots where people typing text without diacritic symbols. I would like to present these two topics during 25 min. It will be just intro that few local NLP enthusiasts are interesting in out language from ML perspective.