Mar 5 – 8, 2024
Lahan Select Gyeongju, South Korea
Asia/Seoul timezone

PACuna: Automated Fine-Tuning of Language Models for Particle Accelerators

Mar 7, 2024, 5:20 PM
20m
Lahan Select Gyeongju, South Korea

Lahan Select Gyeongju, South Korea

Lahan Select Gyeongju, South Korea
Oral (16mins + 4 mins) Tools for Humans Tools for Humans

Speaker

Antonin Sulc (DESY MCS)

Description

Navigating the landscape of particle accelerators has become increasingly challenging with recent surges in contributions. These intricate devices challenge comprehension, even within individual facilities.
To address this, we introduce PACuna, a fine-tuned language model refined through publicly available accelerator resources like conferences, pre-prints, and books.
We automated data collection and question generation to minimize expert involvement and make the data publicly available.
PACuna demonstrates proficiency in addressing accelerator questions, validated by experts.
Our approach shows adapting language models to scientific domains by fine-tuning technical texts and auto-generated corpora capturing the latest developments can further produce pre-trained models to answer some specific questions that commercially available assistants cannot and can serve as intelligent assistants for individual facilities.

Primary Keyword foundation models

Primary authors

Annika Eichler (Deutschles Elektronen Synchrotron DESY) Antonin Sulc (DESY MCS) Raimund Kammering (DESY) Tim Wilksen (DESY MCS)

Presentation materials