Soft Alignment Objectives for Robust Adaptation of Language Generation

Warning

This publication doesn't include Faculty of Economics and Administration. It includes Faculty of Informatics. Official publication website can be found on muni.cz.
Authors

ŠTEFÁNIK Michal KADLČÍK Marek SOJKA Petr

Year of publication 2023
Type Article in Proceedings
Conference Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
MU Faculty or unit

Faculty of Informatics

Citation
Web https://aclanthology.org/2023.acl-long.492
Doi http://dx.doi.org/10.18653/v1/2023.acl-long.492
Keywords generation; robustness; machine translation; adaptation
Description Domain adaptation allows generative language models to address specific flaws caused by the domain shift of their application. However, the traditional adaptation by further training on in-domain data rapidly weakens the model's ability to generalize to other domains, making the open-ended deployments of the adapted models prone to errors. This work introduces novel training objectives built upon a semantic similarity of the predicted tokens to the reference. Our results show that (1) avoiding the common assumption of a single correct prediction by constructing the training target from tokens' semantic similarity can largely mitigate catastrophic forgetting of adaptation, while (2) preserving the adaptation in-domain quality, (3) with negligible additions to compute costs. In the broader context, the objectives grounded in a continuous token similarity pioneer the exploration of the middle ground between the efficient but na\"{\i}ve exact-match token-level objectives and expressive but computationally- and resource-intensive sequential objectives.
Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.