Calc-X and Calcformers: Empowering Arithmetical Chain-of-Thought through Interaction with Symbolic Systems

Varování

Publikace nespadá pod Ekonomicko-správní fakultu, ale pod Fakultu informatiky. Oficiální stránka publikace je na webu muni.cz.
Autoři

KADLČÍK Marek ŠTEFÁNIK Michal SOTOLÁŘ Ondřej MARTINEK Vlastimil

Rok publikování 2023
Druh Článek ve sborníku
Konference Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Main track
Fakulta / Pracoviště MU

Fakulta informatiky

Citace
www Manuscript in proceedings
Doi http://dx.doi.org/10.18653/v1/2023.emnlp-main.742
Klíčová slova language models; dataset; arithmetic reasoning; multistep reasoning
Popis Despite outstanding performance on many generation tasks, language models are notoriously inclined to make factual errors in tasks requiring arithmetic reasoning. To enable language models to circumvent this deficiency and offload critical computation to a symbolic system, we create a collection of Calc-X datasets that demonstrates the appropriate use of a calculator in reasoning chains. We survey and unify several existing chain-of-thoughts datasets into a proposed novel format, resulting in a standard collection of over 300,000 samples requiring arithmetic reasoning. Finally, we use the new collection to train open-source calculator-assisted language models and show that models trained on Calc-X almost double the accuracy of generating correct results compared to baselines. We make all Calc-X datasets and models publicly available.
Související projekty:

Používáte starou verzi internetového prohlížeče. Doporučujeme aktualizovat Váš prohlížeč na nejnovější verzi.