arxiv:2406.00075

Arbitrary Length Generalization for Addition

Published on May 31, 2024

Authors:

Abstract

This paper introduces a novel training methodology that enables a small Transformer model to generalize the addition of two-digit numbers to numbers with unseen lengths of digits. The proposed approach employs an autoregressive generation technique, processing from right to left, which mimics a common manual method for adding large numbers. To the best of my knowledge, this methodology has not been previously explored in the literature. All results are reproducible, and the corresponding R code is available at: https://github.com/AGPatriota/ALGA-R/.

View arXiv page View PDF Add to collection

Community

AGPatriota

Jun 7, 2024

In order to test for random numbers with 100 digits run the following inside your R session opened in the main folder:
source('Testing-Digits.R')

Change the seed by commenting set.seed(10) in the file Testing-Digits.R

If you want to test the algorithm for variable digit numbers, replace
x = paste0(sample(0:9,n0, replace=TRUE), collapse="")
and y = paste0(sample(0:9,n0, replace=TRUE), collapse="")
with
x = paste0(sample(0:9,sample(1:n0,1), replace=TRUE), collapse="")
and y = paste0(sample(0:9,sample(1:n0,1), replace=TRUE), collapse=""):

Best regards,
Alexandre G. Patriota,
Department of Statistics,
Institute of Mathematics and Statistics,
University of São Paulo, Brazil.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment

Upvote

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2406.00075 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2406.00075 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2406.00075 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.