\_[- 1 ]{} &= f\_[-1]{}\^(\_), &\
\_[i,j,k]{} &= k\_i(\_j, \_k), &\
\_[k, ]{} &= \_[k,k,k]{}\^[-1]{}\_[k,,k]{}\_[k,,k]{}\^T \_[k,k,k]{}\^[-1]{}.

The intution behind Theorem \[thm:gen-ogd\] is as follows:

-   Within-task generalisation: The generalisation error on the most recent task leverages the information learned during training on the previous tasks. The bound is tighter compared to learning from scratch, since it depends on the residual ${\mbox{${\mbox{$\mathbf{\tilde{y}}$}}$}}_{\tau}$. Therefore it captures the transferability of knowledge across tasks.

-   Outside-task generalisation (${\mbox{$\mathcal{T}$}}_{T - 1}$): The generalisation bound increases only with respect to the Rademacher complexity when training with OGD. The tightness of this bound for OGD is due to Lemma \[lem:train-unchanged\]. This lemma is valid for OGD and not for SGD, which implies that tighter generalisation is guaranteed compared to SGD.

-   Outside-task generalisation (${\mbox{$\mathcal{T}$}}_{\tau}, \tau \leq T- 2$): The upper bound depends on the similarity between the outside task and the latest task; the more dissimilar the subsequent tasks are, the more the upper bound diverges from the initial upper bound. This bound captures catastrophic forgetting as a function of the tasks dissimilarity. This bound is the same for OGD and SGD.

These bounds share some similarities with the bounds derived by @Arora:2019:FineGrained, @liu2019understanding and @hu2019simple, where 