}\pgfsys@color@cmyk@fill{1}{0}{0}{0}\pgfsys@invoke{
}\definecolor[named]{pgffillcolor}{rgb}{0,1,1}{}\pgfsys@rect{-22.44563pt}{-8.06pt}{44.89127pt}{16.12pt}\pgfsys@stroke\pgfsys@invoke{
} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-18.44563pt}{-2.03pt}\pgfsys@invoke{
}\hbox{{\definecolor[named]{.}{rgb}{0,1,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,1,1}\pgfsys@color@cmyk@stroke{1}{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@cmyk@fill{1}{0}{0}{0}\pgfsys@invoke{
}\hbox{{\definecolor[named]{.}{rgb}{0,1,1}\color[rgb]{0,1,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,1,1}\pgfsys@color@cmyk@stroke{1}{0}{0}{0}\pgfsys@color@cmyk@fill{1}{0}{0}{0}$\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\frac{3351953103\pi^{2}}{4294967296}$}}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{
{}{}{}{}{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\\\ \end{pmatrix}$
(4.28)

This lifting matrix displays some interesting patterns that we will now
explore. Firstly there is the equality of lifts on the diagonal with those on
the first off diagonal on the upper-half triangle. This can be understood by
considering the application of two $G$ modes to a diagonal term (for which
$s=m-\frac{1}{2}$) since

$G^{-}_{-,-1/2}G^{+}_{+,-1/2}|\alpha
d\rangle_{++(m,m-1/2)}^{--}=i\,G^{-}_{-,-1/2}|\alpha\alpha\rangle_{++++(m,m)}=2im|\alpha
d\rangle_{++(m,m+1/2)}^{--}\ .$ (4.29)

We can then use the relation (B.9) at each step of (4.29) to obtain the simple
equality

$E^{(2)}\big{(}|\alpha
d\rangle_{++(m,m-1/2)}^{--}\big{)}=E^{(2)}\big{(}|\alpha
d\rangle_{++(m,m+1/2)}^{--}\big{)}\ .$ (4.30)

The use of these two relations is justified due to the fact that
$G^{-}_{-,\frac{1}{2}}|\alpha d\rangle_{++(m,m-1/2)}^{--}=0$ and
$G^{+}_{+,\frac{1}{2}}|\alpha\alpha\rangle_{++++(m,m)}=0$ and that at each
step in (4.29) only one state is created and hence (B.9) could be
straightforwardly applied.

Secondly, there is also a single pair of equal lifts in the first column of
the lifting matrix (4.28) (in black circles) which can be understood from the
action of $L_{-1}$ on the level-$3/2$ state in this family

$L_{-1}|\alpha d\rangle_{++(1,1/2)}^{--}=|\alpha
d\rangle_{++(2,1/2)}^{--}+|\alpha d\rangle_{++(1,3/2)}^{--}\ .$ (4.31)

The $L_{-1}$ relation between lifts (B.5) can then be applied, since the
condition $L_{1}|\alpha d\rangle_{++(1,1/2)}^{--}=0$ holds, yielding

$E^{(2)}\big{(}L_{-1}|\alpha
d\rangle_{++(1,1/2)}^{--}\big{)}=E^{(2)}\big{(}|\alpha
d\rangle_{++(1,1/2)}^{--}\big{)}\ .$ (4.32)

Due to the descendant state (4.31) being a sum of two basis states the lifting
relation (4.32) cannot be immediately extrapolated to a relation between
elements of the lifting matrix (4.28). Using the identity (B) with
$|\phi_{1}\rangle=|\alpha d\rangle_{++(2,1/2)}^{--}$ and
$|\phi_{2}\rangle=|\alpha d\rangle_{++(1,3/2)}^{--}$ along with the fact that
$E^{(2)}(\phi_{1};\phi_{2})=0$ and $\langle\phi_{1}|\phi_{2}\rangle=0$ here,
(B.9) then gives

$\displaystyle E^{(2)}\big{(}|\alpha
d\rangle_{++(2,1/2)}^{--}\big{)}=\frac{1}{2}\bigg{[}3E^{(2)}\big{(}|\alpha
d\rangle_{++(1,1/2)}^{--}\big{)}-E^{(2)}\big{(}|\alpha
d\rangle_{++(1,3/2)}^{--}\big{)}\bigg{]}\ .$ (4.33)

Using the previously derived relation (4.30) with $m=1$ we then obtain

$E^{(2)}\big{(}|\alpha d\rangle_{++(2,1/2)}^{--}\big{)}=E^{(2)}\big{(}|\alpha
d\rangle_{++(1,1/2)}^{--}\big{)}\ ,$ (4.34)

as seen from the lifting matrix (4.28).

###  4.4 Vanishing lift of the stress tensor from free field realisation

As a final check that encompasses a large number of the lifts from Sections
3.2 and 3.4 we compute the lift of the stress tensor. Clearly, from both
general principles and from the fact that any single-mode single-copy
excitation of the global NS vacuum can be written as a single global mode on
the vacuum, this state has vanishing lift. However, from the perspective of
the free-field realisation of the CFT the stress tensor state is given in
terms of bosons and fermions as (A.9c) and its modes can be written as

$\displaystyle L_{m}$
$\displaystyle=-\frac{1}{2}\sum_{n}\epsilon^{BA}\epsilon^{\dot{B}\dot{A}}\alpha_{B\dot{B},n}\alpha_{A\dot{A},m-n}-\frac{1}{2}\sum_{r}\bigg{(}m-r+\frac{1}{2}\bigg{)}\epsilon_{\beta\alpha}\epsilon_{BA}d^{\beta
B}_{r}d^{\alpha A}_{m-r}\ .$ (4.35)

The stress tensor initial state is then given by

$\displaystyle L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle$
$\displaystyle=-\frac{1}{2}\sum_{n}\epsilon^{BA}\epsilon^{\dot{B}\dot{A}}|\alpha\alpha\rangle_{B\dot{B}A\dot{A}(-n,n+2)}-\frac{1}{2}\sum_{r}\bigg{(}\\!-2-r+\frac{1}{2}\bigg{)}\epsilon_{\beta\alpha}\epsilon_{BA}|dd\rangle^{\beta
B\alpha A}_{(-r,r+2)}$
$\displaystyle=-\frac{1}{2}\epsilon^{BA}\epsilon^{\dot{B}\dot{A}}|\alpha\alpha\rangle_{B\dot{B}A\dot{A}(1,1)}+\frac{1}{2}\epsilon_{\beta\alpha}\epsilon_{BA}|dd\rangle^{\beta
B\alpha A}_{(1/2,3/2)}\ ,$ (4.36)

and so the lift of $L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle$ in terms of
the states of Sections 3.2 and 3.4 is non-trivial and its vanishing must come
about from cancellation between terms.

In order to compute the lift of $L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle$
from (2.15) it is necessary to compute the lift-moving amplitude

$A^{(1)(1)}_{L_{-2}}(w_{2},w_{1})\equiv\langle
0_{\scriptscriptstyle{N\\!S}}|\big{(}L_{2}^{(1)}\big{)}\,\Big{(}G^{+}_{-,-\frac{1}{2}}\sigma^{-}\Big{)}(w_{2})\Big{(}G^{-}_{+,-\frac{1}{2}}\sigma^{+}\Big{)}(w_{1})\,\big{(}L_{-2}^{(1)}\big{)}|0_{\scriptscriptstyle{N\\!S}}\rangle\
,$ (4.37)

with the initial and final states broken down as (4.4). This amplitude has the
initial and final excitations placed on copy $1$; as in the computations of
Section 3, the amplitude will then be symmetrised over copies. In this
amplitude there will then be four types of contributions, where the initial
and final states can each either be of the form $|\alpha\alpha\rangle$ or
$|dd\rangle$. The contributions with both initial and final states being of
the same family will be related to the lifts computed in Sections 3.2 and 3.4,
whereas the contributions with initial and final states being from different
families will require additional computation.

####  4.4.1 $\alpha\alpha-\alpha\alpha$ contributions

The contributions to (4.37) with initial and final states being of the
$|\alpha\alpha\rangle$ family are given by

$\displaystyle A^{(1)(1)}_{\alpha\alpha\alpha\alpha}$
$\displaystyle\equiv\frac{1}{4}\epsilon^{DC}\epsilon^{\dot{D}\dot{C}}\epsilon^{BA}\epsilon^{\dot{B}\dot{A}}$
$\displaystyle\quad\times\langle
0_{\scriptscriptstyle{N\\!S}}|\big{(}\alpha^{(1)}_{D\dot{D},1}\alpha^{(1)}_{C\dot{C},1}\big{)}\,\Big{(}G^{+}_{-,-\frac{1}{2}}\sigma^{-}\Big{)}(w_{2})\Big{(}G^{-}_{+,-\frac{1}{2}}\sigma^{+}\Big{)}(w_{1})\,\big{(}\alpha^{(1)}_{B\dot{B},-1}\alpha^{(1)}_{A\dot{A},-1}\big{)}|0_{\scriptscriptstyle{N\\!S}}\rangle\
.$ (4.38)

The terms in the sums over $SU(2)$ indices where the quantum numbers of the
final-state modes are opposite to those of the initial-state modes are
directly related to the lifts of those states, however, all of the terms are
computed exactly as in Section 3.2, requiring a computation on the $t$-plane
of the form

$\displaystyle A^{(1)(1)}_{\alpha\alpha\alpha\alpha}$
$\displaystyle\sim\oint_{\infty}\frac{dt_{5}}{2\pi
i}\oint_{\infty}\frac{dt_{6}}{2\pi i}\oint_{-a}\frac{dt_{4}}{2\pi
i}\oint_{-a}\frac{dt_{3}}{2\pi
i}\frac{z_{5}(t_{5})z_{6}(t_{6})}{z_{4}(t_{4})z_{3}(t_{3})}\epsilon^{DC}\epsilon^{\dot{D}\dot{C}}\epsilon^{BA}\epsilon^{\dot{B}\dot{A}}$
$\displaystyle\qquad\qquad\qquad\qquad\times\langle\partial
X_{D\dot{D}}(t_{6})\partial
X_{C\dot{C}}(t_{5})G^{+}_{-}(t_{2})G^{-}_{+}(t_{1})\partial
X_{B\dot{B}}(t_{4})\partial X_{A\dot{A}}(t_{3})\rangle\ .$ (4.39)

The correlator of fields on the $t$-plane in (4.4.1) is given in (3.2).
Performing the integrals over the insertions of the deformation operators and
symmetrising over copies yields a total $\alpha\alpha-\alpha\alpha$
contribution to the lift of $L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle$ of

$E^{(2)}_{\alpha\alpha\alpha\alpha}\big{(}L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle\big{)}=\frac{3\pi^{2}}{4}\lambda^{2}\
.$ (4.40)

####  4.4.2 $\alpha\alpha-dd$ contributions

The contributions to (4.37) with initial states being of the $|dd\rangle$
family and final states of the $|\alpha\alpha\rangle$ family are given by

$\displaystyle A^{(1)(1)}_{\alpha\alpha dd}$
$\displaystyle\equiv\frac{1}{4}\epsilon^{DC}\epsilon^{\dot{D}\dot{C}}\epsilon_{\beta\alpha}\epsilon_{BA}$
$\displaystyle\qquad\times\langle
0_{\scriptscriptstyle{N\\!S}}|\big{(}\alpha^{(1)}_{D\dot{D},1}\alpha^{(1)}_{C\dot{C},1}\big{)}\,\Big{(}G^{+}_{-,-\frac{1}{2}}\sigma^{-}\Big{)}(w_{2})\Big{(}G^{-}_{+,-\frac{1}{2}}\sigma^{+}\Big{)}(w_{1})\,\big{(}d^{(1)\beta
B}_{-1/2}d^{(1)\alpha A}_{-3/2}\big{)}|0_{\scriptscriptstyle{N\\!S}}\rangle\
.$ (4.41)

None of these terms are directly related to lifts, however, the computation of
these contributions requires only a small modification to methods of Section 3
to yield (on the $t$-plane)

$\displaystyle A^{(1)(1)}_{\alpha\alpha dd}$
$\displaystyle\sim\oint_{\infty}\frac{dt_{5}}{2\pi
i}\oint_{\infty}\frac{dt_{6}}{2\pi i}\oint_{-a}\frac{dt_{4}}{2\pi
i}\oint_{-a}\frac{dt_{3}}{2\pi
i}\frac{z_{5}(t_{5})z_{6}(t_{6})}{z_{4}(t_{4})z_{3}(t_{3})^{2}}\bigg{(}\frac{dz_{4}}{dt_{4}}\bigg{)}^{\\!\frac{1}{2}}\\!\bigg{(}\frac{dz_{3}}{dt_{3}}\bigg{)}^{\\!\frac{1}{2}}\\!\bigg{(}\frac{t_{4}-t_{1}}{t_{4}-t_{2}}\bigg{)}^{\\!q_{\beta}}\\!\bigg{(}\frac{t_{3}-t_{1}}{t_{3}-t_{2}}\bigg{)}^{\\!q_{\alpha}}$
$\displaystyle\qquad\qquad\times\epsilon^{DC}\epsilon^{\dot{D}\dot{C}}\epsilon_{\beta\alpha}\epsilon_{BA}\langle\partial
X_{D\dot{D}}(t_{6})\partial
X_{C\dot{C}}(t_{5})G^{+}_{-}(t_{2})G^{-}_{+}(t_{1})\psi^{\beta
B}(t_{4})\psi^{\alpha A}(t_{3})\rangle\ .$ (4.42)

The $t$-plane correlator above can be computed by expanding the $G$ fields via
(A.9b) and using Wick contractions

$\displaystyle\langle$ $\displaystyle\partial X_{D\dot{D}}(t_{6})\partial
X_{C\dot{C}}(t_{5})G^{+}_{-}(t_{2})G^{-}_{+}(t_{1})\psi^{\beta
B}(t_{4})\psi^{\alpha
A}(t_{3})\rangle=\frac{-\epsilon_{CD}\epsilon_{\dot{C}\dot{D}}}{(t_{6}-t_{5})^{2}(t_{2}-t_{1})^{2}}\Bigg{[}\frac{2\epsilon^{\alpha\beta}\epsilon^{AB}}{(t_{4}-t_{3})(t_{2}-t_{1})}$
$\displaystyle-\frac{\epsilon^{+\beta}\epsilon^{-\alpha}\epsilon^{AB}}{(t_{2}-t_{4})(t_{1}-t_{3})}-\frac{\epsilon^{-\beta}\epsilon^{+\alpha}\epsilon^{AB}}{(t_{2}-t_{3})(t_{1}-t_{4})}\Bigg{]}+\frac{\epsilon_{\dot{D}-}\epsilon_{\dot{C}+}}{(t_{6}-t_{2})^{2}(t_{5}-t_{1})^{2}}\Bigg{[}\frac{-\epsilon^{\alpha\beta}\epsilon^{AB}\epsilon_{CD}}{(t_{4}-t_{3})(t_{2}-t_{1})}$
$\displaystyle-\frac{\epsilon^{+\beta}\epsilon^{-\alpha}\delta^{B}_{D}\delta^{A}_{C}}{(t_{2}-t_{4})(t_{1}-t_{3})}+\frac{\epsilon^{-\beta}\epsilon^{+\alpha}\delta^{A}_{D}\delta^{B}_{C}}{(t_{2}-t_{3})(t_{1}-t_{4})}\Bigg{]}+\frac{\epsilon_{\dot{D}+}\epsilon_{\dot{C}-}}{(t_{6}-t_{1})^{2}(t_{5}-t_{2})^{2}}\Bigg{[}\frac{\epsilon^{\alpha\beta}\epsilon^{AB}\epsilon_{CD}}{(t_{4}-t_{3})(t_{2}-t_{1})}$
$\displaystyle-\frac{\epsilon^{+\beta}\epsilon^{-\alpha}\delta^{A}_{D}\delta^{B}_{C}}{(t_{2}-t_{4})(t_{1}-t_{3})}+\frac{\epsilon^{-\beta}\epsilon^{+\alpha}\delta^{B}_{D}\delta^{A}_{C}}{(t_{2}-t_{3})(t_{1}-t_{4})}\Bigg{]}\
.$ (4.43)

Performing the integrals over the insertions of the deformation operators and
symmetrising over copies yields a total $\alpha\alpha-dd$ contribution to the
lift of $L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle$ of

$E^{(2)}_{\alpha\alpha
dd}\big{(}L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle\big{)}=-\frac{3\pi^{2}}{4}\lambda^{2}\
.$ (4.44)

####  4.4.3 $dd-\alpha\alpha$ contributions

The contributions to (4.37) with initial states being of the
$|\alpha\alpha\rangle$ family and final states of the $|dd\rangle$ family are
given by

$\displaystyle A^{(1)(1)}_{dd\alpha\alpha}$
$\displaystyle\equiv\frac{1}{4}\epsilon_{\delta\gamma}\epsilon_{DC}\epsilon^{BA}\epsilon^{\dot{B}\dot{A}}$
$\displaystyle\qquad\times\langle
0_{\scriptscriptstyle{N\\!S}}|\big{(}d^{(1)\delta D}_{3/2}d^{(1)\gamma
C}_{1/2}\big{)}\,\Big{(}G^{+}_{-,-\frac{1}{2}}\sigma^{-}\Big{)}(w_{2})\Big{(}G^{-}_{+,-\frac{1}{2}}\sigma^{+}\Big{)}(w_{1})\,\big{(}\alpha^{(1)}_{B\dot{B},-1}\alpha^{(1)}_{A\dot{A},-1}\big{)}|0_{\scriptscriptstyle{N\\!S}}\rangle\
.$ (4.45)

None of these terms are directly related to lifts, however, the computation of
these contributions requires only a small modification to methods of Section 3
to yield (on the $t$-plane)

$\displaystyle A^{(1)(1)}_{dd\alpha\alpha}$
$\displaystyle\sim\oint_{\infty}\frac{dt_{5}}{2\pi
i}\oint_{\infty}\frac{dt_{6}}{2\pi i}\oint_{-a}\frac{dt_{4}}{2\pi
i}\oint_{-a}\frac{dt_{3}}{2\pi
i}\frac{z_{6}(t_{6})}{z_{4}(t_{4})z_{3}(t_{3})}\bigg{(}\frac{dz_{6}}{dt_{6}}\bigg{)}^{\\!\frac{1}{2}}\\!\bigg{(}\frac{dz_{5}}{dt_{5}}\bigg{)}^{\\!\frac{1}{2}}\\!\bigg{(}\frac{t_{6}-t_{1}}{t_{6}-t_{2}}\bigg{)}^{\\!q_{\delta}}\\!\bigg{(}\frac{t_{5}-t_{1}}{t_{5}-t_{2}}\bigg{)}^{\\!q_{\gamma}}$
$\displaystyle\qquad\qquad\times\epsilon_{\delta\gamma}\epsilon_{DC}\epsilon^{BA}\epsilon^{\dot{B}\dot{A}}\langle\psi^{\delta
D}(t_{6})\psi^{\gamma C}(t_{5})G^{+}_{-}(t_{2})G^{-}_{+}(t_{1})\partial
X_{B\dot{B}}(t_{4})\partial X_{A\dot{A}}(t_{3})\rangle\ .$ (4.46)

The $t$-plane correlator above can be computed by expanding the $G$ fields via
(A.9b) and using Wick contractions

$\displaystyle\langle$ $\displaystyle\psi^{\delta D}(t_{6})\psi^{\gamma
C}(t_{5})G^{+}_{-}(t_{2})G^{-}_{+}(t_{1})\partial X_{B\dot{B}}(t_{4})\partial
X_{A\dot{A}}(t_{3})\rangle=\frac{-\epsilon_{CD}\epsilon_{\dot{C}\dot{D}}}{(t_{4}-t_{3})^{2}(t_{2}-t_{1})^{2}}\Bigg{[}\frac{2\epsilon^{\alpha\beta}\epsilon^{AB}}{(t_{6}-t_{5})(t_{2}-t_{1})}$
$\displaystyle+\frac{\epsilon^{\beta-}\epsilon^{\alpha+}\epsilon^{AB}}{(t_{6}-t_{2})(t_{5}-t_{1})}+\frac{\epsilon^{\beta+}\epsilon^{\alpha-}\epsilon^{AB}}{(t_{6}-t_{1})(t_{5}-t_{2})}\Bigg{]}+\frac{\epsilon_{+\dot{D}}\epsilon_{-\dot{C}}}{(t_{2}-t_{4})^{2}(t_{1}-t_{3})^{2}}\Bigg{[}\frac{\epsilon^{\alpha\beta}\epsilon^{AB}\epsilon_{CD}}{(t_{6}-t_{5})(t_{2}-t_{1})}$
$\displaystyle-\frac{\epsilon^{\beta-}\epsilon^{\alpha+}\delta^{B}_{D}\delta^{A}_{C}}{(t_{6}-t_{2})(t_{5}-t_{1})}+\frac{\epsilon^{\beta+}\epsilon^{\alpha-}\delta^{A}_{D}\delta^{B}_{C}}{(t_{6}-t_{1})(t_{5}-t_{2})}\Bigg{]}+\frac{\epsilon_{-\dot{D}}\epsilon_{+\dot{C}}}{(t_{2}-t_{3})^{2}(t_{1}-t_{4})^{2}}\Bigg{[}\frac{-\epsilon^{\alpha\beta}\epsilon^{AB}\epsilon_{CD}}{(t_{6}-t_{5})(t_{2}-t_{1})}$
$\displaystyle-\frac{\epsilon^{\beta-}\epsilon^{\alpha+}\delta^{A}_{D}\delta^{B}_{C}}{(t_{6}-t_{2})(t_{5}-t_{1})}+\frac{\epsilon^{\beta+}\epsilon^{\alpha-}\delta^{B}_{D}\delta^{A}_{C}}{(t_{6}-t_{1})(t_{5}-t_{2})}\Bigg{]}\
.$ (4.47)

Performing the integrals over the insertions of the deformation operators and
symmetrising over copies yields a total $dd-\alpha\alpha$ contribution to the
lift of $L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle$ of

$E^{(2)}_{dd\alpha\alpha}\big{(}L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle\big{)}=-\frac{3\pi^{2}}{4}\lambda^{2}\
.$ (4.48)

####  4.4.4 $dd-dd$ contributions

The contributions to (4.37) with initial and final states being of the
$|dd\rangle$ family are given by

$\displaystyle A^{(1)(1)}_{dddd}$
$\displaystyle\equiv\frac{1}{4}\epsilon_{\delta\gamma}\epsilon_{DC}\epsilon_{\beta\alpha}\epsilon_{BA}$
$\displaystyle\qquad\times\langle
0_{\scriptscriptstyle{N\\!S}}|\big{(}d^{(1)\delta D}_{3/2}d^{(1)\gamma
C}_{1/2}\big{)}\,\Big{(}G^{+}_{-,-\frac{1}{2}}\sigma^{-}\Big{)}(w_{2})\Big{(}G^{-}_{+,-\frac{1}{2}}\sigma^{+}\Big{)}(w_{1})\,\big{(}d^{(1)\beta
B}_{-1/2}d^{(1)\alpha A}_{-3/2}\big{)}|0_{\scriptscriptstyle{N\\!S}}\rangle\
.$ (4.49)

The terms in the sums over $SU(2)$ indices where the quantum numbers of the
final-state modes are opposite to those of the initial-state modes are
directly related to the lifts of those states, however, all of the terms are
computed exactly as in Section 3.2, requiring a computation on the $t$-plane
of the form

$\displaystyle A^{(1)(1)}_{dddd}$
$\displaystyle\sim\oint_{\infty}\\!\frac{dt_{5}}{2\pi
i}\oint_{\infty}\\!\frac{dt_{6}}{2\pi i}\oint_{-a}\\!\frac{dt_{4}}{2\pi
i}\oint_{-a}\\!\frac{dt_{3}}{2\pi
i}\frac{z_{6}(t_{6})}{z_{4}(t_{4})z_{3}(t_{3})^{2}}\bigg{(}\frac{dz_{6}}{dt_{6}}\frac{dz_{5}}{dt_{5}}\frac{dz_{4}}{dt_{4}}\frac{dz_{3}}{dt_{3}}\bigg{)}^{\\!\\!\frac{1}{2}}\\!\bigg{(}\frac{t_{6}-t_{1}}{t_{6}-t_{2}}\bigg{)}^{\\!\\!q_{\delta}}\bigg{(}\frac{t_{5}-t_{1}}{t_{5}-t_{2}}\bigg{)}^{\\!\\!q_{\gamma}}$
$\displaystyle\times\\!\bigg{(}\frac{t_{4}-t_{1}}{t_{4}-t_{2}}\bigg{)}^{\\!\\!q_{\beta}}\\!\bigg{(}\frac{t_{3}-t_{1}}{t_{3}-t_{2}}\bigg{)}^{\\!\\!q_{\alpha}}\epsilon_{\delta\gamma}\epsilon_{\beta\alpha}\epsilon_{DC}\epsilon_{BA}\langle\psi^{\delta
D}(t_{6})\psi^{\gamma C}(t_{5})G^{+}_{-}(t_{2})G^{-}_{+}(t_{1})\psi^{\beta
B}(t_{4})\psi^{\alpha A}(t_{3})\rangle\ .$ (4.50)

The correlator of fields on the $t$-plane in (4.4.1) is given in (3.4).
Performing the integrals over the insertions of the deformation operators and
symmetrising over copies yields a total $dd-dd$ contribution to the lift of
$L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle$ of

$E^{(2)}_{dddd}\big{(}L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle\big{)}=\frac{3\pi^{2}}{4}\lambda^{2}\
.$ (4.51)

The total lift of the state $L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle$ is
then obtained by summing the four types of contributions from (4.40), (4.44),
(4.48) and (4.51) to get

$\displaystyle
E^{(2)}\big{(}L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle\big{)}$
$\displaystyle=E^{(2)}_{\alpha\alpha\alpha\alpha}\big{(}L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle\big{)}+E^{(2)}_{\alpha\alpha
dd}\big{(}L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle\big{)}+E^{(2)}_{dd\alpha\alpha}\big{(}L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle\big{)}+E^{(2)}_{dddd}\big{(}L_{-2}|0_{\scriptscriptstyle{N\\!S}}\rangle\big{)}$
$\displaystyle=0\ ,$ (4.52)

as expected. This provides a non-trivial check the low-level results of
Sections 3.2 and 3.4.

##  5 Discussion

The D1-D5 CFT has a set of BPS states that are simply described at the
orbifold point: all states with purely left moving (or purely right moving)
excitations are BPS at this point in moduli space. As we move away from this
point some states lift, with the number of unlifted states being bounded from
below by an index. However, the question of which states lift and by how much
remains. This pattern of lifting is an interesting issue to study and a full
understanding of this problem has proved to be elusive so far. Progress has
been made in studying the lift of various families of states; some of these
computations were summarised in the introduction. The present paper studied a
new family of states: those with two oscillator excitations on a single copy
of the $c=6$ seed CFT.

The expectation value of the lift for a general state in this family was
obtained in terms of a fixed number of nested contour integrals on a given
integrand; this integrand depends on the mode numbers of the oscillators in
the state. We evaluate these contour integrals to obtain the explicit value of
the lift for various subfamilies of states. Explicit lifting matrices for mode
numbers up to order $7$ were presented for examples of each of the three types
of states in (3.18), (3.30) and (3.42). Explicit values of lifts were also
computed for two types of subfamilies: (i) the states where one mode number
was held to be small and the other was allowed to grow large (we presented the
first $20$ lifts) and (ii) where both mode numbers were taken to have the same
value; this mode number reached up to a value $20$. This explicit data can be
found in tables 1, 2 and 3. The choice to present explicit lifts for these
states was purely for space and computational reasons; the method described in
this paper can scale arbitrarily given the resources.

We note that the lifts found in this paper are lifts in the theory for a
general number $N$ of copies of the $c=6$ theory. The lifting method used sees
only two copies of the seed CFT at a time and thus it is sufficient to
consider only two copies in intermediate steps. As given in (3.2), the fact
that there are $N$ copies in total can be easily reinstated using
combinatorics. Also of note is that one could equally use a different basis of
2-mode states from what we have used; one where states are grouped into
representations of the various $SU(2)$ symmetries of the algebra. In this
basis the fact that the lifts of states in the same multiplet are equal would
be explicit. However, we find it more convenient to use the basis of Section 3
with this method of computing lifts. In either case, the additional relations
between lifts discussed in Section 4 that stem from superconformal Ward
identities would still be present.

Having explicit values of the lift up to such high levels allows one to
observe a smooth curve passing through the plot of lifts (see e.g. Figure 3).
In [33] it was found that for single-copy superconformal primary states of
high dimension $h$, the lift grew as $\sim\sqrt{h}$. In the present case, we
do not have a closed form expression for the lift for all levels, but the
lifts we have explicitly computed do appear to follow a behavior
$\sim\sqrt{h}$ for large $h$. It will be interesting to find out if this
$\sim\sqrt{h}$ behavior is a general feature of lifts at large dimensions.

## Acknowledgements

We would like to thank Bin Guo for discussions. This work is supported in part
by DOE grant DE-SC0011726.

##  Appendix A The $\mathcal{N}=4$ superconformal algebra

We follow the notation of appendix A of [36]. As described in Section 2.1 the
indices $\alpha=(+,-)$ and $\bar{\alpha}=(+,-)$ correspond to the subgroups
$SU(2)_{L}$ and $SU(2)_{R}$ arising from rotations on $S^{3}$ and the indices
$A=(+,-)$ and $\dot{A}=(+,-)$ correspond to the subgroups $SU(2)_{1}$ and
$SU(2)_{2}$ arising from rotations in $T^{4}$. We use the convention

$\epsilon_{+-}=1\quad,\quad\epsilon^{+-}=-1\ .$ (A.1)

###  A.1 Commutation relations

The commutation relations for the small $\mathcal{N}=4$ superconformal algebra
are

$\displaystyle\big{[}L_{m},L_{n}\big{]}$
$\displaystyle=\frac{c}{12}m(m^{2}-1)\delta_{m+n,0}+(m-n)L_{m+n}\ ,$ (A.2a)
$\displaystyle\big{[}J^{a}_{m},J^{b}_{n}\big{]}$
$\displaystyle=\frac{c}{12}m\,\delta^{ab}\delta_{m+n,0}+i\epsilon^{ab}_{\,\,\,\,c}\,J^{c}_{m+n}\
,$ (A.2b)
$\displaystyle\big{\\{}G^{\alpha}_{\dot{A},r},G^{\beta}_{\dot{B},s}\big{\\}}$
$\displaystyle=\epsilon_{\dot{A}\dot{B}}\bigg{[}\epsilon^{\alpha\beta}\frac{c}{6}\Big{(}r^{2}-\frac{1}{4}\Big{)}\delta_{r+s,0}+\big{(}\sigma^{aT}\big{)}^{\alpha}_{\gamma}\>\epsilon^{\gamma\beta}(r-s)J^{a}_{r+s}+\epsilon^{\alpha\beta}L_{r+s}\bigg{]}\
,$ (A.2c) $\displaystyle\big{[}J^{a}_{m},G^{\alpha}_{\dot{A},r}\big{]}$
$\displaystyle={\frac{1}{2}}\big{(}\sigma^{aT}\big{)}^{\alpha}_{\beta}\,G^{\beta}_{\dot{A},m+r}\
,$ (A.2d) $\displaystyle\big{[}L_{m},G^{\alpha}_{\dot{A},r}\big{]}$
$\displaystyle=\Big{(}\frac{m}{2}-r\Big{)}G^{\alpha}_{\dot{A},m+r}\ ,$ (A.2e)
$\displaystyle\big{[}L_{m},J^{a}_{n}\big{]}$ $\displaystyle=-nJ^{a}_{m+n}\ ,$
(A.2f)

where $\sigma^{aT}$ are the transpose of the Pauli sigma matrices and the
right-moving modes satisfy an analogous set of relations. A different basis
for the $J$ currents is also often used; instead of $J^{a}$ we have
$J^{3},J^{\pm}$ where

$J^{\pm}=J^{1}\pm iJ^{2}\ .$ (A.3)

We will not have need for the full contracted large $\mathcal{N}=4$
superconformal algebra of the D1-D5 CFT, but this can nonetheless be found in
Appendix A of [34] or Appendix A.4 of [36] with the same conventions. We do,
however, give our conventions for the realisation of part of this algebra in
terms of the free fermions $\psi^{\alpha A}$ with modes $d^{\alpha A}_{r}$,
and the free bosons $\partial X_{\\!A\dot{A}}$ with modes
$\alpha_{A\dot{A},n}$. The mode expansions of the fields are given by

$\displaystyle\partial X_{A\dot{A}}(z)$
$\displaystyle=-i\sum_{n}z^{-n-1}\,\alpha_{A\dot{A},n}\ ,$ (A.4a)
$\displaystyle\psi^{\alpha A}(z)$ $\displaystyle=\sum_{r}z^{-r-1/2}\,d^{\alpha
A}_{r}\ ,$ (A.4b)

and likewise the inverse relations are

$\displaystyle\alpha_{A\dot{A},n}$ $\displaystyle=i\oint\frac{dz}{2\pi
i}z^{n}\partial X_{A\dot{A}}(z)\ ,$ (A.5a) $\displaystyle d^{\alpha A}_{s}$
$\displaystyle=\oint\frac{dz}{2\pi i}z^{n-\frac{1}{2}}\psi^{\alpha A}(z)\ .$
(A.5b)

The brackets of the $\alpha$ and $d$ modes are

$\displaystyle\big{[}\alpha_{A\dot{A},n},\alpha_{B\dot{B},m}\big{]}$
$\displaystyle=-n\frac{c}{6}\epsilon_{AB}\,\epsilon_{\dot{A}\dot{B}}\,\delta_{n+m,0}\
,$ (A.6a) $\displaystyle\big{\\{}d^{\alpha A}_{r},d^{\beta B}_{s}\big{\\}}$
$\displaystyle=-\frac{c}{6}\epsilon^{\alpha\beta}\epsilon^{AB}\delta_{r+s,0}\
,$ (A.6b) $\displaystyle\big{[}\alpha_{A\dot{A},n},d^{\beta B}_{s}\big{]}$
$\displaystyle=0\ ,$ (A.6c)

and likewise for the right-moving fields. The commutators of currents with the
$\alpha$ and $d$ modes are

$\displaystyle\big{[}L_{n},\alpha_{A\dot{A},m}\big{]}$
$\displaystyle=-m\alpha_{A\dot{A},m_{n}}\quad\ ,\qquad\big{[}L_{n},d^{\alpha
A}_{s}\big{]}=-\Big{(}\frac{n}{2}+s\Big{)}d^{\alpha A}_{n+s}\ ,$ (A.7a)
$\displaystyle\big{[}J^{a}_{n},\alpha_{A\dot{A},n}\big{]}$
$\displaystyle=0\quad\ ,\qquad\big{[}J^{a}_{n},d^{\alpha
A}_{s}\big{]}=\frac{1}{2}\big{(}\sigma^{Ta}\big{)}^{\alpha}_{\
\beta}\,d^{\beta A}_{s+n}\ ,$ (A.7b)
$\displaystyle\big{[}G^{\alpha}_{\dot{A},s},\alpha_{B\dot{B},n}\big{]}$
$\displaystyle=-in\epsilon_{AB}\epsilon_{\dot{A}\dot{B}}d^{\alpha
A}_{s+n}\quad\ ,\qquad\big{\\{}G^{\alpha}_{\dot{A},r},d^{\beta
B}_{s}\big{\\}}=i\epsilon^{\alpha\beta}\epsilon^{AB}\alpha_{A\dot{A},r+s}\ .$
(A.7c)

The 2-point functions for the free bosonic and fermionic fields are

$\displaystyle\langle\partial X_{A\dot{A}}(z_{1})\partial
X_{B\dot{B}}(z_{2})\rangle$
$\displaystyle=\frac{\epsilon_{AB}\epsilon_{\dot{A}\dot{B}}}{(z_{1}-z_{2})^{2}}\
,$ (A.8a) $\displaystyle\langle\psi^{\alpha A}(z_{1})\psi^{\beta
B}(z_{2})\rangle$
$\displaystyle=-\frac{\epsilon^{\alpha\beta}\epsilon^{AB}}{z_{1}-z_{2}}\ .$
(A.8b)

In terms of these free bosons and fermions the currents are given as

$\displaystyle J^{a}$
$\displaystyle=\frac{1}{4}\epsilon_{\alpha\gamma}\epsilon_{AC}\,\psi^{\gamma
C}\big{(}\sigma^{aT}\big{)}^{\alpha}_{\ \beta}\psi^{\beta A}\ ,$ (A.9a)
$\displaystyle G^{\alpha}_{\dot{A}}$ $\displaystyle=\psi^{\alpha A}\partial
X_{A\dot{A}}\ ,$ (A.9b) $\displaystyle T$
$\displaystyle=\frac{1}{4}\epsilon^{AB}\epsilon^{\dot{A}\dot{B}}\partial
X_{B\dot{B}}\partial
X_{A\dot{A}}+\frac{1}{2}\epsilon_{\alpha\beta}\epsilon_{AB}\psi^{\beta
B}\partial\psi^{\alpha A}\ ,$ (A.9c)

with $a=1,2,3$ being an $SU(2)$ vector representation index.

###  A.2 Hermitian conjugation

Suppose we consider an amplitude on the cylinder, in the NS sector, with the
form

$A={}_{\scriptscriptstyle{N\\!S}}\langle
0|\,{\mathcal{O}}^{\dagger}\big{(}\tau=T,\sigma=0\big{)}{\mathcal{O}}\big{(}\tau=-T,\sigma=0\big{)}|0\rangle_{\scriptscriptstyle{N\\!S}}\
.$ (A.10)

Then we should have $A\geq 0$. This requirement helps determine the way
Hermitian conjugates are defined in our CFT. Note that contractions between
$su(2)$ indices are done using antisymmetric tensors like
$\epsilon_{\alpha\beta}$, and this fact gives rise to certain negative signs
in the definitions of Hermitian conjugates. For the supercharges, we use the
following rules

$\displaystyle\Big{(}G^{+}_{+}(\tau,\sigma)\Big{)}^{\dagger}$
$\displaystyle=-G^{-}_{-}(-\tau,\sigma)\quad,\quad\Big{(}G^{+}_{-}(\tau,\sigma)\Big{)}^{\dagger}=G^{-}_{+}(-\tau,\sigma)\
,$ (A.11) $\displaystyle\Big{(}\bar{G}^{+}_{+}(\tau,\sigma)\Big{)}^{\dagger}$
$\displaystyle=-\bar{G}^{-}_{-}(-\tau,\sigma)\quad,\quad\Big{(}\bar{G}^{+}_{-}(\tau,\sigma)\Big{)}^{\dagger}=\bar{G}^{-}_{+}(-\tau,\sigma)\
,$

while for the degree-2 twist operators, our conventions are

$\big{(}\sigma^{--}(\tau,\sigma)\big{)}^{\dagger}=-\sigma^{++}(-\tau,\sigma)\quad\
,\qquad\big{(}\sigma^{-+}(\tau,\sigma)\big{)}^{\dagger}=\sigma^{+-}(-\tau,\sigma)\
.$ (A.12)

Likewise, in terms of the free fields of the orbifold theory the conjugation
conventions we use are

$\displaystyle\big{(}\alpha_{++,n}\big{)}^{\dagger}$
$\displaystyle=-\alpha_{--,-n}\quad,\quad\big{(}\alpha_{+-,n}\big{)}^{\dagger}=\alpha_{-+,-n}\
,$ (A.13) $\displaystyle\big{(}d^{++}_{s}\big{)}^{\dagger}$
$\displaystyle=-d^{--}_{-s}\quad,\quad\big{(}d^{+-}_{s}\big{)}^{\dagger}=d^{-+}_{-s}\
.$ (A.14)

These conventions ensure that the correlators

$\langle\big{(}\alpha_{A\dot{A},n}\big{)}^{\\!\dagger}\,\alpha_{A\dot{A},n}\rangle=\langle\big{(}d^{\alpha
A,n}\big{)}^{\\!\dagger}\,d^{\alpha A,n}\rangle=1\ ,$ (A.15)

for using the brackets (A.6).

##  Appendix B Mapping amplitude relations to lift relations

In Section 4 we found interesting relations between amplitudes of certain
states and those of their descendants (see (4.7) and (4.15)). Here we describe
how to map those amplitude relations to relations between the lifts of states.
Consider the following simplified form of the definition of lift given in
(2.15)

$\displaystyle E^{(2)}(\phi)=\kappa\,\frac{A(\phi)}{\bra{\phi}\ket{\phi}}\ ,$
(B.1)

where $\kappa$ is a constant and $A(\phi)$ is an amplitude related to (2.16)
with initial and final states being $\phi$. Taking, for instance, the
amplitude relation

$A(\phi)=2h_{\psi}A(\psi)\ ,$ (B.2)

obtained in Section 4.1 by considering the states $|\psi\rangle$ and
$\ket{\phi}=L_{-1}\ket{\psi}$ and using (B.1) we find that at the level of
lifts

$E^{(2)}(\phi)=2h_{\psi}\frac{{\bra{\psi}\ket{\psi}}}{{\bra{\phi}\ket{\phi}}}\,E^{(2)}(\psi)\
.$ (B.3)

Since the norm of the descendant state is given by

$\langle\phi|\phi\rangle=\langle\psi|L_{1}L_{-1}|\psi\rangle=2\langle\psi|L_{0}|\psi\rangle=2h_{\psi}\langle\psi|\psi\rangle\
,$ (B.4)

we can simplify (B.3) to give the lifting relation

$E^{(2)}(\phi)=E^{(2)}(\psi)\ ,$ (B.5)

i.e. that the lift of a state satisfying $L_{1}|\psi\rangle=0$ is equal to the
lift of its $L_{-1}$ descendant. Equally one can start with the amplitude
relation

$A^{\beta\alpha}_{\dot{B}\dot{A}}(\widetilde{\phi})=K^{\beta\alpha}_{\dot{B}\dot{A}}A(\widetilde{\psi})\
,$ (B.6)

with $K^{\beta\alpha}_{\dot{B}\dot{A}}$ defined in (4.16) obtained by
considering the states $|\widetilde{\psi}\,\rangle$ and
$|\widetilde{\phi}\,\rangle^{\alpha}_{\dot{A}}=G^{\alpha}_{\dot{A},-\frac{1}{2}}|\widetilde{\psi}\,\rangle$.
This can be recast in terms of lifts using (B.1) and imposing that the index
$\beta$ is opposite to $\alpha$ and $\dot{B}$ opposite to $\dot{A}$, giving

$E^{(2)}\big{(}\widetilde{\phi}^{\,\alpha}_{\dot{A}}\big{)}=K^{\alpha}_{\\!\dot{A}}\>\frac{\langle\widetilde{\psi}\,|\widetilde{\psi}\,\rangle}{{}^{\,\alpha}_{\dot{A}}\langle\widetilde{\phi}\,|\widetilde{\phi}\,\rangle^{\alpha}_{\dot{A}}}\,E^{(2)}\big{(}\widetilde{\psi}\big{)}\
,$ (B.7)

where there is no implied summation over the repeated indices $\alpha$ and
$\dot{A}$ and $K^{\alpha}_{\dot{A}}$ is defined as
$K^{\beta\alpha}_{\dot{B}\dot{A}}$ with the above conditions imposed on
$\beta$ and $\dot{B}$. Since the norm of the descendant state is given by
(there no sum over repeated indices)

${}^{\,\alpha}_{\dot{A}}\langle\widetilde{\phi}\,|\widetilde{\phi}\,\rangle^{\alpha}_{\dot{A}}=\langle\widetilde{\psi}\,|\big{(}G^{\alpha}_{\dot{A},-\frac{1}{2}}\big{)}^{\\!\dagger}G^{\alpha}_{\dot{A},-\frac{1}{2}}|\widetilde{\psi}\,\rangle=K^{\alpha}_{\dot{A}}\langle\widetilde{\psi}\,|\widetilde{\psi}\,\rangle\
,$ (B.8)

we can simplify (B.7) to give the lifting relation

$E^{(2)}(\widetilde{\phi}^{\alpha}_{\dot{A}})=E^{(2)}(\widetilde{\psi})\ ,$
(B.9)

i.e. that the lift of a state satisfying
$\big{(}G^{\alpha}_{\dot{A},-\frac{1}{2}}\big{)}^{\\!\dagger}|\widetilde{\psi}\,\rangle=0$
is equal to the lift of its $G^{\alpha}_{\dot{A},-\frac{1}{2}}$ descendant.

Given a chosen basis of states for which the lift is being computed, i.e. the
set used in this paper

$\bigg{\\{}\,|\alpha\alpha\rangle_{B\dot{B}A\dot{A}(m,n)}\ ,\ \ |\alpha
d\rangle^{\alpha A}_{B\dot{B}(n,s)}\ ,\ \ |dd\rangle^{\beta B\alpha
A}_{r,s}\,\bigg{\\}}\ ,$ (B.10)

the descendant states $|\phi\rangle$ and
$|\widetilde{\phi}\,\rangle^{\alpha}_{\dot{A}}$ can generally be written as a
sum of basis states. The lift of the descendant state (the left-hand side of
the relations (B.5) and (B.9)) can not then be immediately written in terms of
the lifts of basis states from (B.10), which are the data points computed in
Section 3.

In the case that the descendant state is a sum of two basis states, i.e. if we
have

$|\phi\rangle=|\phi_{1}\rangle+|\phi_{2}\rangle\ ,$ (B.11)

where $|\phi_{1}\rangle$ and $|\phi_{2}\rangle$ are states in the set (B.10),
then we have

$\displaystyle E^{(2)}(\phi)$ $\displaystyle=E^{(2)}(\phi_{1}+\phi_{2})$
$\displaystyle=\kappa\,\frac{A(\phi_{1}+\phi_{2})}{\langle\phi_{1}+\phi_{2}|\phi_{1}+\phi_{2}\rangle}$
$\displaystyle=\kappa\,\frac{A(\phi_{1})+A(\phi_{2})+2A(\phi_{1};\phi_{2})}{\langle\phi_{1}|\phi_{1}\rangle+\langle\phi_{2}|\phi_{2}\rangle+2\langle\phi_{1}|\phi_{2}\rangle}$
$\displaystyle=\frac{\langle\phi_{1}|\phi_{1}\rangle
E^{(2)}(\phi_{1})+\langle\phi_{2}|\phi_{2}\rangle
E^{(2)}(\phi_{2})+2\sqrt{\langle\phi_{1}|\phi_{1}\rangle\langle\phi_{2}|\phi_{2}\rangle}\,E^{(2)}(\phi_{1};\phi_{2})}{\langle\phi_{1}|\phi_{1}\rangle+\langle\phi_{2}|\phi_{2}\rangle+2\langle\phi_{1}|\phi_{2}\rangle}\
.$ (B.12)

Here $A(\phi_{f};\phi_{i})$ is a generalisation of the integrated amplitude
$A(\phi)$ used in lifting computations where the initial and final states are
not necessarily equal, with $A(\phi_{i};\phi_{i})=A(\phi_{i})$ and
$E^{(2)}(\phi_{f};\phi_{i})$ is the associated lift-like quantity defined
analogously to (B.1) as

$\displaystyle
E^{(2)}(\phi_{f};\phi_{i})\equiv\kappa\,\frac{A(\phi_{f};\phi_{i})}{\sqrt{\bra{\phi_{f}}\ket{\phi_{f}}\bra{\phi_{i}}\ket{\phi_{i}}}}\
.$ (B.13)

Thus in the case that the descendant state splits as (B.11) then (B) can be
used as the left-hand side of the relations (B.5) and (B.9). This argument can
easily be generalised to the case of
$|\phi\rangle=\sum_{k=1}^{K}|\phi_{k}\rangle$ with the states
$|\phi_{k}\rangle$ being in the set (B.10).

## References

  * [1] A. Strominger and C. Vafa, _Microscopic origin of the Bekenstein-Hawking entropy_ , _Phys. Lett. B_ 379 (1996) 99 [hep-th/9601029].
  * [2] J.M. Maldacena, G.W. Moore and A. Strominger, _Counting BPS black holes in toroidal Type II string theory_ , hep-th/9903163.
  * [3] I. Bena, E.J. Martinec, S.D. Mathur and N.P. Warner, _Snowmass White Paper: Micro- and Macro-Structure of Black Holes_ , 2203.04981.
  * [4] I. Bena, E.J. Martinec, S.D. Mathur and N.P. Warner, _Fuzzballs and Microstate Geometries: Black-Hole Structure in String Theory_ , 2204.13113.
  * [5] O. Lunin and S.D. Mathur, _AdS / CFT duality and the black hole information paradox_ , _Nucl. Phys. B_ 623 (2002) 342 [hep-th/0109154].
  * [6] S.D. Mathur, _The Fuzzball proposal for black holes: An Elementary review_ , _Fortsch. Phys._ 53 (2005) 793 [hep-th/0502050].
  * [7] I. Kanitscheider, K. Skenderis and M. Taylor, _Fuzzballs with internal excitations_ , _JHEP_ 06 (2007) 056 [0704.0690].
  * [8] I. Bena and N.P. Warner, _Black holes, black rings and their microstates_ , _Lect. Notes Phys._ 755 (2008) 1 [hep-th/0701216].
  * [9] B.D. Chowdhury and A. Virmani, _Modave Lectures on Fuzzballs and Emission from the D1-D5 System_ , in _5th Modave Summer School in Mathematical Physics_ , 1, 2010 [1001.1444].
  * [10] M. Shigemori, _Superstrata_ , _Gen. Rel. Grav._ 52 (2020) 51 [2002.01592].
  * [11] C. Vafa, _Instantons on D-branes_ , _Nucl. Phys. B_ 463 (1996) 435 [hep-th/9512078].
  * [12] R. Dijkgraaf, _Instanton strings and hyperKahler geometry_ , _Nucl. Phys. B_ 543 (1999) 545 [hep-th/9810210].
  * [13] F. Larsen and E.J. Martinec, _U(1) charges and moduli in the D1 - D5 system_ , _JHEP_ 06 (1999) 019 [hep-th/9905064].
  * [14] N. Seiberg and E. Witten, _The D1 / D5 system and singular CFT_ , _JHEP_ 04 (1999) 017 [hep-th/9903224].
  * [15] G.E. Arutyunov and S.A. Frolov, _Four graviton scattering amplitude from S**N R**8 supersymmetric orbifold sigma model_ , _Nucl. Phys. B_ 524 (1998) 159 [hep-th/9712061].
  * [16] G.E. Arutyunov and S.A. Frolov, _Virasoro amplitude from the S**N R**24 orbifold sigma model_ , _Theor. Math. Phys._ 114 (1998) 43 [hep-th/9708129].
  * [17] A. Jevicki, M. Mihailescu and S. Ramgoolam, _Gravity from CFT on S**N(X): Symmetries and interactions_ , _Nucl. Phys. B_ 577 (2000) 47 [hep-th/9907144].
  * [18] J.R. David, G. Mandal and S.R. Wadia, _Microscopic formulation of black holes in string theory_ , _Phys. Rept._ 369 (2002) 549 [hep-th/0203048].
  * [19] J.M. Maldacena, _The Large N limit of superconformal field theories and supergravity_ , _Adv. Theor. Math. Phys._ 2 (1998) 231 [hep-th/9711200].
  * [20] E. Gava and K.S. Narain, _Proving the PP wave / CFT(2) duality_ , _JHEP_ 12 (2002) 023 [hep-th/0208081].
  * [21] C.-M. Chang and Y.-H. Lin, _Words to describe a black hole_ , _JHEP_ 02 (2023) 109 [2209.06728].
  * [22] A.A. Lima, G.M. Sotkov and M. Stanishkov, _Microstate Renormalization in Deformed D1-D5 SCFT_ , _Phys. Lett. B_ 808 (2020) 135630 [2005.06702].
  * [23] A.A. Lima, G.M. Sotkov and M. Stanishkov, _Renormalization of twisted Ramond fields in D1-D5 SCFT 2_, _JHEP_ 03 (2021) 202 [2010.00172].
  * [24] A.A. Lima, G.M. Sotkov and M. Stanishkov, _Correlation functions of composite Ramond fields in deformed D1-D5 orbifold SCFT 2_, _Phys. Rev. D_ 102 (2020) 106004 [2006.16303].
  * [25] A.A. Lima, G.M. Sotkov and M. Stanishkov, _Dynamics of R-neutral Ramond fields in the D1-D5 SCFT_ , 2012.08021.
  * [26] A.A. Lima, G.M. Sotkov and M. Stanishkov, _On the Dynamics of Protected Ramond Ground States in the D1-D5 CFT_ , 2103.04459.
  * [27] A.A. Lima, G.M. Sotkov and M. Stanishkov, _Ramond States of the D1-D5 CFT away from the free orbifold point_ , in _14th International Workshop on Lie Theory and Its Applications in Physics_ , 12, 2021 [2112.10832].
  * [28] A.A. Lima, G.M. Sotkov and M. Stanishkov, _Four-point functions with multi-cycle fields in symmetric orbifolds and the D1-D5 CFT_ , 2202.12424.
  * [29] N. Benjamin, S. Bintanja, A. Castro and J. Hollander, _The stranger things of symmetric product orbifold CFTs_ , _JHEP_ 11 (2022) 054 [2208.11141].
  * [30] B. Guo and S.D. Mathur, _Dynamical evolution in the D1D5 CFT_ , _JHEP_ 12 (2022) 107 [2208.05992].
  * [31] B. Guo and S.D. Hampton, _Bootstrapping the effect of the twist operator in symmetric orbifold CFTs_ , _JHEP_ 02 (2023) 184 [2206.01623].
  * [32] B. Guo and S.D. Hampton, _Bootstrapping multi-wound twist effects in symmetric orbifold CFTs_ , 2307.14255.
  * [33] B. Guo, M.R.R. Hughes, S.D. Mathur and M. Mehta, _Universal lifting in the D1-D5 CFT_ , _JHEP_ 10 (2022) 148 [2208.07409].
  * [34] B. Guo and S.D. Mathur, _Lifting of level-1 states in the D1D5 CFT_ , _JHEP_ 03 (2020) 028 [1912.05567].
  * [35] B. Guo and S.D. Mathur, _Lifting at higher levels in the D1D5 CFT_ , _JHEP_ 11 (2020) 145 [2008.01274].
  * [36] S. Hampton, S.D. Mathur and I.G. Zadeh, _Lifting of D1-D5-P states_ , _JHEP_ 01 (2019) 075 [1804.10097].
  * [37] M.R. Gaberdiel, C. Peng and I.G. Zadeh, _Higgsing the stringy higher spin symmetry_ , _JHEP_ 10 (2015) 101 [1506.02045].
  * [38] N. Benjamin, C.A. Keller and I.G. Zadeh, _Lifting 1/4-BPS states in $AdS_{3}\times S^{3}\times T^{4}$_, _JHEP_ 10 (2021) 089 [2107.00655].
  * [39] G. Giribet, C. Hull, M. Kleban, M. Porrati and E. Rabinovici, _Superstrings on AdS 3 at $\mathcal{k}=$ 1_, _JHEP_ 08 (2018) 204 [1803.04420].
  * [40] M.R. Gaberdiel and R. Gopakumar, _Tensionless string spectra on AdS 3_, _JHEP_ 05 (2018) 085 [1803.04423].
  * [41] L. Eberhardt, M.R. Gaberdiel and R. Gopakumar, _The Worldsheet Dual of the Symmetric Product CFT_ , _JHEP_ 04 (2019) 103 [1812.01007].
  * [42] L. Eberhardt, M.R. Gaberdiel and R. Gopakumar, _Deriving the AdS 3/CFT2 correspondence_, _JHEP_ 02 (2020) 136 [1911.00378].
  * [43] L. Eberhardt, _AdS 3/CFT2 at higher genus_, _JHEP_ 05 (2020) 150 [2002.11729].
  * [44] L. Eberhardt, _A perturbative CFT dual for pure NS-NS AdS 3 strings_, 2110.07535.
  * [45] L. Eberhardt and M.R. Gaberdiel, _String theory on AdS 3 and the symmetric orbifold of Liouville theory_, _Nucl. Phys. B_ 948 (2019) 114774 [1903.00421].
  * [46] A. Dei, L. Eberhardt and M.R. Gaberdiel, _Three-point functions in AdS 3/CFT2 holography_, _JHEP_ 12 (2019) 012 [1907.13144].
  * [47] A. Brollo, D. le Plat, A. Sfondrini and R. Suzuki, _The Tensionless Limit of Pure-Ramond-Ramond AdS3/CFT2_ , 2303.02120.
  * [48] A. Brollo, D. le Plat, A. Sfondrini and R. Suzuki, _More on the tensionless limit of pure-Ramond-Ramond AdS3/CFT2_ , 2308.11576.
  * [49] A. Schwimmer and N. Seiberg, _Comments on the N=2, N=3, N=4 Superconformal Algebras in Two-Dimensions_ , _Phys. Lett. B_ 184 (1987) 191.
  * [50] A. Sevrin, W. Troost and A. Van Proeyen, _Superconformal Algebras in Two-Dimensions with N=4_ , _Phys. Lett. B_ 208 (1988) 447.
  * [51] M.R. Gaberdiel and R. Gopakumar, _Stringy Symmetries and the Higher Spin Square_ , _J. Phys. A_ 48 (2015) 185402 [1501.07236].
  * [52] B. Guo and S. Hampton, _Partial Spectral Flow in the D1D5 CFT_ , 2112.10573.
  * [53] O. Lunin and S.D. Mathur, _Metric of the multiply wound rotating string_ , _Nucl. Phys. B_ 610 (2001) 49 [hep-th/0105136].
  * [54] O. Lunin and S.D. Mathur, _Correlation functions for M**N / S(N) orbifolds_ , _Commun. Math. Phys._ 219 (2001) 399 [hep-th/0006196].


# Open Data Fabric: A Decentralized Data Exchange and Transformation Protocol
With Complete Reproducibility and Provenance

Sergii Mikhtoniuk  Kamu Data Inc.VancouverCanada<EMAIL_ADDRESS>and
Özge Nilay Yalçın  University of British ColumbiaVancouverCanada
<EMAIL_ADDRESS>
###### Abstract.

Data is the most powerful decision-making tool at our disposal. However,
despite the exponentially growing volumes of data generated in the world,
putting it to effective use still presents many challenges. Relevant data
seems to be never there when it is needed - it remains siloed, hard to find,
hard to access, outdated, and of bad quality. As a result, governments,
institutions, and businesses remain largely impaired in their ability to make
data-driven decisions. At the same time, data science is undergoing a
reproducibility crisis. The results of the vast majority of studies cannot be
replicated by other researchers, and provenance often cannot be established,
even for data used in medical studies that affect lives of millions. We are
losing our ability to collaborate at a time when significant improvements to
data are badly needed.

We believe that the fundamental reason lies in the modern data management
processes being entirely at odds with the basic principles of collaboration
and trust. Our field needs a fundamental shift of approach in how data is
viewed, how it is shared and transformed. We must transition away from
treating data as static, from exchanging it as anemic binary blobs, and
instead focus on making multi-party data management more sustainable: such as
reproducibility, verifiability, provenance, autonomy, and low latency. In this
paper, we present the Open Data Fabric, a new decentralized data exchange and
transformation protocol designed from the ground up to simplify data
management and enable collaboration around data on a similar scale as
currently seen in open-source software.

##  1\. Introduction

Modern data science and data engineering are rapidly advancing fields that
feed into many other disciplines like medical and social sciences, where
manipulating large datasets has long become the norm for many researchers.
However, there is a growing number of concerns in those communities about the
sustainability of these advancements, specifically around reproducibility and
data provenance issues (Miyakawa, 2020; Baker, 2016).

These issues are not new in the science community (Marx, 2013) but became much
more evident during the COVID-19 pandemic. Research efforts in the face of a
global emergency required efficient collaboration around rapidly growing
datasets under constant time pressure and increased attention and scrutiny
(Baker et al., 2020). Under these conditions maintaining reproducibility of
the results, which is currently an increasingly labor intensive process, was
hardly a priority. A striking consequence of this, however, was the alarming
retraction rate of the COVID-related publications from peer-reviewed journals
(Nicole Shu Ling and Bor Luen, 2020).

One such example is the hydroxychloroquine study (Mehra et al., 2020) that was
published in the influential journal, The Lancet, and claimed to use data of
more than 96,000 COVID-19 patients in 671 hospitals worldwide. After the
publication, the journal received a flood of concerns about the accuracy of
results as some basic numbers were not checking out. The publication was later
retracted when some of the included hospitals claimed that they had no
arrangements to supply such data to anyone and a growing realization that
provenance of source data could not be established. By that time, however, the
damage was done - a data provenance issue that was left unchecked has
significantly disrupted the life-saving efforts, derailed many other studies,
and spread even more confusion at this critical time. The increasing awareness
of these issues led to multiple action calls from the scientific community to
create better collaboration platforms and manage data in a reliable way
(Zahuranec, 2020; The White House, 2020; Group, 2020).

We can easily identify multiple issues in modern data supply chains that
contribute to these problems: publishing source data in non-machine-readable
formats, data siloing, poor data quality, heterogeneous formats, poor
infrastructure, lack of publisher incentives, missing feedback loop between
publishers and consumers, etc. These issues make data consumers spend
disproportionate amounts of time and resources on obtaining data and getting
it into a usable state. The fallacy here is that since no mechanism currently
exists to make such improvements reproducible and verifiable, this entire
process creates yet another dataset that is entirely disjointed from its
sources. This issue is present in every cycle where data is downloaded from
the trusted source, modified, and then re-published. The amount of time it
takes for another researcher to prove the validity of such dataset is often
comparable to re-doing the whole work from scratch. Therefore, while any
respectable research project starts with data from trusted data sources, it
always ends up producing data that cannot be readily trusted and reused.

In this paper, we will take a detailed look at the prerequisites for building
trust and collaboration and how we built the Open Data Fabric protocol to
satisfy them.

##  2\. Design Considerations

Figure 1. Root and derivative dataset compose a data flow graph.

Open Data Fabric (ODF) (Kamu Data Inc., 2020b) is an open protocol
specification for decentralized exchange and transformation of semi-structured
data that aims to holistically address many shortcomings of the modern data
management systems and workflows. Our goal is to develop a method of data
exchange that would:

  * •

Address the problems of reproducibility, verifiability, and provenance in
modern data supply chains.

  * •

Create an environment of verifiable trust between participants without the
need for a central authority.

  * •

Enable worldwide collaboration around data cleaning, enrichment, and
derivation.

  * •

Achieve a high degree of data reuse, making quality data readily available.

  * •

Improve the liquidity of data by speeding up the data propagation times from
publishers to consumers.

  * •

Close the feedback loop between data consumers and publishers, allowing them
to collaborate on better data availability, recency, and design.

In this section, we will show how these goals were incorporated into the
system’s design.

###  2.1. Data Flow

Raw data is rarely consumed in its original form - it often needs to go
through a series of transformation, aggregation, and enrichment steps before
it can be acted upon. In ODF, we make a clear distinction between the
following two types of data:

  * •

Source data, represented in ODF by Root Datasets, comes directly from external
systems. The organization that publishes such data has complete authority over
it and is fully accountable for its veracity.

  * •

Derivative data, represented in ODF by Derivative Data-sets is produced by
transforming and combining other data. It is secondary to the source data but
is equally important since derivative data is what’s being presented to the
decision-makers, used for training models, or fed into various automation.

The multi-stage process through which source data is transformed into
actionable (derivative) data has many forms. It can manifest itself as
reporting chains where people perform recurrent analysis and summarization
tasks (e.g. payroll, company management reports), or as complex automated
workflows (e.g. enterprise data pipelines). In ODF, we find it more intuitive
and effective to reason about these processes as computational graphs (see
Figure 1), where data continuously flows from its sources to consumers via
multiple transformation stages.

###  2.2. The Foundation of Collaboration

As shown previously, verifying the validity of derivative data can be cost-
prohibitive compared to re-doing the work from scratch. This is the reason why
complex multi-stage data processing chains mainly exist within the boundaries
of organizations, where employees can implicitly trust one-another.
Collaboration on data between the organizations today often leads to
centralization - creation of data spaces where access control, audit and user
privileges can be enforced. But centralization is time-consuming, expensive,
and often impossible, as different parties want to maintain full control and
ownership of their data.

Collaboration on data in a decentralized setting, such as research, remains
unsolved. The validity of source data used in publications is often left
unchecked during the review process, and the issue of trust hinges mostly on
the reputation of the authors or the institution they represent. This may
unfairly penalize young researchers and lesser known universities when it
comes to publishing their work.

It is therefore ODF’s goal to reduce the time it takes consumers to establish
validity of derivative data down to minutes and enable collaboration on data
in decentralized environments where complete trust between participants is not
possible.

A modern epitome of effective collaboration between independent parties is the
Open-Source Software ecosystem. What started as an exchange of ideas within a
close-knit community quickly grew into a worldwide movement. The emergence of
distributed version control systems (DVCS) (Otte, 2009) ensured that the
collaboration could function even at a rapidly growing scale. Many recently
developed data management systems try to apply some of the ideas behind DVCS,
like diff-based history and branching, directly to data (Liquidata Inc.,
[n.d.]; Armbrust et al., 2020). Instead of borrowing specific technical
decisions, however, in ODF we tried to understand what properties of DVCS made
them so good when it comes to collaboration. We think the answer is that they
help to build trust. They can build trust even between people who have never
met before, who may not even share a common language. They do so via
properties like tracking the complete history of changes, allowing to
attribute any change to its author, to safely roll back any previous change,
and by making any malicious behavior easy to expose and remedy.

Applying these ideas to data, we have identified three pillars that would
build trust and are essential to collaboration around data:

  * •

Reproducibility \- the ability to reproduce the results is the cornerstone of
the scientific method without which the process and findings of one party
cannot be followed by others.

  * •

Verifiability \- when considering to use some data for a project it is
essential to be able to understand whether it comes from a reliable publisher
and whether it truthfully declares which transformations it underwent.

  * •

Provenance \- regardless of how many transformation stages the data went
through, it’s important to be able to trace any individual value back to its
source and understand what data contributed to its existence and its value.

###  2.3. Reproducibility and Verifiability in Source Data

The path towards better reproducibility and verifiability has to start at the
source. The requirements here are very simple: two different parties at
different times should be able to access the exact same data and validate that
this data comes unaltered from the trusted source. Surprisingly, a majority of
data sources today fail to satisfy these basic requirements.

Consider for example GeoNames 111GeoNames. https://www.geonames.org/ and
NaturalEarth 222NaturalEarth. http://naturalearthdata.com/ \- a few of the
major publishers of open GIS data. On a periodic basis they provide datasets
containing the latest known state of their domain. These state ”snapshots” are
published destructively by overwriting all the previous data. Naturally,
everyone who uses the same URL to download data from these sources at
different times is likely to get somewhat different data.

Another example is the NYC Open Data Catalog 333NYC Open Data.
https://opendata.cityofnewyork.us/, which includes some datasets with a full
history of certain events. Such ”ledger” only grows over time as new events
get appended. This approach is much better than snapshotting as it never loses
data, however, it is still quite hard for two parties to obtain the same data.
Whoever downloads data first is likely to have a subset of the ledger obtained
by one who downloads data later. Consumers have to rely on other means to
coordinate which subset of the ledger they will be using (e.g. counting rows,
using timestamps, record identifiers), which is highly unreliable (see Section
3.4).

The problems of reproducibility and verifiability of modern data start at the
source data, as the bad practices like state snapshots and destructive updates
are considered to be the norm for many major data publishers today. For our
purposes however we have identified two crucial properties we’d like to
achieve:

  * •

It must be possible to obtain a stable reference to data that can be shared
between parties and used to obtain the same data at any future point in time.

  * •

Data source has to provide a mechanism to ensure that data obtained this way
was not maliciously or accidentally altered.

One of the most prevalent strategy for achieving these properties today is to
copy the entire dataset from the source onto a durable storage and assign it a
version or a unique identifier. This approach is very common in the enterprise
data science and popularized by data management tools like Quilt 444Quilt.
https://quiltdata.com/ and DVC 555DVC. https://dvc.org/. While it may work
well in the closed, trusted environments such as enterprises, this approach is
not suitable for a distributed setting, when working with open data, and with
fast-moving data sources. Once copied, versioned snapshots essentially become
fully independent datasets since no mechanism exists to reliably link them to
the trusted source. Such copies contribute to the overall noise, only
exacerbate the problem of provenance, and should be avoided.

Similar issues arise in many modern attempts to create various data hubs and
data portals, such as Dataverse 666Dataverse. https://dataverse.org/,
DataWorld 777DataWorld. https://data.world/, Knoema 888Knoema.
https://knoema.com/, and increasingly popular data sharing on platforms like
Kaggle 999Kaggle. https://www.kaggle.com/datasets and GitHub 101010GitHub.
https://github.com/. While the goal of simplifying discovery and federating
data is noble, without a mechanism to link that data back to the trusted
source all they do in fact is create more disjoint and non-trustworthy
datasets.

To satisfy these properties in ODF, we changed our perspective on what ”data”
means to us, and make its definition more strict.

###  2.4. Data Model

##### Events

ODF was primarily designed for mission-critical data. When data is used to
gain insight and drive decision-making, discarding or modifying data is akin
to rewriting history. The history of all data observed by the system must be
preserved.

In the ODF, data is treated as a ledger of historical records. History only
grows, it is never deleted or altered - thus all data that gets into the
system is immutable. ODF treats all records in data as events or relational
propositions that were believed to be true at a specific time. This view
entirely rejects the idea of storing state snapshot data since it lacks the
necessary properties. Building on the ideas of Event Sourcing (Fowler, 2005)
and Stream-Table Duality (Noll, 2020) we treat state data as a byproduct of
history, which can always be reconstructed by projecting the events onto the
time axis. State data is therefore considered as a simple optimization for
queries that operate with current time projections.

##### Bitemporality

Storing historical events alone is not enough to implement stable data
references. As we mentioned previously, data consumers could agree to use a
subset of the data history by utilizing some internal properties of the
dataset (e.g. event timestamps, identifiers, or record counting), but it’s
highly error-prone.

Imagine a scenario where two parties agree to use the event timestamps to
delimit the data and use all events until current time T. The reproducibility
in this case can be compromised by multiple factors, such as the delay it
takes data to appear in the dataset, publisher performing back-fills of the
events for a certain period prior to T, or corrective events with time less
than T being emitted in future upon discovering errors in some old data

To prevent such situations and issue stable references to data, ODF applies
the ideas of Bitemporal Data Modelling (Date et al., 2002; Johnston, 2014) to
the event records. We augment the schema with an extra ”System Time” column,
which contains the time when an event first entered the system. System Time is
guaranteed to be monotonically increasing, so a stable reference to data can
be as simple as having and ID of the dataset and a system time timestamp. Note
that this requirement does not apply for Event Time.

Event time, therefore, tells us when something happened from the outside
world’s perspective, and is usually the most useful one for querying and
joining data. System time, on the other hand, gives us a reference point for
when something has occurred from the perspective of the system and allows us
to establish before-after relationships for data within one dataset and
datasets that are part of the same computation graph.

###  2.5. Reproducibility and Verifiability in Derivative Data

For derivative data, which is obtained by transforming and combining data from
other datasets, reproducibility can be thought of as having an ability to
repeat all transformation steps and obtain the same results as the original.
Conversely, verifiability is an ability to ensure that the data presented to
you as the result of some transformation was produced without being
accidentally or maliciously altered. Verifiability allows us to assess
trustworthiness in two simple steps: ensuring all source data comes from
reliable publishers, and auditing all applied transformations.

From these definitions, we can extract the following requirements: determinism
\- all transformations should be guaranteed to result in the same output given
the same input, and transparency \- all transformations should be known.

These requirements are simple but extremely hard to meet in modern data
science. A typical project can consist of hundreds of moving parts such as
frameworks and libraries, operating systems, and hardware. Most of these
components aren’t purposely built with determinism in mind, so the burden of
achieving reproducibility lies entirely on the person who implements the
project. Achieving determinism is therefore a non-trivial problem that
requires deep understanding of the execution environment and eliminating all
sources of randomness. It is not surprising that such a significant
undertaking is often left out completely to meet the project timelines. As a
result, the modern data science is currently in a state of reproducibility
crisis (Miyakawa, 2020; Baker, 2016).

We believe that there is no way of getting around this problem - data
processing systems have to be built with reproducibility in mind. Until
determinism becomes an intrinsic property of such systems, we employ a series
of techniques to remove as much burden as possible to ensure reproducibility
(see Section 4).

##### Derivative Data Transience

Considering that source data is immutable and all derivative transformations
are deterministic, derivative data of any transformation graph in ODF can be
fully reconstructed by starting from the source data and re-applying all
transformations. The derived data thus can be considered as a form of caching.
As we will discuss later, this approach can potentially reduce overall data
storage costs globally, since such data doesn’t need to be stored durably or
be heavily replicated.

###  2.6. Applying Stream Processing to Historical Data

Data science today is dominated by the batch processing workflows, whose key
characteristic is treating data as a finite set of records. But a significant
portion of data is not static - new data points are constantly being produced
and datasets are continuously updated. The more data-driven we are, the more
we will continue to the push data processing towards real-time speeds.
Applying the same batch processing techniques to recent data, however, is a
harmful oversimplification that exposes us to the multitude of problems
associated with temporal data: data arriving late, arriving out of order,
accounting for corrections that could be issued for data that has been already
processed, misalignment of data arrival cadences between datasets that are
being joined etc. The reliance of batch processing on data completeness often
results in incorrect results, with errors concentrated in recent data - data
everyone cares about the most. What’s worse, as new data arrives batch
workflows may produce different results for the same time periods, effectively
overwriting the history - special care has to be taken to make such
corrections explicit.

It would be practically impossible to write a conventional batch processing
logic that correctly handles all the temporal edge cases on a mass scale. Even
if we would write such logic, its complexity would negate the benefits of
verifiability - what’s the use of being able to audit the transformation code
that is too complex to fully understand?

In the past few years, there has been some major advancements in the field of
streaming data processing. Systems like Google Data Flow (Akidau et al.,
2015), Apache Spark (Zaharia et al., 2016), Apache Flink (Carbone et al.,
2015), and Naiad and its modern implementation Timely Dataflow (Murray et al.,
2013) have developed a great apparatus for dealing with many of these problems
in a very intuitive way. Stream processing paradigm acknowledges a simple fact
- that data processing is a balancing act between latency and correctness. It
gives user the power to control this trade-off in a fully automated way.

Figure 2. Bitemporality and watermarks in event streams

One of the key mechanisms of stream processing is called the watermark.
Watermark is a type of metadata that flows along regular data in the stream
and tells the processing system that at certain system time $Ts$ with a high
probability $P$ the system has observed all events prior to event time $Te$.
The Figure 2 represents the watermark as a curve on a bitemporal event diagram
that separates data that is late by an expected amount and data that is
exceptionally late. Watermarks can be predictive (a system can constantly
adjust it based on the observed difference between event and system time), or
it can be set manually (e.g. an owner of the dataset can ”hint” all data
consumers that the data for the time period prior $Te$ has been fully
entered). Using watermarks a stream processing system can delay the processing
by the exact amount needed to handle out-of-order and late data and let users
perform computations in the event time space, which is a lot more natural and
easier to reason about than the arrival time. The exceptional cases of late
data and backfills can also be dealt with automatically and explicitly by
issuing correction or retraction events or recording that certain data points
were ignored. All this makes stream processing a lot more autonomous,
reliable, and composable than batch.

Conventionally, stream processing is employed to build highly responsive
systems that process near real-time data. In ODF, however, we saw many
benefits in applying the semantics of stream processing to historical data,
even data that is updated very infrequently. Every dataset in ODF is treated
as a potentially infinite stream of events. Expanding on the idea that batch
processing is a special case of stream processing (Tzoumas, 2015), we use
stream processing as our primary data transformation method.

ODF does not prescribe any specific language or framework and aims to support
multiple implementations. Our first two prototype transformation engines,
which we will cover in Section 4.3, use streaming dialects of SQL. An example
streaming SQL query that is using stream-to-stream anti-join to detect late
shipments can be seen below:

⬇

1SELECT o.order_time, o.order_id

2FROM orders as o

3LEFT JOIN shipments as s

4 ON o.order_id = s.order_id

5 AND s.shipment_time BETWEEN

6 o.order_time AND o.order_time + INTERVAL ’1’ WEEK

7WHERE s.shipment_id = NULL

The benefits of this approach include:

  * •

Users can define a query once and potentially run it forever. This allows us
to minimize the latency with which data propagates through the system.

  * •

Streaming queries are expressive and are closer to ”which question is being
asked” as opposed to ”how to compute the result”. They are usually much more
concise than equivalent batch queries.

  * •

Queries can be expressed in a way that is agnostic of how and how often the
new data arrives. Whether the data is ingested once a month in Gigabyte
batches, in micro-batches every hour, or as a true near real-time stream -
processing logic can stay the same, produce the same results, and guarantee
the best propagation times possible.

  * •

Streaming queries are declarative, while batch processing is usually
imperative. Declarative nature lets us perform static analysis of queries and
automatically derive provenance in many cases without tracking any extra data.

  * •

High-level abstractions like windowing, watermarks, and triggers allow users
to produce maximally correct results within the configurable latency window,
preventing cascading error effects typically seen in similar batch workflows.

Figure 3. Metadata captures all dataset life-cycle events

The property of low latency that emerges from the described combination of our
data model and stream processing is worth highlighting. Currently, we often
see situations where critical data (e.g. employment situation reports, or
COVID-19 cases reports in the early days of pandemic) is released so
infrequently that it always has a dramatic effect on the stock markets, or
prompts over-corrective actions from leaders and governments. We believe that
the prevalence of batch workflows is a major contributing factor that adds
significant delays to every data transformation stage, no matter how well-
automated it is. By using stream processing and allowing people to define
transformations in a way agnostic to how often data arrives the end-to-end
propagation time of data can be reduced from weeks/months to mere seconds. As
new data arrives, it is immediately made available to the consumer in its most
usable form. This satisfies one the main guiding design principles of ODF that
the frequency with which data is presented to consumers should be optimized
for their experience and usability, not dictated by limitations of the data
pipeline.

###  2.7. Metadata Tracking

In ODF, data never appears in the system alone, as we would not be able to
tell whether it can be trusted. Metadata, therefore, is an essential part of a
dataset. It contains every aspect of where the data came from, how it was
transformed, and everything that ever influenced how data looks like
throughout its entire lifetime (see Figure 3). We think of metadata as a
digital passport of data that is instrumental to reproducibility,
verifiability, and data provenance.

So far, when talking about datasets, we have been assuming that the inputs,
the transformations, the schema of the result and more were constant. While
new events can be ingested and would propagate through the system, the
processing graph itself was frozen in time. This was a deliberate
oversimplification. As the nature of businesses change, new requirements
arrive, defects are detected and fixed - it’s not a matter of if but when the
time comes to make changes. Calling data a potentially infinite stream would
not make sense without providing a way to improve and evolve it over time.
Having to create a new dataset every time you need to change the schema or
update the transformation would mean that the entire downstream processing
graph would have to be rebuilt from scratch. In order to support backward-
compatible schema changes, evolution of transformations over time, and to
provide a way to correct past mistakes in data and queries we have developed a
ledger-based mechanism for recording the dataset metadata over time. We will
look at it in more detail when discussing the Metadata Chain (see Section
3.2).

###  2.8. Provenance

An ability to easily understand how a specific data point came to be is
crucial for building trust and showing that data can be relied upon. While
verifiability can tell us which data sources were used to produce the results
and which transformations were performed, this is often too coarse-grained.
Provenance, on the other hand, is the ability to trace a specific piece of
data back to its ultimate source, understand which events have directly
contributed to its value, and what data was considered to determine its
existence in the output.

Provenance (Cheney et al., 2009) is an is extensively studied problem in
modern data science (Simmhan et al., 2005), but in practice it still didn’t
fully manifest in any widely applied system or a state of the art enterprise
data pipeline. Most solutions implement it only on the dataset level, as a
simplified form called lineage. Granular provenance is much challenging to
achieve as it needs to span through virtually every component of the data
pipeline, and potentially across many independent systems. Even if fully
implemented, its value would be limited due to mutable nature of data in many
modern data systems.

We believe that not being able to answer provenance questions fast can
undermine the credibility of even fully trustworthy data, so ODF was designed
with complete provenance in mind and supports it on multiple levels. Firstly,
the metadata tracking creates a link between output and input data blocks for
every iteration of a transformation. This lets ODF improve provenance
granularity by limiting the search space from entire datasets to small blocks
of data within them. Since both data and metadata are immutable, this link is
never lost. Secondly, the declarative nature of the streaming transformations
allows us to easily analyze the structure of queries. For simple map-style
queries, provenance can be derived automatically without tracking any
additional information. Thirdly, for more complex queries we require
provenance to be supported by an underlying data processing engine (Müller et
al., 2018). ODF defines the provenance query API that specific engine
implementations need to implement (see Section 3.3).

Figure 4. Metadata as a chain of immutable blocks

###  2.9. Data Sharing

As a distributed protocol, ODF was designed with data sharing efficiency in
mind and the previously covered features directly contribute towards this
goal.

The source data is irreducible by definition. ODF makes no assumptions that
this data can be retrieved from anywhere else in case its lost, thus every
peer that publishes root datasets is responsible for storing them durably, in
a replicated and highly-available way. The derivative data, as we covered, is
considered transient; therefore all parties that publish derivative datasets
can use the cheapest data hosting available (or no hosting at all) without the
need for durability or heavy replication.

The immutability property of both data and metadata ensures they can be easily
and safely replicated without complex synchronization mechanisms. Metadata
being cryptographically linked to the raw data means that there is no need to
encrypt the data itself or use a secure channel for distributing it unless we
want data to remain private. It is usually sufficient to securely distribute
the metadata and use it to establish the authenticity of the downloaded data.
Metadata is several orders of magnitude smaller than associated data, so it
can be easily hosted and widely shared.

With all these properties combined, ODF has the potential to significantly
reduce data storage and distribution costs globally, compared to the
widespread copy-and-version approach that often results in circulation of
thousands of similar copies of a dataset taken at different points in time.

##  3\. Implementation

A prototype implementation of the Open Data Fabric protocol is currently
available in our Kamu CLI tool (Kamu Data Inc., 2020a). Data transformation
can be performed using two of our stream processing engine implementations
based on Apache Spark (Zaharia et al., 2016) and Apache Flink (Carbone et al.,
2015). This section will introduce some key components of the ODF and
technologies used to implement them.

###  3.1. Data Ingestion

Root datasets are the points of entry for external data into the system. Our
vision is that eventually all root datasets will be owned by organizations
with full authority over certain data (i.e. trusted publishers) and will be
provided directly in ODF-compliant format. As a fallback mechanism, we also
provide a way to link a root dataset to some external source (e.g. using the
URL) and periodically ingest its data into the system. Such configuration
duplicates the data, but it is necessary to guarantee that the properties of
data are preserved and insulate the rest of the system from a wide range of
current bad practices in data publishing.

For data sources that always preserve the entire history ODF has to do little
but copy and de-duplicate the data with records that were ingested previously.
For data sources that publish data in a destructive way (e.g. periodic state
snapshots) ODF takes on the task of ”historization” - transforming state data
into event form using the Change Data Capture (Petrie et al., 2018; Ankorion,
2005) (CDC) techniques.

###  3.2. Metadata Chain

The Metadata Chain’s purpose is to capture all information and events that
influenced the way data looks right now. This includes: where the data comes
from, how it was processed, its schema, and current watermark (see Figure 4).

Its design borrows heavily from the existing ledger-based systems such as
blockchain (Zheng et al., 2017) and version control systems (Otte, 2009). Just
like all data in ODF, the metadata chain is append-only and immutable. It
consists of individual blocks that are cryptographically linked together and
also contain hashes of the associated data slices, so the overall data
structure is similar to a Merkle Tree (Merkle, 1988).

Metadata chain is designed for extensibility and can carry other kinds of
information, such as:

  * •

Extra meaning and structure of knowledge (semantics, ontology)

  * •

Relevant policies, terms, rules, compliance, and regulations (governance)

  * •

License, privacy and security concerns (stewardship)

  * •

Information that aids discovery

  * •

Interoperability data to connect ODF to other ledger-based systems

We see it as a crucial building block that will allow us to collectively push
the quality of data further by standardizing and automating best data science
and engineering practices.

Figure 5. Engine execution environment

###  3.3. Engine

Data processing technologies are evolving rapidly. As a data exchange
protocol, we would like ODF to outlive most of them and be able to adapt to
new technologies as they emerge. We also want ODF to be inclusive of any data
processing languages and dialects used in different branches of data science.
Therefore we designed ODF to be unopinionated as to which language is used to
define transformations or which framework performs them, as long as they
satisfy all the criteria necessary for the protocol to function (e.g.
determinism).

A specific implementation of ODF’s data processing contract is called an
Engine. Engines are responsible for applying queries defined in the datasets
to input data and returning the result. For example, the two of our prototype
engines based on Apache Spark (Zaharia et al., 2016) and Apache Flink (Carbone
et al., 2015) allow us to transform data using a series of Streaming SQL
(Begoli et al., 2018) statements. Since engines are in full control of all
data transformations, they are also responsible for answering the provenance
queries.

ODF takes a few extra steps to guarantee the deterministic and reproducible
properties of transformations done by the engines.

####  3.3.1. Execution Environment

Engines run in a fully isolated environment called the ”sandbox”, implemented
using the OCI Containers (The Linux Foundation, [n.d.]) technology (see Figure
5). Sandbox is designed to prevent engines from accessing any external
resources except for the inputs and outputs of a current transformation (e.g.
on the Internet or user’s file system) as potential sources of undesired non-
determinism.

This may sound very restrictive, and it is. After all many common data
processing tasks like geolocation rely on the use of external APIs which would
be inaccessible under the sandbox model. However, we strongly believe that
this is a necessary step in managing data correctly. External resources like
APIs are run by companies that can disappear over night, they also often
evolve without following strict versioning policies - it is impossible to
achieve reproducible results in such environment. Running any ”black box”
operations like API calls would require us to re-classify derivative datasets
as source data and admit that such data is non-reproducible. Instead we
envision that the software algorithms and ML models used by such
transformations will be incorporated into the ODF as the engine extensions or
pure data, and this transition will be one of the focus points of our future
research.

####  3.3.2. Engine Versioning

To further strengthen the reproducibility guarantees of the system we
associate every transformation with an exact version of an engine that was
used to perform it. This excludes the possibility of any code changes in the
engine producing different results than what was originally observed. For this
purpose we use the full SHA digest of the engine’s OCI image.

As dataset evolves over time it may start depending on too many different
versions of a certain engine, unsustainably increasing the amount of images
user needs to download to fully validate the dataset. We use a special engine
upgrade procedure to remedy this problem.

####  3.3.3. Checkpoints & Watermarks

Some computations over the input data like windowed aggregations or temporal
joins may require engine to maintain some state. Since engine is required to
fully consume input data during transformation (as it will never see it again)
some of this state may need to be preserved in between the invocations of a
query. For this purpose ODF allows engines to maintain checkpoints \- a piece
of opaque and fully engine-specific data used to store intermediate state.
Along with data and metadata, checkpoints are an integral part of a dataset.
If checkpoint is lost the entire computation will have to be restarted from
scratch.

Figure 6. Example use of watermarks in the event stream with an allowed
lateness of t=4.

Every dataset in ODF also has a watermark (Akidau et al., 2015) \- a metadata
tuple $(Ts,Te)$ we described earlier. For example, if a root dataset receives
new data on a monthly basis, its watermark can lag by over a month behind the
wall clock time. It effectively prevents all derivative processing from
proceeding past that time point until the data arrives, ensuring the
correctness of computations. Figures 2 and 6 shows examples of how watermarks
can prevent late processing errors in event streams. Watermarks can be set on
root datasets both via fixed offset or manually, and they then fully
automatically propagate through derivative datasets based on the nature of
transformations. Watermarks are elevated from the checkpoints into the
metadata as they are an important consumer-facing property.

###  3.4. Coordinator

The coordinator is an application responsible for maintaining the invariants
and transactional semantics of the system. It handles all operations related
to the metadata chain and guarantees its integrity and validity. The
coordinator implements data ingestion and data sharing logic, but delegates
all data processing to the engines.

Complexity of the metadata management is fully contained in the coordinator.
From an engine’s perspective the transformations look just like conventional
stream processing, so the underlying data processing frameworks don’t require
any customizations or being made ODF-aware. Turning an existing data
processing library into an engine is a matter of creating a thin wrapper
around it that conforms to the ODF engine interface.

Besides the query execution logic, the coordinator also expects engines to
implement a few extra operations related to the dataset life-cycle, such as
input schema change, query change, and an engine upgrade handlers.

##  4\. Conclusion

The future of data management is a distributed network of streaming
transformations, where data from trusted publishers propagates rapidly through
the computational graph and is always readily available to decision-makers,
automation, and AI/ML. Most of the technologies that can make this vision a
reality are either already here or within our reach, but we think a mindset
shift is also necessary for data-centric disciplines to abandon the local
optima of ignoring temporal dimension of data, of treating data as mere binary
blobs, constantly losing and rewriting our digital history.

Figure 7. ODF as one of the pillars of digital democracy.

Open Data Fabric is our attempt to achieve this state by defining the
properties we want to get from data first and then designing a system around
them. We saw how several key decisions like immutability of data,
deterministic transformations, and using stream processing technologies
coupled with ledger-based metadata tracking create a positive feedback loop
with profound effects on data sharing efficiency and collaboration potential.
We believe that its adoption will be a monumental step towards better data. It
won’t be easy, as it proposes a set of much stricter rules for processing data
than what we are used to - no more manual tweaking, no more ”black box” API
calls - but we think the results are well worth it. Modern data processing
frameworks will also have to step up to the challenge to deliver better stream
and temporal processing capabilities and make determinism an intrinsic
property.

In the long term, we see Open Data Fabric becoming one of the pillars of the
future generation digital democracy, as the primary supply chain for
structured data, which is readily consumable by the peer-to-peer web protocols
and provides reliable factual data to the blockchain smart contracts to build
a better more connected world. We hope others, even if they don’t share our
vision fully, will find some of the ideas presented here useful as we all
continue to push forward the state of the art in this fascinating, remarkably
complex field.

## References

  * (1)
  * Akidau et al. (2015) Tyler Akidau, Robert Bradshaw, Craig Chambers, Slava Chernyak, Rafael J. Fernández-Moctezuma, Reuven Lax, Sam McVeety, Daniel Mills, Frances Perry, Eric Schmidt, and Sam Whittle. 2015\.  The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing.  _Proceedings of the VLDB Endowment_ 8 (2015), 1792–1803.
  * Ankorion (2005) Itamar Ankorion. 2005\.  Change data capture efficient ETL for real-time bi.  _Information Management_ 15, 1 (2005), 36\.
  * Armbrust et al. (2020) Michael Armbrust, Tathagata Das, Liwen Sun, Burak Yavuz, Shixiong Zhu, Mukul Murthy, Joseph Torres, Herman van Hovell, Adrian Ionescu, Alicja Łuszczak, et al. 2020\.  Delta lake: high-performance ACID table storage over cloud object stores.  _Proceedings of the VLDB Endowment_ 13, 12 (2020), 3411–3424.
  * Baker et al. (2020) Dannon Baker, Marius Van Den Beek, Daniel Blankenberg, Dave Bouvier, John Chilton, Nate Coraor, Frederik Coppens, Ignacio Eguinoa, Simon Gladman, Björn Grüning, et al. 2020\.  No more business as usual: Agile and effective responses to emerging pathogen threats require open data and open analytics.  _PLoS pathogens_ 16, 8 (2020), e1008643.
  * Baker (2016) Monya Baker. 2016\.  Reproducibility crisis.  _Nature_ 533, 26 (2016), 353–66.
  * Begoli et al. (2018) Edmon Begoli, Jesús Camacho-Rodríguez, Julian Hyde, Michael J Mior, and Daniel Lemire. 2018\.  Apache calcite: A foundational framework for optimized query processing over heterogeneous data sources. In _Proceedings of the 2018 International Conference on Management of Data_. 221–230.
  * Carbone et al. (2015) Paris Carbone, Asterios Katsifodimos, Stephan Ewen, Volker Markl, Seif Haridi, and Kostas Tzoumas. 2015.  Apache flink: Stream and batch processing in a single engine.  _Bulletin of the IEEE Computer Society Technical Committee on Data Engineering_ 36, 4 (2015), 28–38.
  * Cheney et al. (2009) James Cheney, Laura Chiticariu, and Wang-Chiew Tan. 2009\.  Provenance in Databases: Why, How, and Where.  _Found. Trends Databases_ 1, 4 (apr 2009), 379–474.  https://doi.org/10.1561/1900000006
  * Date et al. (2002) Christopher John Date, Hugh Darwen, and Nikos Lorentzos. 2002\.  _Temporal data & the relational model_.  Elsevier.
  * Fowler (2005) Martin Fowler. 2005\.  Event Sourcing.  martinfowler.com/eaaDev/EventSourcing.html.  Accessed: 2020-09-10.
  * Group (2020) RDA COVID-19 Working Group. 2020\.  RDA COVID-19 Recommendations and Guidelines on Data Sharing.  https://doi.org/10.15497/rda00052
  * Johnston (2014) Tom Johnston. 2014\.  _Bitemporal Data: Theory and Practice_.  Elsevier Science & Technology, San Francisco.
  * Kamu Data Inc. (2020a) Kamu Data Inc. 2020a.  Kamu CLI - Reference implementation of ODF coordinator.  https://github.com/kamu-data/kamu-cli.
  * Kamu Data Inc. (2020b) Kamu Data Inc. 2020b.  Open Data Fabric - Protocol Specification.  http://opendatafabric.org.
  * Liquidata Inc. ([n.d.]) Liquidata Inc. [n.d.].  Dolt - Git for data.  https://github.com/liquidata-inc/dolt.
  * Marx (2013) Vivien Marx. 2013\.  THE BIG CHALLENGES OF BIG DATA.  _Nature_ 498, 7453 (2013), 255.  arXiv:https://www.nature.com/articles/498255a https://www.nature.com/articles/498255a
  * Mehra et al. (2020) Mandeep R. Mehra, Sapan S. Desai, Frank Ruschitzka, and Amit N. Patel. 2020.  RETRACTED: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis.  _The Lancet_ 0, 0 (May 2020).  https://doi.org/10.1016/S0140-6736(20)31180-6 Retraction in: Lancet. 2020 Jun 5;:null. Erratum in: Lancet. 2020 May 30;: PMID: 32450107; PMCID: PMC7255293.
  * Merkle (1988) Ralph C. Merkle. 1988\.  A Digital Signature Based on a Conventional Encryption Function. In _Advances in Cryptology — CRYPTO ’87_ , Carl Pomerance (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 369–378.
  * Miyakawa (2020) Tsuyoshi Miyakawa. 2020\.  No raw data, no science: another possible source of the reproducibility crisis.  _Molecular brain_ 13, 1 (2020), 24–24.
  * Müller et al. (2018) Tobias Müller, Benjamin Dietrich, and Torsten Grust. 2018\.  You Say ’What’, i Hear ’where’ and ’Why’: (Mis-)Interpreting SQL to Derive Fine-Grained Provenance.  _Proc. VLDB Endow._ 11, 11 (July 2018), 1536–1549.  https://doi.org/10.14778/3236187.3236204
  * Murray et al. (2013) Derek G Murray, Frank McSherry, Rebecca Isaacs, Michael Isard, Paul Barham, and Martín Abadi. 2013\.  Naiad: a timely dataflow system. In _Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles_. 439–455.
  * Nicole Shu Ling and Bor Luen (2020) Yeo-Teh Nicole Shu Ling and Tang Bor Luen. 2020.  An alarming retraction rate for scientific publications on Coronavirus Disease 2019 (COVID-19).  _Accountability in Research_ 0, 0 (2020), 1–7.  https://doi.org/10.1080/08989621.2020.1782203 arXiv:https://doi.org/10.1080/08989621.2020.1782203  PMID: 32573274.
  * Noll (2020) Michael Noll. 2020\.  Streams and Tables in Apache Kafka: A Primer.  https://www.confluent.io/blog/kafka-streams-tables-part-1-event-streaming.
  * Otte (2009) Stefan Otte. 2009\.  Version control systems.  _Computer Systems and Telematics_ (2009), 11–13.
  * Petrie et al. (2018) Kevin Petrie, Dan Potter, and Itamar Ankorion. 2018.  _Streaming Change Data Capture_ (1 ed.).  O’Reilly Media, Inc.
  * Simmhan et al. (2005) Yogesh L Simmhan, Beth Plale, and Dennis Gannon. 2005\.  A survey of data provenance in e-science.  _ACM Sigmod Record_ 34, 3 (2005), 31–36.
  * The Linux Foundation ([n.d.]) The Linux Foundation. [n.d.].  Open Container Initiative.  https://opencontainers.org/.
  * The White House (2020) The White House. 2020\.  Call to Action to the Tech Community on New Machine Readable COVID-19 Dataset.  https://www.whitehouse.gov/briefings-statements/call-action-tech-community-new-machine-readable-covid-19-dataset/.
  * Tzoumas (2015) Kostas Tzoumas. 2015\.  Batch is a special case of streaming.  https://www.ververica.com/blog/batch-is-a-special-case-of-streaming.
  * Zaharia et al. (2016) Matei Zaharia, Reynold S Xin, Patrick Wendell, Tathagata Das, Michael Armbrust, Ankur Dave, Xiangrui Meng, Josh Rosen, Shivaram Venkataraman, Michael J Franklin, et al. 2016\.  Apache spark: a unified engine for big data processing.  _Commun. ACM_ 59, 11 (2016), 56–65.
  * Zahuranec (2020) Andrew J. Zahuranec. 2020\.  Call for Action: Toward Building the Data Infrastructure and Ecosystem We Need to Tackle Pandemics and Other Dynamic Societal and Environmental Threats.  http://thegovlab.org/call-for-action-toward-building-the-data-infrastructure-and-ecosystem-we-need-to-tackle-pandemics-and-other-dynamic-societal-and-environmental-threats/.
  * Zheng et al. (2017) Zibin Zheng, Shaoan Xie, Hongning Dai, Xiangping Chen, and Huaimin Wang. 2017.  An overview of blockchain technology: Architecture, consensus, and future trends. In _2017 IEEE international congress on big data (BigData congress)_. IEEE, 557–564.  https://doi.org/10.1109/BigDataCongress.2017.85