the ordering of the $q$-boson algebras in the tensor product produces an
isomorphic algebra, as a consequence of the universal Yang-Baxter equation for
the universal R-matrix of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$. Hence, the
algebra of local operators on the fused surface defect
$\mathcal{S}_{L,M,N}^{\mathbf{(1,0)}}$ can be denoted as
$\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$ without ambiguity.

The algebra $\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$ can be thought of as the
multiplicative uplift of the $Y$-algebra studied in Gaiotto:2017euk . As we
will explain, the multiplicative uplift is achieved by replacing
$\mathbb{C}\times\mathbb{C}^{\times}$ by
$\mathbb{C}^{\times}\times\mathbb{C}^{\times}$ for the holomorphic part of the
worldvolume of the twisted M-theory. In this sense, we call
$\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$ the $q$-deformed $Y$-algebra as well,
which may be denoted by $q{Y}_{L,M,N}$.

##  3 M2-M5 intersections, R-matrices, and Miura operators

So far, we have introduced the algebra of local operators on M2- and M5-branes
and surjective homomorphisms from the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$. In this
section, we study the intersections of M2-branes and M5-branes. Passing to the
5d Chern-Simons theory on
$\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$,
the M2-M5 intersections descend to the intersections between topological line
defects and holomorphic surface defects. Note that the M2-brane wraps one, say
$\mathbb{R}^{2}_{{\varepsilon}_{c}}$, of the three topological planes, while
an M5-brane wraps two, say
$\mathbb{R}^{2}_{{\varepsilon}_{c^{\prime}+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c^{\prime}-1}}$,
of them. Depending on the relative orientation, we distinguish two
possibilities:

  * •

If $c=c^{\prime}$, the two defects are completely transverse so that, when
visualized in the 3-dimensional worldvolume that they span, the line defect
can only pass through the surface defect. By slightly abusing the terminology,
we call this configuration the transverse M2-M5 intersection.

  * •

If $c\neq c^{\prime}$, the line defect can end on the surface defect at the
non-transverse M2-M5 intersection. Thus the line defect may terminate at the
intersection, or may continue across the intersection. In the former case, the
M2-brane lies on a semi-infinite line in $\mathbb{R}_{t}$, stretched from the
intersection point to either positive or negative infinity.

We will investigate these two cases separately.

In this section, we will restrict to the case where the holomorphic part of
the support of M5-branes is $\mathbb{C}^{\times}_{X}=C^{\mathbf{(1,0)}}$. We
will come back to more generic M5-brane configurations in section 5.

###  3.1 R-matrix for transverse M2-M5 intersection

Let us begin with the cases where there are a single M2-brane and a single
M5-brane. Due to the triality, we may fix the M2-brane to be supported on
$\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}$. Then, the M5-brane
wrapping
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{C}^{\times}_{X}$
is totally transverse to the M2-brane. Even though they do not intersect at
all in the 11-dimensional worldvolume, we shall call such a configuration a
transverse M2-M5 intersection.

The transverse M2-M5 intersection supports a space of local operators. We can
write a generic element of this space as

$\displaystyle
R^{(3)}=R_{\text{M2}_{0,0,1},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}\in\text{M2}_{0,0,1}\widehat{\otimes}\,\text{M5}_{0,0,1}^{\mathbf{(1,0)}}.$
(67)

Under the condition that the support of the M2-brane is
$\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}$ and the holomorphic
part of the support of the M5-brane is
$\mathbb{C}^{\times}_{X}=C^{\mathbf{(1,0)}}$, we use the abbreviation
$R^{(3)}$ where the superscript indicates the topological part of the M5-brane
support
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}$
(by $\\{1,2\\}=\\{1,2,3\\}\setminus\\{3\\}$). Passing to the 5d
$\mathfrak{gl}(1)$ Chern-Simons theory, the transverse M2-M5 intersection
should descend to a gauge-invariant intersection between a line defect and a
surface defect. The local operators at the intersection are thereby
constrained by the gauge invariance condition.

As we discussed in section 1.2, the fully quantum-corrected gauge-invariance
condition is expected to be given by

$\displaystyle
R^{(3)}\Delta_{\text{M2}_{0,0,1},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}(g)=\Delta_{\text{M2}_{0,0,1},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}^{\text{op}}(g)R^{(3)},\quad\text{for
any }g\in U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1)),$ (68)

where $\Delta_{\text{M2}_{0,0,1},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}$ is the
mixed coproduct, obtained by composing the coproduct with the respective
representations,

$\displaystyle\Delta_{\text{M2}_{0,0,1},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}=(\rho_{\text{M2}_{0,0,1}}\otimes\rho_{\text{M5}_{0,0,1}^{\mathbf{(1,0)}}})\Delta\quad:\quad
U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\to\text{M2}_{0,0,1}\,\widehat{\otimes}\,\text{M5}_{0,0,1}^{\mathbf{(1,0)}},$
(69)

and $\Delta^{\text{op}}=\sigma\circ\Delta$ is the opposite coproduct.

$R^{(3)}$$\times$$0$$\mathcal{S}_{0,0,1}^{\mathbf{(1,0)}}$$\mathcal{L}_{0,0,1}$$\mathbb{C}^{\times}_{X}$$\mathbb{R}_{t}$
Figure 7: Transverse M2-M5 intersection

The constraint (68) can be solved exactly. We show in appendix C that it fully
determines $R^{(3)}$ up to an overall scaling. The solution is obtained as the
product:

$\displaystyle R^{(3)}=\bar{R}^{(3)}K^{(3)},$ (70)

with the Cartan part given as

$\displaystyle\begin{split}K^{(3)}=\left(q_{3}^{\frac{d}{2}}\otimes
1\right)\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{X_{1}^{-r}}{1-q_{3}^{r}}\otimes\frac{a_{r}^{(3)}}{q_{3}^{r/2}-q_{3}^{-r/2}}\right]=\left(q_{3}^{\frac{d}{2}}\otimes
1\right)V_{3}(X_{1}),\end{split}$ (71)

where we defined the vertex operator
$V_{3}(X_{1}):=\prod_{j=0}^{\infty}\varphi_{3}^{+}(q_{3}^{-j}X_{1})=\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{X_{1}^{-r}}{1-q_{3}^{r}}\otimes\frac{a_{r}^{(3)}}{q_{3}^{r/2}-q_{3}^{-r/2}}\right]$.
The rest is given by

$\displaystyle\bar{R}^{(3)}=1-e^{-a_{0}^{(3)}\frac{\log q_{1}\log q_{2}}{\log
q_{3}}}q_{3}^{-1}\xi_{3}(X_{1})q_{3}^{-D_{X_{1}}}$ (72)

Note that $X_{1}\in\text{M2}_{0,0,1}$ commutes with itself so that the vertex
operators with the argument $X_{1}$ are well-defined.

###  3.2 Non-transverse M2-M5 intersections

We investigate the intersection between a single M2-brane and a single
M5-brane that share one topological $\mathbb{R}^{2}$-plane in their supports.
We fix the support of the M2-brane as
$\mathbb{R}^{2}_{{\varepsilon}_{3}}\times\mathbb{R}_{t}$. All other cases
follow simply by the triality. The holomorphic part of the support of the
M5-brane is chosen as $\mathbb{C}^{\times}_{X}$.

Since we are assuming the support of the M5-brane shares
$\mathbb{R}^{2}_{{\varepsilon}_{3}}$, the only two options are
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}\times\mathbb{R}_{t}\times\mathbb{C}^{\times}_{X}$
with $c\in\\{1,2\\}$. We also use the notation
$\\{\bar{c}\\}=\\{1,2\\}\setminus\\{c\\}$.

####  3.2.1 Intertwiners from intersections with ends

Unlike the transverse case, the M2-brane can end on the M5-brane from either
direction of the line $\mathbb{R}_{t}$. We show such non-transverse M2-M5
intersections with an end are associated to intertwiners of the quantum
toroidal algebra of $\mathfrak{gl}(1)$.

First, we consider the case of the M2-brane ending on the M5-brane from the
negative direction of $\mathbb{R}_{t}$. In other words, the M2-brane is
stretched between the negative infinity and the intersection point in
$\mathbb{R}_{t}$. It is helpful to view the other semi-infinite line as being
occupied by a trivial line defect, so that the local operator supported at the
intersection is an element of222222Compare with the transverse R-matrix
$R^{(3)}$ (67), where both semi-infinite lines above and below from the
intersection point support the same line defect $\mathcal{L}_{0,0,1}$. There,
we had $\text{M2}_{0,0,1}=\text{End}(\mathcal{V}_{3}(u))$ in turn.

$\displaystyle\begin{split}&\Phi_{3,c}\in\text{Hom}(\mathcal{V}_{3}(u),\mathbb{C})\otimes\text{M5}_{c}^{\mathbf{(1,0)}},\qquad
c\in\\{1,2\\}.\end{split}$ (73)

The intersection must be gauge-invariant. As argued in section 1.2 for the
line defect passing through the surface defect, the classical gauge-invariance
constraint for the local operator $\Phi_{3,c}$ reads

$\displaystyle\Phi_{3,c}\left(\rho_{\text{M2}_{0,0,1}}(g)\otimes\text{id}+\text{id}\otimes\rho_{\text{M5}_{c}^{\mathbf{(1,0)}}}(g)\right)=\rho_{\text{M5}_{c}^{\mathbf{(1,0)}}}(g)\Phi_{3,c}$
(74)

for any $g\in
U\left(\mathscr{O}_{q_{2}}(\mathbb{C}^{\times}\times\mathbb{C}^{\times})\right)$.

Turning on $q_{1}\neq 1$, the gauge-invariance condition for intersection gets
quantum corrections, which are non-linear in the generators. By a similar
argument for the transverse M2-M5 intersection, we expect that the full
quantum gauge-invariance condition reads

$\displaystyle\Phi_{3,c}\Delta_{\text{M2}_{0,0,1},\text{M5}_{c}^{\mathbf{(1,0)}}}(g)=\rho_{\text{M5}^{\mathbf{(1,0)}}_{c}}(g)\Phi_{3,c},\qquad\text{for
any }g\in U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1)),$ (75)

for each $c\in\\{1,2\\}$.232323Using the counit
$\epsilon=\rho_{\text{M2}_{0,0,0}}$ associated to the trivial line defect (see
footnote 17), the gauge transformation on right hand side of (75) can be
rephrased as
$(\rho_{\text{M2}_{0,0,0}}\otimes\rho_{\text{M5}_{0,0,1}^{\mathbf{(1,0)}}})\Delta^{\text{op}}(g)=(1\otimes\rho_{\text{M5}_{0,0,1}^{\mathbf{(1,0)}}})(\epsilon\otimes\text{id})\Delta^{\text{op}}(g)=\rho_{\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}(g)$.
The gauge-invariance condition for all the other cases
$\Phi_{c,c^{\prime}}\in\text{Hom}(\mathcal{V}_{c},\mathbb{C})\otimes\text{M5}_{c^{\prime}}^{\mathbf{(1,0)}}$,
$c\neq c^{\prime}$ simply follows by the triality. In this sense, the non-
transverse M2-M5 intersection provides the intertwiner between the
representations $\text{M2}_{0,0,1}$ and $\text{M5}_{c}^{\mathbf{(1,0)}}$ of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$. The constraint
(75) determines $\Phi_{3,c}$ up to an overall scaling, yielding Feigin2009ACA

$\displaystyle\begin{split}\Phi_{3,c}&=e^{\frac{\log q_{\bar{c}}}{\log
q_{c}}\left({\log X_{1}}\otimes a_{0}^{(c)}\right)}\left(1\otimes
q_{\bar{c}}^{d^{\perp}}\right)\exp\left[-\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{3}^{\frac{r}{2}}}{\left(q_{c}^{\frac{r}{2}}-q_{c}^{-\frac{r}{2}}\right)^{2}\left(q_{3}^{\frac{r}{2}}-q_{3}^{-\frac{r}{2}}\right)}X_{1}^{r}\otimes
a_{-r}^{(c)}\right]\\\
&\qquad\times\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{3}^{-\frac{r}{2}}q_{c}^{-\frac{r}{2}}}{\left(q_{c}^{\frac{r}{2}}-q_{c}^{-\frac{r}{2}}\right)^{2}\left(q_{3}^{\frac{r}{2}}-q_{3}^{-\frac{r}{2}}\right)}X_{1}^{-r}\otimes
a_{r}^{(c)}\right].\end{split}$ (76)

Note that $\log X_{1}$ makes sense as an element of
$\text{Hom}(\mathcal{V}_{3}(u),\mathbb{C})$ since $X_{1}$ acts on
$\mathcal{V}_{3}(u)$ as an element of $\mathbb{C}^{\times}$. Noting that
$\mathcal{V}_{3}(u)^{*}=\text{Hom}(\mathcal{V}_{3}(u),\mathbb{C})$, we may
also expand $\Phi_{3,c}$ in the dual basis
$\\{[u]_{i}^{(3)*}\;|\;i\in\mathbb{Z}\\}$ satisfying
$[u]_{i}^{(3)*}([u]_{j}^{(3)})=\delta_{i,j}$ as
$\Phi_{3,c}=\sum_{i\in\mathbb{Z}}(\Phi_{3,c})_{i}[u]^{(3)*}_{i}$, where the
coefficients $(\Phi_{3,c})_{i}$ is obtained simply by replacing $X_{1}$ by
$uq_{3}^{i}$ due to its action on $[u]_{i}^{(3)}\in\mathcal{V}_{3}(u)$.

It should be stressed that $\Phi_{3,c}$ provides a reconstruction of a vertex
operator in the $q$-boson quantum vertex algebra in the realm of the
representation theory of the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$. Moreover, in
the degeneration limit of the $q$-boson to the free boson
$\widehat{\mathfrak{gl}}(1)$ vertex algebra, it reduces to the usual
degenerate vertex operator. The degenerate limit is $R\to 0$ where
$q_{a}^{R{\varepsilon}_{a}}$, in which the $q$-boson commutation relation (56)
reduces to that of $\widehat{\mathfrak{gl}}(1)$, given by

$\displaystyle[\tilde{a}_{m}^{(c)},\tilde{a}_{n}^{(c)}]=-\frac{1}{{\varepsilon}_{c+1}{\varepsilon}_{c-1}}m\delta_{m+n,0},\qquad
m,n\in\mathbb{Z},$ (77)

with the redefinition $\tilde{a}_{n}^{(c)}=\frac{a_{n}^{(c)}}{\log q_{c}}$.
Then, we indeed have $\Phi_{3,c}(X_{1})\to
e^{{\varepsilon}_{\bar{c}}\phi(X_{1})}$, where
$\phi(X_{1})=d^{\perp}+\tilde{a}_{0}^{(c)}\log
X_{1}-\sum_{n\in\mathbb{Z}\setminus\\{0\\}}\frac{1}{n}\tilde{a}_{n}^{(c)}X_{1}^{-n}$
is the free boson field.

$\Phi^{*}_{3,1}$$\Phi_{3,1}$$\times$$0$$\mathcal{S}_{1,0,0}^{\mathbf{(1,0)}}$$\mathcal{L}_{0,0,1}$$\mathcal{L}_{0,0,1}$$\mathbb{C}^{\times}_{X}$$\mathbb{R}_{t}$
Figure 8: Non-transverse M2-M5 intersections

Next, we turn to the M2-brane ending on the M5-brane from the positive
$\mathbb{R}_{t}$-direction. The M2-brane lies on the semi-infinite line
between the intersection point and the positive infinity in $\mathbb{R}_{t}$,
and we treat the other semi-infinite line as supporting a trivial line defect.
The local operator at the intersection thus lies in

$\displaystyle\Phi^{*}_{3,c}\in\text{Hom}(\mathbb{C},\mathcal{V}_{3}(u))\otimes\text{M5}_{c}^{\mathbf{(1,0)}},\qquad
c\in\\{1,2\\}.$ (78)

By the argument similar to the previous case, the gauge-invariance condition
at the quantum level $q_{1}\neq 1$ is expected to be written as

$\displaystyle\Phi^{*}_{3,c}\rho_{\text{M5}_{c}^{\mathbf{(1,0)}}}(g)=\Delta^{\text{op}}_{\text{M2}_{0,0,1},\text{M5}_{c}^{\mathbf{(1,0)}}}(g)\Phi^{*}_{3,c},\qquad
c\in\\{1,2\\},$ (79)

The constraint can be solved to determine $\Phi^{*}_{3,c}$ up to an overall
scaling as

$\displaystyle\begin{split}\Phi^{*}_{3,c}&=\left(1\otimes
q_{\bar{c}}^{-d^{\perp}}\right)e^{-\frac{\log q_{\bar{c}}}{\log
q_{c}}\left({\log X_{1}}\otimes
a_{0}^{(c)}\right)}\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{3}^{\frac{r}{2}}q_{c}^{\frac{r}{2}}}{\left(q_{c}^{\frac{r}{2}}-q_{c}^{-\frac{r}{2}}\right)^{2}\left(q_{3}^{\frac{r}{2}}-q_{3}^{-\frac{r}{2}}\right)}X_{1}^{r}\otimes
a_{-r}^{(c)}\right]\\\
&\qquad\times\exp\left[-\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{3}^{-\frac{r}{2}}}{\left(q_{c}^{\frac{r}{2}}-q_{c}^{-\frac{r}{2}}\right)^{2}\left(q_{3}^{\frac{r}{2}}-q_{3}^{-\frac{r}{2}}\right)}X_{1}^{-r}\otimes
a_{r}^{(c)}\right].\end{split}$ (80)

Noting that
$\mathcal{V}_{3}(u)\simeq\text{Hom}(\mathbb{C},\mathcal{V}_{3}(u))$, we may
expand $\Phi^{*}_{3,c}$ in the basis elements as
$\Phi^{*}_{3,c}=\sum_{i\in\mathbb{Z}}(\Phi^{*}_{3,c})_{i}[u]_{i}^{(3)}$. The
coefficients $(\Phi^{*}_{3,c})_{i}$ are obtained simply by replacing $X_{1}$
by $uq_{3}^{i}$ due to the action of $[u]^{(3)*}_{i}$ on $X_{1}$.

Again, we observe that $\Phi^{*}_{3,c}$ gives a reconstruction of a vertex
operator in the $q$-boson algebra as an intertwiner of representations of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$. In the
degeneration limit of the $q$-boson to the free boson
$\widehat{\mathfrak{gl}}(1)$, it is straightforward to see it reduces to the
degenerate vertex operator, $\Phi^{*}_{3,c}(X_{1})\to
e^{-{\varepsilon}_{\bar{c}}\phi(X_{1})}$.

####  3.2.2 R-matrices for non-transverse intersections and fusion of
intertwiners

Having established the non-transverse intersection where M2-brane ends on the
M5-brane, now we turn to the non-transverse intersection where M2-brane passes
through the M5-brane. The local operator at the intersection is an element of

$\displaystyle
R^{(c)}\in\text{M2}_{0,0,1}\widehat{\otimes}\,\text{M5}_{c}^{\mathbf{(1,0)}},\qquad
c\in\\{1,2\\}.$ (81)

The gauge-invariance condition reads

$\displaystyle\begin{split}&R^{(c)}\Delta_{\text{M2}_{0,0,1},\text{M5}_{c}^{\mathbf{(1,0)}}}(g)=\Delta^{\text{op}}_{\text{M2}_{0,0,1},\text{M5}_{c}^{\mathbf{(1,0)}}}(g)R^{(c)},\qquad
c\in\\{1,2\\}.\end{split}$ (82)

The constraint can be solved exactly. As we demonstrate in appendix C, the
constraint determines $R^{(c)}$ up to an overall scaling, yielding

$\displaystyle R^{(c)}=\bar{R}^{(c)}K^{(c)},\qquad c\in\\{1,2\\},$ (83)

where the Cartan part is given by

$\displaystyle\begin{split}\hfil\displaystyle\begin{split}K^{(c)}&=\left(q_{c}^{\frac{d}{2}}\otimes
1\right)\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{X_{1}^{-r}}{1-q_{3}^{r}}\otimes\frac{a_{r}^{(c)}}{q_{c}^{r/2}-q_{c}^{-r/2}}\right]\\\
&=\left(q_{c}^{\frac{d}{2}}\otimes
1\right)V_{c}(X_{1})=V_{c}(q_{c}^{\frac{1}{2}}X_{1})\left(q_{c}^{\frac{d}{2}}\otimes
1\right),\end{split}\end{split}$ (84)

where we defined the vertex operator
$V_{c}(X_{1}):=\prod_{j=0}^{\infty}\varphi_{c}^{+}(q_{3}^{-j}X_{1})=\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{X_{1}^{-r}}{1-q_{3}^{r}}\otimes\frac{a_{r}^{(c)}}{q_{c}^{r/2}-q_{c}^{-r/2}}\right]$.
The rest is expanded with respect to the $d^{\perp}$-grading as

$\displaystyle\begin{split}\bar{R}^{(c)}&=\sum_{m=0}^{\infty}\kappa_{1}^{m}\sum_{k_{1},k_{2},\cdots
k_{m}\in\mathbb{Z}}\left(\rho_{\text{M2}_{0,0,1}}\otimes\rho_{\text{M5}_{c}^{\mathbf{(1,0)}}}\right)\left(E_{k_{1}}E_{k_{2}}\cdots
E_{k_{m}}\otimes F_{-k_{1}}F_{-k_{2}}\cdots F_{-k_{m}}\right)\\\
&=\sum_{m=0}^{\infty}q_{\bar{c}}^{m}e^{-ma_{0}^{(c)}\frac{\log q_{c+1}\log
q_{c-1}}{\log
q_{c}}}\left(\prod_{j=0}^{m-1}\frac{1-q_{c}q_{3}^{-j}}{1-q_{3}^{-j-1}}\right):\prod_{j=0}^{m-1}\xi_{c}(q_{3}^{-j}X_{1}):q_{3}^{-mD_{X_{1}}}.\end{split}$
(85)

Here, the second equality follows from the normal-ordering relation,

$\displaystyle\xi_{c}(X)\xi_{c}(X^{\prime})=\frac{\left(1-\frac{X^{\prime}}{X}\right)\left(1-\frac{q_{c}X^{\prime}}{X}\right)}{\left(1-\frac{q_{c-1}^{-1}X^{\prime}}{X}\right)\left(1-\frac{q_{c+1}^{-1}X^{\prime}}{X}\right)}:\xi_{c}(X)\xi_{c}(X^{\prime}):,\qquad
c\in\\{1,2,3\\}.$ (86)

In fact, the above expressions are valid even at $c=3$ for the transverse
M2-M5 intersection. It is obvious that the expression for the Cartan part
$K^{(c)}$ (84) applies to $c=3$ to reproduce $K^{(3)}$ (71). For (279), if we
set $c=3$ the term in the parenthesis in the second line vanishes when $m\geq
2$. The infinite summation thus truncates to the first two terms, recovering
(72) that we already obtained for the transverse R-matrix.

Having obtained the exact expressions for the R-matrices $R^{(c)}$, we can
also reorganize them into a suggestive form. Let us consider the non-
transverse cases $c\in\\{1,2\\}$. It is crucial to consider the following
normal-ordered product of the intertwiners,

$\displaystyle:\Phi^{*}_{3,c}(X_{1})\Phi_{3,c}(q_{c}^{\frac{1}{2}}q_{3}^{-m}X_{1}):=q_{\bar{c}}^{\frac{1}{2}(1\otimes
a_{0}^{(c)})}e^{-ma_{0}^{(c)}\frac{\log q_{\bar{c}}\log q_{3}}{\log
q_{c}}}:\prod_{j=0}^{m-1}\xi_{c}(q_{3}^{-j}X_{1}):V_{c}(q_{c}^{\frac{1}{2}}q_{3}^{-m}X_{1}).$
(87)

Using this identity, the R-matrix (83) turns into

$\displaystyle\begin{split}{R}^{(c)}&=\sum_{m=0}^{\infty}(-1)^{m}q_{c}^{-m}q_{3}^{\frac{m(m-1)}{2}}q_{\bar{c}}^{-\frac{1}{2}(1\otimes
a_{0}^{(c)})}\frac{(q_{3};q_{3})_{\nu_{c}}}{(q_{3};q_{3})_{m}(q_{3};q_{3})_{\nu_{c}-m}}\\\
&\qquad\times:\Phi^{*}_{3,c}(X_{1})\Phi_{3,c}(q_{c}^{\frac{1}{2}}q_{3}^{-m}X_{1}):(Z_{1}^{m}\otimes
1)\left(q_{c}^{\frac{d}{2}}\otimes 1\right),\end{split}$ (88)

where $\nu_{c}=\frac{\log q_{c}}{\log q_{3}}$ and
$(z;q)_{\nu}=\frac{(z;q)_{\infty}}{(q^{\nu}z;q)_{\infty}}$ is the
$q$-Pochhammer symbol. We can further convert this infinite summation into a
contour integral as

$\displaystyle\begin{split}R^{(c)}&=-(q_{3};q_{3})_{\nu_{c}}q_{\bar{c}}^{-\frac{1}{2}(1\otimes
a_{0}^{(c)})}\oint\frac{dX}{2\pi
iX}\frac{\left(q_{\bar{c}}^{-1}\frac{X}{X_{1}};q_{3}\right)_{\infty}}{\left(\frac{X}{X_{1}};q_{3}\right)_{\infty}}e^{-\frac{\log
q_{\bar{c}}}{\log q_{3}}\log\frac{X}{X_{1}}}\\\
&\qquad\qquad\qquad\qquad\qquad\qquad\times:\Phi^{*}_{3,c}(X_{1})\Phi_{3,c}(q_{c}^{\frac{1}{2}}X):Z_{1}^{-\frac{\log\frac{X}{X_{1}}}{\log
q_{3}}}\left(q_{c}^{\frac{d}{2}}\otimes 1\right),\end{split}$ (89)

where the contour is chosen in such a way that the integral picks up the
residues from all the simple poles at $X=X_{1}q_{3}^{-m}$,
$m\in\mathbb{Z}_{\geq 0}$. Finally, using the normal-ordering relation between
the two intertwiners,

$\displaystyle\Phi^{*}_{3,c}(X)\Phi_{3,c}(q_{c}^{\frac{1}{2}}X^{\prime})=e^{-\frac{\log
q_{\bar{c}}}{\log q_{3}}\left(\log\frac{X^{\prime}}{X}+\frac{1}{2}\log
q_{c}\right)}\frac{\left(q_{\bar{c}}^{-1}\frac{X^{\prime}}{X};q_{3}\right)_{\infty}}{\left(\frac{X^{\prime}}{X};q_{3}\right)_{\infty}}:\Phi^{*}_{3,c}(X)\Phi_{3,c}(q_{c}^{\frac{1}{2}}X^{\prime}):,$
(90)

we can undo the normal-ordering in the integrand, yielding

$\displaystyle\begin{split}R^{(c)}&=-(q_{3};q_{3})_{\nu_{c}}q_{\bar{c}}^{\frac{1}{2}\left(\frac{\log
q_{c}}{\log q_{3}}-1\otimes a_{0}^{(c)}\right)}\left(\oint\frac{dX}{2\pi
iX}\,\Phi^{*}_{3,c}(X_{1})\Phi_{3,c}(q_{c}^{\frac{1}{2}}X)\,Z_{1}^{-\frac{\log\frac{X}{X_{1}}}{\log
q_{3}}}\right)\left(q_{c}^{\frac{d}{2}}\otimes 1\right).\end{split}$ (91)

Here, we view the integral in the parenthesis as an element in
$\text{M2}_{0,0,1}\widehat{\otimes}\,\text{M5}_{c}^{\mathbf{(1,0)}}$ by first
performing the integral in $X$ and then invoking
$X_{1},Z_{1}\in\text{M2}_{0,0,1}$.

It is remarkable that the non-transverse R-matrices can be reconstructed by
using the intertwiners as building blocks. We can visualize this as a fusion
of intertwiners initiated by two non-transverse M2-M5 intersections
approaching each other (see Figure 9). The two M2-branes, ending on the
M5-brane from below and from above respectively, can join at a single point on
$\mathbb{C}^{\times}_{X}$ across the M5-brane. The outcome is a single
M2-brane passing through the M5-brane, which gives rise to the non-transverse
R-matrix $R^{(c)}$ discussed above.

$\Phi^{*}_{3,1}$$\Phi_{3,1}$$\times$$0$$\mathcal{S}_{1,0,0}^{\mathbf{(1,0)}}$$\mathcal{L}_{0,0,1}$$\mathcal{L}_{0,0,1}$$\mathbb{C}^{\times}_{X}$$\mathbb{R}_{t}$

$R^{(1)}$$\times$$0$$\mathcal{S}_{1,0,0}^{\mathbf{(1,0)}}$$\mathcal{L}_{0,0,1}$$\mathcal{L}_{0,0,1}$

Figure 9: Non-transverse R-matrix from fusion of intertwiners

The transverse R-matrix $R^{(3)}$ can also be expressed as a contour integral,
even though extra care is needed: recall that in that case, there is no gauge-
invariant intersection where the M2-brane ends on the M5-brane, since the two
do not actually meet at all in the 11-dimensional spacetime. However, we will
still define $\Phi_{3,3}$ and $\Phi^{*}_{3,3}$ as elements in
$\text{M2}_{0,0,1}\otimes\text{M5}_{0,0,1}^{\mathbf{(1,0)}}$, mimicking
$\Phi_{3,c}$ and $\Phi^{*}_{3,c}$ for $c\in\\{1,2\\}$, as follows:

$\displaystyle\begin{split}&\Phi_{3,3}=e^{\frac{\log q_{1}\log q_{2}}{(\log
q_{3})^{2}}\left({\log X_{1}}\otimes a_{0}^{(3)}\right)}e^{\frac{\log
q_{1}\log q_{2}}{\log q_{3}}(1\otimes
d^{\perp})}\exp\left[-\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{3}^{\frac{r}{2}}}{\left(q_{3}^{\frac{r}{2}}-q_{3}^{-\frac{r}{2}}\right)^{3}}X_{1}^{r}\otimes
a_{-r}^{(3)}\right]\\\
&\qquad\qquad\qquad\qquad\qquad\times\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{3}^{-{r}}}{\left(q_{3}^{\frac{r}{2}}-q_{3}^{-\frac{r}{2}}\right)^{3}}X_{1}^{-r}\otimes
a_{r}^{(3)}\right],\\\ &\Phi^{*}_{3,3}=e^{-\frac{\log q_{1}\log q_{2}}{\log
q_{3}}(1\otimes d^{\perp})}e^{-\frac{\log q_{1}\log q_{2}}{(\log
q_{3})^{2}}\left({\log X_{1}}\otimes
a_{0}^{(3)}\right)}\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{3}^{r}}{\left(q_{3}^{\frac{r}{2}}-q_{3}^{-\frac{r}{2}}\right)^{3}}X_{1}^{r}\otimes
a_{-r}^{(3)}\right]\\\
&\qquad\qquad\qquad\qquad\qquad\times\exp\left[-\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{3}^{-\frac{r}{2}}}{\left(q_{3}^{\frac{r}{2}}-q_{3}^{-\frac{r}{2}}\right)^{3}}X_{1}^{-r}\otimes
a_{r}^{(3)}\right].\end{split}$ (92)

Let us stress again that individual $\Phi_{3,3}$ and $\Phi^{*}_{3,3}$ are not
gauge-invariant, and they do not provide intertwiners for the associated
representations of the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$. Nevertheless,
they can be used together to express the gauge-invariant objects such as the
transverse R-matrix $R^{(3)}$. Indeed, let us note that their normal-ordered
product leads to

$\displaystyle\begin{split}&:\Phi^{*}_{3,3}(X_{1})\Phi_{3,3}(q_{3}^{\frac{1}{2}}X_{1}):=e^{\frac{1}{2}\frac{\log
q_{1}\log q_{2}}{\log q_{3}}(1\otimes
a_{0}^{(3)})}V_{3}(q_{3}^{\frac{1}{2}}X_{1}),\\\
&:\Phi^{*}_{3,3}(X_{1})\Phi_{3,3}(q_{3}^{-\frac{1}{2}}X_{1}):=e^{-\frac{1}{2}\frac{\log
q_{1}\log q_{2}}{\log q_{3}}(1\otimes
a_{0}^{(3)})}\xi_{3}(X_{1})V_{3}(q_{3}^{-\frac{1}{2}}X_{1}).\end{split}$ (93)

Thus, the transverse R-matrix $R^{(3)}$ is reorganized as

$\displaystyle\begin{split}R^{(3)}&=e^{-\frac{1}{2}\frac{\log q_{1}\log
q_{2}}{\log q_{3}}(1\otimes
a_{0}^{(3)})}:\Phi^{*}_{3,3}(X_{1})\left(1-q_{3}^{-1}q_{3}^{-D_{X_{1}}}\right)\Phi_{3,3}(q_{3}^{\frac{1}{2}}X_{1}):\left(q_{3}^{\frac{d}{2}}\otimes
1\right)\\\ &=(1-q_{3}^{-1})e^{-\frac{1}{2}\frac{\log q_{1}\log q_{2}}{\log
q_{3}}(1\otimes a_{0}^{(3)})}\\\ &\quad\times\left(\oint\frac{dX}{2\pi
iX}\frac{1}{\left(1-\frac{X_{1}}{X}\right)\left(1-\frac{q_{3}^{-1}X_{1}}{X}\right)}:\Phi^{*}_{3,3}(X_{1})\Phi_{3,3}(q_{3}^{\frac{1}{2}}X):Z_{1}^{-\frac{\log\frac{X}{X_{1}}}{\log
q_{3}}}\right)\left(q_{3}^{\frac{d}{2}}\otimes 1\right),\end{split}$ (94)

where the contour is chosen to enclose two simple poles at $X=X_{1}$ and
$X=X_{1}q_{3}^{-1}$. Thus, we observe that the transverse R-matrix $R^{(3)}$
can be re-expressed in terms of the vertex operators $\Phi_{3,3}$ and
$\Phi^{*}_{3,3}$.

Nevertheless, we cannot undo the normal-ordering of the two vertex operators
to further simplify the expression in this case, unlike the non-transverse
analogue (91). In this sense, the two vertex operators here are not truly
isolated within itself. This is consistent with the fact that individual
$\Phi_{3,3}$ or $\Phi^{*}_{3,3}$ is not gauge-invariant; that is, the gauge-
invariant transverse M2-M5 intersection $R^{(3)}$ cannot be a composite of two
independent gauge-invariant M2-M5 intersections.

##### Remark

We may define the inverse of the intertwiners simply by flipping the signs of
the exponents. For instance,

$\displaystyle\begin{split}\left(\Phi^{*}_{3,c}\right)^{-1}&:=e^{\frac{\log
q_{\bar{c}}}{\log q_{c}}\left({\log X_{1}}\otimes
a_{0}^{(c)}\right)}\left(1\otimes
q_{\bar{c}}^{d^{\perp}}\right)\exp\left[-\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{3}^{\frac{r}{2}}q_{c}^{\frac{r}{2}}}{\left(q_{c}^{\frac{r}{2}}-q_{c}^{-\frac{r}{2}}\right)^{2}\left(q_{3}^{\frac{r}{2}}-q_{3}^{-\frac{r}{2}}\right)}X_{1}^{r}\otimes
a_{-r}^{(c)}\right]\\\
&\qquad\times\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{q_{3}^{-\frac{r}{2}}}{\left(q_{c}^{\frac{r}{2}}-q_{c}^{-\frac{r}{2}}\right)^{2}\left(q_{3}^{\frac{r}{2}}-q_{3}^{-\frac{r}{2}}\right)}X_{1}^{-r}\otimes
a_{r}^{(c)}\right].\end{split}$ (95)

Note that there is a non-trivial relation

$\displaystyle\left(\Phi^{*}_{3,c}(X_{1})\right)^{-1}=q_{\bar{c}}^{-\frac{1}{2}(1\otimes
a_{0}^{(c)})}\Phi_{3,c}(q_{c}^{\frac{1}{2}}X_{1})V_{c}(q_{c}^{\frac{1}{2}}X_{1})^{-1}.$
(96)

Using this relation, the $\bar{R}^{(c)}$ part of the R-matrix can be
reorganized as

$\displaystyle\begin{split}\bar{R}^{(c)}&=\sum_{m=0}^{\infty}(-1)^{m}q_{c}^{-m}q_{3}^{\frac{m(m-1)}{2}}\frac{(q_{3};q_{3})_{\nu_{c}}}{(q_{3};q_{3})_{m}(q_{3};q_{3})_{\nu_{c}-m}}:\Phi^{*}_{3,c}(X_{1})\,q_{3}^{-mD_{X_{1}}}\left(\Phi^{*}_{3,c}(X_{1})\right)^{-1}:\\\
&=\sum_{m=0}^{\infty}(-1)^{m}q_{c}^{-m}q_{3}^{\frac{m(m-1)}{2}}\frac{(q_{3};q_{3})_{\nu_{c}}}{(q_{3};q_{3})_{m}(q_{3};q_{3})_{\nu_{c}-m}}:\left(\Phi^{*}_{3,c}(X_{1})\,q_{3}^{-D_{X_{1}}}\left(\Phi^{*}_{3,c}(X_{1})\right)^{-1}\right)^{m}:\\\
&=:\left(q_{c}^{-1}\Phi^{*}_{3,c}(X_{1})\,q_{3}^{-D_{X_{1}}}\left(\Phi^{*}_{3,c}(X_{1})\right)^{-1};q_{3}\right)_{\nu_{c}}:\\\
&=:\left(\Lambda(X_{1})q_{3}^{-D_{X_{1}}};q_{3}\right)_{\nu_{c}}:,\end{split}$
(97)

where the $m$-th power is taken out to the whole vertex operator with the help
of the normal-ordering in the second line, and we used the $q$-binomial
expansion,

$\displaystyle(z;q)_{\nu}=\sum_{m=0}^{\infty}(-z)^{m}q^{\frac{m(m-1)}{2}}\frac{(q;q)_{\nu}}{(q;q)_{m}(q;q)_{\nu-m}},$
(98)

for the third line.

Note that (279) does not satisfy the gauge-invariance condition (83) by
itself, and therefore is not an R-matrix. In section 3.3.2, we will refer to
the full R-matrix $R^{(c)}=\bar{R}^{(c)}K^{(c)}$ as the Miura operator,
establishing the Miura transformation for the $q$-deformed $W$\- and
$Y$-algebras using the R-matrix properties. We will show that the expression
generalizing (279) can be obtained for the product of Miura operators,
recovering the ones used in Harada:2021xnm .

###  3.3 Miura transformation from M2-M5 intersections

We show that the R-matrices $R^{(c)}$ that we obtained from the M2-M5
intersections indeed follow from mapping the universal R-matrix of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$ to the
associated representations. We also derive the Miura transformations for the
$q$-deformed $Y$-algebras using the R-matrix properties and their expressions
in terms of the intertwiners.

####  3.3.1 Universal R-matrix of quantum toroidal algebra of
$\mathfrak{gl}(1)$

The universal R-matrix of the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$ is an element
$\mathcal{R}\in
U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\,\widehat{\otimes}\,U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$
satisfying

$\displaystyle\mathcal{R}\Delta(g)=\Delta^{\text{op}}(g)\mathcal{R},\qquad\text{for
any }g\in U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1)),$ (99a)
$\displaystyle(\text{id}\otimes\Delta)\mathcal{R}=\mathcal{R}_{13}\mathcal{R}_{12}$
(99b)
$\displaystyle(\Delta\otimes\text{id})\mathcal{R}=\mathcal{R}_{13}\mathcal{R}_{23}.$
(99c)

A direct consequence of (99) is the universal Yang-Baxter equation,

$\displaystyle\mathcal{R}_{12}\mathcal{R}_{13}\mathcal{R}_{23}=\mathcal{R}_{23}\mathcal{R}_{13}\mathcal{R}_{12}.$
(100)

Using the Drinfeld double construction of the quantum toroidal algebra of
$\mathfrak{gl}(1)$, the universal R-matrix of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$ is obtained by
the product of two parts Garbali:2021qko :

$\displaystyle\mathcal{R}=\bar{\mathcal{R}}\mathcal{K},$ (101)

where the Cartan part is

$\displaystyle\mathcal{K}=e^{\log C\otimes d+d\otimes\log C+\log
C^{\perp}\otimes d^{\perp}+d^{\perp}\otimes\log
C^{\perp}}\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}H_{-r}\otimes
H_{r}\right],$ (102)

and the rest is expanded according to the $d^{\perp}$-grading as

$\displaystyle\bar{\mathcal{R}}=1\otimes
1+\kappa_{1}\sum_{k\in\mathbb{Z}}E_{k}\otimes F_{-k}+\cdots.$ (103)

The generic expressions for the higher terms are not available as of now, due
to the lack of understanding of the Poincaré-Birkhoff-Witt (PBW) basis of the
quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$.

However, we do not need the generic expression for the universal R-matrix for
our purpose, but only the ones mapped to the representations associated to the
M2- and M5-branes. In particular, we claim that the R-matrices $R^{(c)}$, (70)
and (83), that we established from the M2-M5 intersections are recovered when
the universal R-matrix is represented on
$\text{M2}_{0,0,1}\otimes\text{M5}_{c}^{\mathbf{(1,0)}}$, justifying the
terminology.

Namely, it turns out that

$\displaystyle
R^{(c)}=(\rho_{\text{M2}_{0,0,1}}\otimes\rho_{\text{M5}_{c}^{\mathbf{(1,0)}}})\mathcal{R},\qquad
c\in\\{1,2,3\\},$ (104)

as we now show. For each $c\in\\{1,2,3\\}$, it is clear from (99a) and (104)
that the right hand side (which we just denote by $R^{(c)}$ for brevity)
should satisfy

$\displaystyle
R^{(c)}\Delta_{\text{M2}_{0,0,1},\text{M5}_{c}^{\mathbf{(1,0)}}}(g)=\Delta^{\text{op}}_{\text{M2}_{0,0,1},\text{M5}_{c}^{\mathbf{(1,0)}}}(g)R^{(c)},\qquad\text{for
any }g\in U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1)).$ (105)

Now it is straightforward to see, on one hand,

$\displaystyle
K^{(c)}=(\rho_{\text{M2}_{0,0,1}}\otimes\rho_{\text{M5}_{c}^{\mathbf{(1,0)}}})\mathcal{K},$
(106)

since $\rho_{\text{M2}_{0,0,1}}(C,C^{\perp})=(1,1)$ and
$\rho_{\text{M5}_{c}^{\mathbf{(1,0)}}}(C,C^{\perp})=(q_{c}^{\frac{1}{2}},1)$,
in particular. On the other hand, it is also obvious that

$\displaystyle\begin{split}(\rho_{\text{M2}_{0,0,1}}\otimes\rho_{\text{M5}_{c}^{\mathbf{(1,0)}}})\bar{\mathcal{R}}&=1\otimes
1+\kappa_{1}\sum_{k\in\mathbb{Z}}(\rho_{\text{M2}_{0,0,1}}\otimes\rho_{\text{M5}_{c}^{\mathbf{(1,0)}}})(E_{k}\otimes
F_{-k})+\cdots\\\ &=1-q_{c}^{-1}\xi_{c}(X_{1})Z_{1}+\cdots\end{split}$ (107)

precisely matches with the first two terms of the $d^{\perp}$-graded expansion
of $\bar{R}^{(c)}$. In appendix C, we show that there is unique solution, up
to an overall scaling, for $R^{(c)}=\bar{R}^{(c)}K^{(c)}$ satisfying (105),
(106), and (107). It is none other than our R-matrices $R^{(c)}$ obtained in
(70) and (83) associated to the M2-M5 intersections.

####  3.3.2 Miura transformation from multiple M2-M5 intersections

Having verified that the M2-M5 intersections correspond to the R-matrices of
the quantum toroidal algebra, we are now ready to construct the Miura
transformation for the $q$-deformed $Y$-algebras using the R-matrix
properties. The crucial point is that the concatenation of the R-matrices
under coproduct operations, derived from the axiom (99b), translates into the
Miura transformation being the product of Miura operators.

Let us take $L+M+N$ M5-branes which share the holomorphic support
$C^{(1,0)}=\mathbb{C}^{\times}_{X}$ but are assigned with arbitrary choices
for their topological supports, dictated by $c:I\mapsto c_{I}\in\\{1,2,3\\}$,
$I=1,2,\cdots,L+M+N$. Let us set $|c^{-1}(1)|=L$, $|c^{-1}(2)|=M$, and
$|c^{-1}(3)|=N$. Let us consider the fused surface defect algebra
$\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$ obtained by the surjective homomorphism

$\displaystyle\rho_{\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}=\left(\rho_{\text{M5}_{c_{1}}^{\mathbf{(1,0)}}}\otimes\cdots\otimes\rho_{\text{M5}_{c_{L+M+N}}^{\mathbf{(1,0)}}}\right)\Delta^{L+M+N-1}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M5}_{L,M,N}^{\mathbf{(1,0)}}.$
(108)

Although the construction appears to depend on the ordering of the M5-branes
in the tensor product, we will demonstrate in section 3.3.4 that changing the
ordering results in an isomorphic algebra. Assuming this ordering-
independence, we will simply denote this algebra as
$\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$.

Then, we consider the R-matrix between the representations $\text{M2}_{0,0,1}$
and $\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$. By repetitively applying (99b), it
is given by the product of the basic R-matrices (83),

$\displaystyle
R_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}=R_{L+M+N}^{(c_{L+M+N})}\cdots
R_{2}^{(c_{2})}R_{1}^{(c_{1})}\in\text{M2}_{0,0,1}\widehat{\otimes}\,\text{M5}_{L,M,N}^{\mathbf{(1,0)}},$
(109)

where the product is taken as elements in $\text{M2}_{0,0,1}$. Such a
decomposition of the R-matrix into the product of basic R-matrices can be
visualized as a single M2-brane passing through multiple M5-branes, where a
basic R-matrix is assigned at each intersection point (see Figure 10).

$R_{L+M+N}^{(c_{L+M+N})}$$R_{L+M+N-1}^{(c_{L+M+N-1})}$$R_{1}^{(c_{1})}$$\times$$0$$\times$$\times$$\mathcal{S}_{c_{L+M+N}}^{\mathbf{(1,0)}}$$\mathcal{S}_{c_{L+M+N-1}}^{\mathbf{(1,0)}}$$\mathcal{S}_{c_{1}}^{\mathbf{(1,0)}}$$\mathcal{L}_{0,0,1}$$\mathcal{L}_{0,0,1}$$\mathbb{C}^{\times}_{X}$$\mathbb{R}_{t}$
Figure 10: Miura transformation for the $q$-deformed $Y$-algebra from the
intersection between a single M2-brane and multiple M5-branes

Now, we may expand the product (109) as a series in the difference operator
$Z_{1}=q_{3}^{-D_{X_{1}}}\in\text{M2}_{0,0,1}$. Explicitly, the expansion
gives

$\displaystyle
R_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}=\sum_{m=0}^{\infty}(-1)^{m}T_{m}(X_{1})q_{3}^{-mD_{X_{1}}}.$
(110)

By construction, when each coefficient $T_{m}(X_{1})$ of $Z_{1}^{m}$ is
further expanded as a Laurent series in $X_{1}\in\text{M2}_{0,0,1}$, the
Laurent coefficients are valued in $\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$ since
$R_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}$ is the R-matrix. We
will show in section 3.3.3 that the coefficients in fact generate the whole
algebra $\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$, verifying that the R-matrix
$R_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}$ indeed provides the
Miura transformation for the $q$-deformed $Y$-algebra
$\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$. Moreover, we will also show that they
are completely characterized as commutants of certain screening charges in the
tensor product of the $q$-boson algebras, giving an alternative definition of
the $q$-deformed $Y$-algebra.

Let us remark that in the case of parallel M5-branes with $L=M=0$, the
expansion (110) of the R-matrix truncates, since each $R^{(3)}$ is only first-
order in $Z_{1}=q_{3}^{-D_{X_{1}}}$. As a result, there are only $N$ non-
trivial generating currents $T_{m}(X_{1})$, with $m=1,2,\cdots,N$. We will see
this is precisely the specialization that yields the $q$-deformed $W$-algebra.

The R-matrix (109) admits contour integral presentations. First, let us derive
the contour integral expression where the integrand is normal-ordered.
Substituting the contour integral presentations (89) and (94) for the basic
R-matrices into (109), we get

$\displaystyle\begin{split}&R_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}=(-1)^{L+M+N}q_{3}^{-N}\prod_{I=1}^{L+M+N}(q_{3};q_{3})_{\nu_{c_{I}}}e^{-\sum_{I=1}^{L+M+N}\frac{\log
q_{c_{I}+1}\log q_{c_{I}-1}}{\log q_{3}}a_{0}^{[I]}}\\\
&\qquad\times\oint\prod_{I=1}^{L+M+N}\frac{dX^{(I)}}{2\pi iX^{(I)}}\prod_{I\in
c^{-1}\\{1,2\\}}\frac{\left(q_{\bar{c}_{I}}^{-1}\frac{X^{(I)}}{X^{(I+1)}};q_{3}\right)_{\infty}}{\left(\frac{X^{(I)}}{X^{(I+1)}};q_{3}\right)_{\infty}}\prod_{I\in
c^{-1}(3)}\frac{1}{\left(1-\frac{X^{(I+1)}}{X^{(I)}}\right)\left(1-\frac{q_{3}^{-1}X^{(I+1)}}{X^{(I)}}\right)}\\\
&\qquad\times e^{-\sum_{I\in c^{-1}\\{1,2\\}}\frac{\log q_{\bar{c}_{I}}}{\log
q_{3}}\log\frac{X^{(I)}}{X^{(I+1)}}}\prod_{I=1}^{L+M+N}:\Phi^{*}_{I}\left(X^{(I+1)}\prod_{J=I+1}^{L+M+N}q_{c_{J}}^{\frac{1}{2}}\right)\Phi_{I}\left(X^{(I)}\prod_{J=I}^{L+M+N}q_{c_{J}}^{\frac{1}{2}}\right):\\\
&\qquad\times Z_{1}^{-\sum_{I=1}^{L+M+N}\frac{\log\frac{X^{(I)}}{X_{1}}}{\log
q_{3}}}\prod_{I=1}^{L+M+N}q_{c_{I}}^{\left.\frac{d}{2}\right|_{\text{M2}_{0,0,1}}}\end{split}$
(111)

where we introduced $L+M+N$ integration variables $X^{(I)}$,
$I=1,2,\cdots,L+M+N$ (these should not be confused with $X_{1}$, which is a
generator of $\text{M2}_{0,0,1}$). For brevity, we also used the notation
$X^{(L+M+N+1)}\equiv X_{1}$ and denoted $\Phi_{3,c_{I}}\equiv\Phi_{I}$ and
$\Phi_{3,c_{I}}^{*}\equiv\Phi^{*}_{I}$, while using the abbreviation
$a_{r}^{[I]}\equiv\left(a_{r}^{(c_{I})}\right)^{[I]}$ for the $I$-th $q$-boson
modes. There is no issue of ordering in the product of vertex operators, since
each term only involves a single $q$-boson representation. The integration is
performed starting from $X^{(1)}$ up to $X^{(L+M+N)}$, picking up the residues
from all the simple poles at $X^{(I)}=X^{(I+1)}q_{3}^{-k}$, where $k\geq 0$ if
$I\in c^{-1}\\{1,2\\}$ and $k=0,1$ if $I\in c^{-1}(3)$.

We may partially undo the normal-ordering in the integrand, by using the
normal-ordering relation (90) between the intertwiners when $I\in
c^{-1}\\{1,2\\}$. As a result, we can get rid of the ratio of $q$-Pochhammer
symbols and obtain

$\displaystyle\begin{split}&R_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}=(-1)^{L+M+N}q_{3}^{-N}\prod_{I=1}^{L+M+N}(q_{3};q_{3})_{\nu_{c_{I}}}e^{-\sum_{I=1}^{L+M+N}\frac{\log
q_{c_{I}+1}\log q_{c_{I}-1}}{\log q_{3}}a_{0}^{[I]}}\\\
&\qquad\times\oint\prod_{I=1}^{L+M+N}\frac{dX^{(I)}}{2\pi iX^{(I)}}\prod_{I\in
c^{-1}(3)}\frac{:\Phi^{*}_{I}\left(X^{(I+1)}\prod_{J=I+1}^{L+M+N}q_{c_{J}}^{\frac{1}{2}}\right)\Phi_{I}\left(X^{(I)}\prod_{J=I}^{L+M+N}q_{c_{J}}^{\frac{1}{2}}\right):}{\left(1-\frac{X^{(I+1)}}{X^{(I)}}\right)\left(1-\frac{q_{3}^{-1}X^{(I+1)}}{X^{(I)}}\right)}\\\
&\qquad\times e^{\frac{L+M}{2}\frac{\log q_{1}\log q_{2}}{\log
q_{3}}}\prod_{I\in
c^{-1}\\{1,2\\}}\Phi^{*}_{I}\left(X^{(I+1)}\prod_{J=I+1}^{L+M+N}q_{c_{J}}^{\frac{1}{2}}\right)\Phi_{I}\left(X^{(I)}\prod_{J=I}^{L+M+N}q_{c_{J}}^{\frac{1}{2}}\right)\\\
&\qquad\times Z_{1}^{-\sum_{I=1}^{L+M+N}\frac{\log\frac{X^{(I)}}{X_{1}}}{\log
q_{3}}}\prod_{I=1}^{L+M+N}q_{c_{I}}^{\left.\frac{d}{2}\right|_{\text{M2}_{0,0,1}}}.\end{split}$
(112)

This contour integral presentation explicitly shows that the R-matrix can be
reconstructed from multiple gauge-invariant M2-M5 intersections, each of which
either splits into two with ends (for $I\in c^{-1}\\{1,2\\}$) or cannot be
split (for $I\in c^{-1}(3)$) across the $I$-th M5-brane.

##### Remark

Recall that the basic R-matrices (70) and (83) are decomposed into the product
$R^{(c)}=\bar{R}^{(c)}K^{(c)}$. For the Miura transformation (109), we can
push all the Cartan parts to the right and express it as

$\displaystyle
R_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}=\bar{R}_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}K_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}},$
(113)

where

$\displaystyle\begin{split}K_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}&=K^{(c_{L+M+N})}_{L+M+N}\cdots
K^{(c_{1})}_{1}\\\
&=\prod_{I=1}^{L+M+N}V_{c_{I}}^{[I]}\left(X_{1}\prod_{J=I}^{L+M+N}q_{c_{J}}^{\frac{1}{2}}\right)\times\prod_{I=1}^{L+M+N}q_{c_{I}}^{\left.\frac{d}{2}\right|_{\text{M2}_{0,0,1}}}.\end{split}$
(114)

Here, we remind that
$V_{c_{I}}^{[I]}(X)=\prod_{j=0}^{\infty}\varphi^{+,[I]}_{c_{I}}(q_{3}^{-j}X)=\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{X^{-r}}{1-q_{3}^{r}}\otimes\frac{a_{r}^{[I]}}{q_{c_{I}}^{r/2}-q_{c_{I}}^{-r/2}}\right]$.
Then, we may separate out
$\bar{R}_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}$ from the
whole R-matrix (109) as

$\displaystyle\bar{R}_{\text{M2}_{0,0,1},\text{M5}_{L,M,N}^{\mathbf{(1,0)}}}=\bar{R}_{L+M+N}\cdots\bar{R}_{1},$
(115)

where each $\bar{R}_{I}$, $I=1,2,\cdots,L+M+N$, is computed to be

$\displaystyle\begin{split}\bar{R}_{I}&=\sum_{m=0}^{\infty}(-1)^{m}q_{c_{I}}^{-m}q_{3}^{\frac{m(m-1)}{2}}\frac{(q_{3};q_{3})_{\nu_{c_{I}}}}{(q_{3};q_{3})_{m}(q_{3};q_{3})_{\nu_{c_{I}}-m}}:f_{I}(X_{1})q_{3}^{-mD_{X_{1}}}f_{I}(X_{1})^{-1}:\\\
&=\sum_{m=0}^{\infty}(-1)^{m}q_{c_{I}}^{-m}q_{3}^{\frac{m(m-1)}{2}}\frac{(q_{3};q_{3})_{\nu_{c_{I}}}}{(q_{3};q_{3})_{m}(q_{3};q_{3})_{\nu_{c_{I}}-m}}:\left(f_{I}(X_{1})q_{3}^{-D_{X_{1}}}f_{I}(X_{1})^{-1}\right)^{m}:\\\
&=:\left(q_{c_{I}}^{-1}f_{I}(X_{1})q_{3}^{-D_{X_{1}}}f_{I}(X_{1})^{-1};q_{3}\right)_{\nu_{c_{I}}}:\\\
&\equiv:\left(\Lambda_{I}(X_{1})q_{3}^{-D_{X_{1}}};q_{3}\right)_{\nu_{c_{I}}}:,\end{split}$
(116)

where we defined

$\displaystyle
f_{I}(X_{1}):=\Phi^{*}_{I}\left(X_{1}\prod_{J=I+1}^{L+M+N}q_{c_{J}}^{\frac{1}{2}}\right)\prod_{J=I+1}^{L+M+N}V_{c_{J}}^{[J]}\left(X_{1}\prod_{K=J}^{L+M+N}q_{c_{K}}^{\frac{1}{2}}\right).$
(117)

and

$\displaystyle\begin{split}\Lambda_{I}(X_{1})&:=q_{c_{I}}^{-1}:f_{I}(X_{1})f_{I}(q_{3}^{-1}X_{1})^{-1}:\\\
&=q_{c_{I}}^{-1}e^{-\frac{\prod_{a=1}^{3}\log q_{a}}{(\log
q_{c_{I}})^{2}}a_{0}^{[I]}}\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{\prod_{J=I}^{L+M+N}q_{c_{J}}^{r/2}}{(q_{c_{I}}^{r/2}-q_{c_{I}}^{-r/2})^{2}}a_{-r}^{[I]}X_{1}^{r}\right]\\\
&\times\exp\left[\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\frac{\prod_{J=I+1}^{L+M+N}q_{c_{J}}^{-r/2}}{(q_{c_{I}}^{r/2}-q_{c_{I}}^{-r/2})^{2}}a_{r}^{[I]}X_{1}^{-r}+\sum_{r=1}^{\infty}\frac{\kappa_{r}}{r}\sum_{J=I+1}^{L+M+N}\frac{\prod_{K=J}^{L+M+N}q_{c_{K}}^{-r/2}}{q_{c_{J}}^{r/2}-q_{c_{J}}^{-r/2}}a_{r}^{[J]}X_{1}^{-r}\right].\end{split}$
(118)

The operator (116) is precisely what was suggested to be the Miura operator
for the $q$-deformed $Y$-algebra in Harada:2021xnm , after properly adjusting
the convention. In this sense, the Miura transformation (109) that we
established is related to the one there, realized by (115), by a
multiplication of the vertex operators $V_{c_{I}}^{[I]}$ to the generating
currents $T_{m}$. Note, however, that neither (115) nor individual (116) is an
R-matrix of the quantum toroidal algebra
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$, unlike our
Miura transformation (109) and Miura operators (83). This is why (116)
involves not only the $I$-th $q$-boson but also the mixing from all the $J$-th
$q$-bosons with $J>I$.

####  3.3.3 Screening charges from M2-branes between M5-branes

To fully characterize $\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$ as a subalgebra of
the tensor product of $q$-bosons, we will identify
$\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$ as commutants of a certain set of
operators called screening charges. First, we will construct the screening
charges by the M2-brane placed between each adjacent pair of M5-branes in the
tensor product, and show that they commute with all the generators of
$\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$. Then, we show that there is no more
operator that commutes with all the generators produced by the product of
Miura operators. We thereby prove that the product of Miura operators indeed
gives the Miura transformation for $\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$, the
$q$-deformed $Y$-algebra.

Take any consecutive pair; the $i$-th and the $(i+1)$-th M5-branes. The only
two possibility for the topological part of their support are the following:
1) the two share two topological planes
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}$;
2) the $i$-th M5-brane is supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}$
and the $(i+1)$-th M5-brane is supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c}}\times\mathbb{R}^{2}_{{\varepsilon}_{c\mp
1}}$, so that the two share only one topological plane
$\mathbb{R}^{2}_{{\varepsilon}_{c\mp 1}}$. In the first case, the M2-branes
supported on $\mathbb{R}^{2}_{{\varepsilon}_{c\pm 1}}\times\\{\text{finite
interval}\\}$ can be placed between the two M5-branes (in the
$\mathbb{R}_{t}$-direction), ending on the two M5-branes from above and from
below, respectively. In the second case, the M2-brane supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c\mp 1}}\times\\{\text{finite interval}\\}$
can be placed between the M5-branes, ending on the two from above and from
below, respectively. See Figure 11.

$\Phi_{c\pm 1,c}$$\mathcal{L}_{c\pm 1}$$\Phi^{*}_{c\pm
1,c}$$\times$$0$$\times$$0$$\mathcal{S}_{c}^{\mathbf{(1,0)}}$$\mathcal{S}_{c}^{\mathbf{(1,0)}}$$\mathbb{C}^{\times}_{X}$$\mathbb{R}_{t}$

$\mathcal{L}_{c\mp 1}$$\Phi_{c\mp 1,c\pm 1}$$\Phi^{*}_{c\mp
1,c}$$\times$$0$$\times$$0$$\mathcal{S}_{c\pm
1}^{\mathbf{(1,0)}}$$\mathcal{S}_{c}^{\mathbf{(1,0)}}$

Figure 11: Screening charges from M2-branes stretched between M5-branes

In section 3.2.1, we have associated the intertwiners $\Phi$ and $\Phi^{*}$ of
the quantum toroidal algebra of $\mathfrak{gl}(1)$ to the M2-M5 intersections
with ends. We construct the screening charges by composing them along the
finite support of the M2-brane. Specifically, for the first case where the two
adjacent M5-branes share two topological planes
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}$,
there are two screening charges obtained as

$\displaystyle\begin{split}&S_{c,c}^{c\pm 1}:=\sigma\circ\left(\Phi_{c\pm
1,c}\otimes\Phi^{*}_{c\pm
1,c}\right)\in\text{M5}_{c_{i}=c}^{\mathbf{(1,0)}}\otimes\text{M5}^{\mathbf{(1,0)}}_{c_{i+1}=c}\end{split}$
(119)

The term in the parenthesis should be understood as the composition of
$\Phi_{c\pm 1,c}$ and $\Phi^{*}_{c\pm 1,c}$ as elements in
$\text{Hom}(\mathcal{V}_{c\pm 1}(u),\mathbb{C})$ and
$\text{Hom}(\mathbb{C},\mathcal{V}_{c\pm 1}(u))$, respectively, so that
$\Phi_{c\pm 1,c}\otimes\Phi^{*}_{c\pm
1,c}\in\text{M5}^{\mathbf{(1,0)}}_{c_{i+1}=c}\otimes\text{M5}^{\mathbf{(1,0)}}_{c_{i}=c}$.
Then, the exchange operator $\sigma$ rearranges the ordering of the two
$q$-boson representations in the tensor product as above.

Inserting the complete set of the orthonormal basis states,
$\text{id}=\sum_{i\in\mathbb{Z}}[u]_{i}^{(c\pm 1)}\otimes[u]^{(c\pm
1)*}_{i}\in\text{End}(\mathcal{V}_{c\pm 1}(u))$ ($[u]_{j}^{(c\pm
1)*}([u]_{i}^{(c\pm 1)})=\delta_{i,j}$), we can express the above screening
charges (119) as

$\displaystyle S^{c\pm 1}_{c,c}=\sum_{i\in\mathbb{Z}}(\Phi^{*}_{c\pm
1,c})_{i}\otimes\left(\Phi_{c\pm
1,c}\right)_{i}\equiv\sum_{i\in\mathbb{Z}}\mathcal{S}_{c,c}^{c\pm 1}(uq_{c\pm
1}^{i}).$ (120)

The summand $\mathcal{S}_{c,c}^{c\pm 1}(X)$ is called the screening current.
The summation over integers can be thought of as a trace over the Hilbert
space $\mathcal{V}_{c\pm 1}(u)$ of the line defect worldline theory, which was
to be expected since the M2-brane is stretched between the M5-branes.
Inserting the components of the intertwiners, the two screening charges are
explicitly given by

$\displaystyle\begin{split}S_{c,c}^{c\pm 1}&=q_{c\mp 1}^{-d^{\perp}\otimes
1+1\otimes d^{\perp}}\sum_{i\in\mathbb{Z}}(uq_{c\pm 1}^{i})^{-\frac{\log
q_{c\mp 1}}{\log q_{c}}(a_{0}^{(c)}\otimes 1-1\otimes a_{0}^{(c)})-\frac{\log
q_{c\mp 1}}{\log q_{c\pm 1}}}\\\
&\qquad\qquad\qquad\times\exp\left[\sum_{r=1}^{\infty}\frac{1}{r}\frac{q_{c\mp
1}^{r/2}-q_{c\mp 1}^{-r/2}}{q_{c}^{r/2}-q_{c}^{-r/2}}q_{c\mp
1}^{-r/2}(a_{-r}^{(c)}\otimes 1-1\otimes q_{c}^{-r/2}a^{(c)}_{-r})(uq_{c\pm
1}^{i})^{r}\right]\\\
&\qquad\qquad\qquad\times\exp\left[-\sum_{r=1}^{\infty}\frac{1}{r}\frac{q_{c\mp
1}^{r/2}-q_{c\mp 1}^{-r/2}}{q_{c}^{r/2}-q_{c}^{-r/2}}q_{c\mp
1}^{r/2}(q_{c}^{r/2}a_{r}^{(c)}\otimes 1-1\otimes a_{r}^{(c)})(uq_{c\pm
1}^{i})^{-r}\right],\end{split}$ (121)

This precisely recovers the bosonic screening charges that was used to define
the $q$-deformed $W$-algebras, up to the decoupling of a single $q$-boson
Feigin:1995sf ; 2015arXiv151208779B .

On the other hand, in the second case where the adjacent M5-branes share only
one topological plane $\mathbb{R}^{2}_{{\varepsilon}_{c\mp 1}}$ in their
support, there is a single screening charge obtained as

$\displaystyle S_{c,c\pm 1}^{c\mp 1}:=\sigma\circ\left(\Phi_{c\mp 1,c\pm
1}\otimes\Phi^{*}_{c\mp
1,c}\right)\in\text{M5}_{c_{i}=c}^{\mathbf{(1,0)}}\otimes\text{M5}^{\mathbf{(1,0)}}_{c_{i+1}=c\pm
1}.$ (122)

Inserting the complete set of basis
$\text{id}=\sum_{i\in\mathbb{Z}}[u]_{i}^{(c\mp 1)}\otimes[u]_{i}^{(c\mp 1)*}$,
we can express it into

$\displaystyle S_{c,c\pm 1}^{c\mp 1}=\sum_{i\in\mathbb{Z}}(\Phi^{*}_{c\mp
1,c})_{i}\otimes(\Phi_{c\mp 1,c\pm
1})_{i}\equiv\sum_{i\in\mathbb{Z}}\mathcal{S}_{c,c\pm 1}^{c\mp 1}(uq_{c\mp
1}^{i}).$ (123)

Explicitly, we have

$\displaystyle\begin{split}S_{c,c\pm 1}^{c\mp 1}&=q_{c\pm
1}^{-d^{\perp}\otimes 1}q_{c}^{1\otimes
d^{\perp}}\sum_{i\in\mathbb{Z}}(uq_{c\mp 1}^{i})^{-\frac{\log q_{c\pm 1}}{\log
q_{c}}(a_{0}^{(c)}\otimes 1)+\frac{\log q_{c}}{\log q_{c\pm 1}}(1\otimes
a_{0}^{(c\pm 1)})-\frac{\log q_{c}}{\log q_{c\mp 1}}}\\\
&\times\exp\left[\sum_{r=1}^{\infty}\frac{1}{r}\left(\frac{q_{c\pm
1}^{r/2}-q_{c\pm 1}^{-r/2}}{q_{c}^{r/2}-q_{c}^{-r/2}}q_{c\pm
1}^{-r/2}(a_{-r}^{(c)}\otimes 1)-\frac{q_{c}^{r/2}-q_{c}^{-r/2}}{q_{c\pm
1}^{r/2}-q_{c\pm 1}^{-r/2}}q_{c\mp 1}^{r/2}(1\otimes a_{-r}^{(c\pm
1)})\right)(uq_{c\mp 1}^{i})^{r}\right]\\\
&\times\exp\left[-\sum_{r=1}^{\infty}\frac{1}{r}\left(\frac{q_{c\pm
1}^{r/2}-q_{c\pm 1}^{-r/2}}{q_{c}^{r/2}-q_{c}^{-r/2}}q_{c\mp
1}^{-r/2}(a_{r}^{(c)}\otimes 1)-\frac{q_{c}^{r/2}-q_{c}^{-r/2}}{q_{c\pm
1}^{r/2}-q_{c\pm 1}^{-r/2}}q_{c}^{r/2}(1\otimes a_{r}^{(c\pm
1)})\right)(uq_{c\mp 1}^{i})^{-r}\right].\end{split}$ (124)

Note that this precisely matches with the fermionic screening charge obtained
in 2015arXiv151208779B .

The gauge-invariance conditions (75) and (79) for the intertwiners directly
imply that the screening charges commute with the generators of the fused
surface defect algebra. Namely,

$\displaystyle\left[S_{c_{i},c_{i+1}}^{c_{\text{M2}}},\Delta_{\text{M5}_{c_{i}}^{\mathbf{(1,0)}},\text{M5}_{c_{i+1}}^{\mathbf{(1,0)}}}(g)\right]=0,\qquad\text{for
any }g\in U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1)),$ (125)

where $c_{\text{M2}}=c+1\;\text{or}\;c-1$ if $(c_{i},c_{i+1})=(c,c)$ and
$c_{\text{M2}}=c\mp 1$ if $(c_{i},c_{i+1})=(c,c\pm 1)$. Indeed, a
straightforward computation gives

$\displaystyle\begin{split}&\Delta_{\text{M5}_{c_{i}}^{\mathbf{(1,0)}},\text{M5}_{c_{i+1}}^{\mathbf{(1,0)}}}(g)S_{c_{i},c_{i+1}}^{c_{\text{M2}}}\\\
&=\sigma\left(\rho_{\text{M5}_{c_{i+1}}^{\mathbf{(1,0)}}}(g_{(2)})\otimes\rho_{\text{M5}_{c_{i+1}}^{\mathbf{(1,0)}}}(g_{(1)})\right)\left(\Phi_{c_{\text{M2}},c_{i+1}}\otimes\Phi^{*}_{c_{\text{M2}},c_{i}}\right)\\\
&=\sigma\left[\Phi_{c_{\text{M2}},c_{i+1}}\left(\rho_{\text{M2}}\otimes\rho_{\text{M5}_{c_{i+1}}^{\mathbf{(1,0)}}}\otimes\rho_{\text{M5}_{c_{i}}^{\mathbf{(1,0)}}}\right)(g_{(2),(1)}\otimes
g_{(2),(2)}\otimes g_{(1)})\Phi^{*}_{c_{\text{M2}},c_{i}}\right]\\\
&=\sigma\left[\Phi_{c_{\text{M2}},c_{i+1}}\left(\rho_{\text{M2}}\otimes\rho_{\text{M5}_{c_{i+1}}^{\mathbf{(1,0)}}}\otimes\rho_{\text{M5}_{c_{i}}^{\mathbf{(1,0)}}}\right)(\text{id}\otimes\sigma)(\sigma\otimes\text{id})(g_{(1)}\otimes
g_{(2),(1)}\otimes g_{(2),(2)})\Phi^{*}_{c_{\text{M2}},c_{i}}\right]\\\
&=\sigma\left[\Phi_{c_{\text{M2}},c_{i+1}}\left(\rho_{\text{M2}}\otimes\rho_{\text{M5}_{c_{i+1}}^{\mathbf{(1,0)}}}\otimes\rho_{\text{M5}_{c_{i}}^{\mathbf{(1,0)}}}\right)(\text{id}\otimes\sigma)(\sigma\otimes\text{id})(g_{(1),(1)}\otimes
g_{(1),(2)}\otimes g_{(2)})\Phi^{*}_{c_{\text{M2}},c_{i}}\right]\\\
&=\sigma\left[\Phi_{c_{\text{M2}},c_{i+1}}\left(\rho_{\text{M2}}\otimes\rho_{\text{M5}_{c_{i+1}}^{\mathbf{(1,0)}}}\otimes\rho_{\text{M5}_{c_{i}}^{\mathbf{(1,0)}}}\right)(g_{(1),(2)}\otimes
g_{(2)}\otimes g_{(1),(1)})\Phi^{*}_{c_{\text{M2}},c_{i}}\right]\\\
&=\sigma\left[\left(\Phi_{c_{\text{M2}},c_{i+1}}\otimes\Phi^{*}_{c_{\text{M2}},c_{i}}\right)\left(\rho_{\text{M5}_{c_{i+1}}^{\mathbf{(1,0)}}}(g_{(2)})\otimes\rho_{\text{M5}_{c_{i}}^{\mathbf{(1,0)}}}(g_{(1)})\right)\right]\\\
&=S_{c_{i},c_{i+1}}^{c_{\text{M2}}}\Delta_{\text{M5}_{c_{i}}^{\mathbf{(1,0)}},\text{M5}_{c_{i+1}}^{\mathbf{(1,0)}}}(g),\end{split}$
(126)

where we used the Sweedler’s notation $\Delta(g)=g_{(1)}\otimes g_{(2)}$
omitting the summation. We used (75) for the second equality; the
coassociativity
$(\Delta\otimes\text{id})\Delta=(\text{id}\otimes\Delta)\Delta$ for the fourth
equality; and (79) for the sixth equality. The commutativity (125) was proven
in 2015arXiv151208779B ; Harada:2021xnm  through brute-force computation. It
is worth noting that realizing the screening charges in terms of the gauge-
invariant M2-M5 intersections allows us to avoid the labor.

Further, we can show that the set of screening charges that we obtained is
maximal; in other words, there is no more operator in the tensor product
$\text{M5}_{c_{1}}^{\mathbf{(1,0)}}\otimes\text{M5}_{c_{2}}^{\mathbf{(1,0)}}$
of two $q$-bosons which commutes with all the generators produced by the
product $R^{(c_{2})}_{2}R^{(c_{1})}_{1}$ of two R-matrices. We emphasize that,
in deriving this, it is crucial to have the generating series factorized into
the product of two R-matrices, providing a systematic way to solve the
commutativity constraint. Details can be found in appendix D.

This completes the proof of the two equalities, as promised. First, the
product (109) of R-matrices, when expanded in the generators of
$\text{M2}_{0,0,1}$ as in (110), produces the full set of generators of the
$q$-deformed $Y$-algebra $\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$. Thus, our
R-matrix (109) can be properly called the Miura transformation for the
$q$-deformed $Y$-algebra. Second, the $q$-deformed $Y$-algebra
$\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$ is completely characterized as the
commutants of the screening charges identified in (121) and (124). We may
regard this statement as an alternative definition of the $q$-deformed
$Y$-algebra.

####  3.3.4 Ordering-independence by Yang-Baxter equation

The Miura transformation for the $q$-deformed $Y$-algebra should not depend on
the ordering of the Miura operators. Since we identified the Miura operators
as R-matrices of the quantum toroidal algebra, the change of ordering of the
Miura operators is simply implemented by the universal Yang-Baxter equation
(see Figure 12).

Specifically, let us consider the R-matrices between the $q$-boson
representations by

$\displaystyle
R_{\text{M5}_{c}^{\mathbf{(1,0)}},\text{M5}_{c^{\prime}}^{\mathbf{(1,0)}}}=\left(\rho_{\text{M5}_{c}^{\mathbf{(1,0)}}}\otimes\rho_{\text{M5}_{c^{\prime}}^{\mathbf{(1,0)}}}\right)\mathcal{R}\in\text{M5}_{c}^{\mathbf{(1,0)}}\widehat{\otimes}\,\text{M5}_{c^{\prime}}^{\mathbf{(1,0)}},\qquad
c,c^{\prime}\in\\{1,2,3\\}.$ (127)

We may call these R-matrices the $q$-deformed Maulik-Okounkov R-matrices. For
other studies on these R-matrices between $q$-boson representations, see
Fukuda:2017qki ; Garbali:2020sll ; Negut:2020npc ; Garbali:2021qko .

Now, the universal Yang-Baxter equation (100) represented on
$\text{M2}_{0,0,1}\otimes\text{M5}_{c_{I}}^{\mathbf{(1,0)}}\otimes\text{M5}_{c_{I+1}}^{\mathbf{(1,0)}}$
directly gives

$\displaystyle
R_{\text{M5}_{c_{I}}^{\mathbf{(1,0)}},\text{M5}_{c_{I+1}}^{\mathbf{(1,0)}}}R_{I+1}^{(c_{I+1})}R_{I}^{(c_{I})}=R_{I}^{(c_{I})}R_{I+1}^{(c_{I+1})}R_{\text{M5}_{c_{I}}^{\mathbf{(1,0)}},\text{M5}_{c_{I+1}}^{\mathbf{(1,0)}}},\qquad\text{for
any }I.$ (128)

As a result, the Miura transformations with different orderings of Miura
operators produce isomorphic $q$-deformed $Y$-algebras, which thereby can be
written as $\text{M5}_{L,M,N}^{\mathbf{(1,0)}}$ without ambiguity.

$R_{I+1}^{(c_{I+1})}$$R_{I}^{(c_{I})}$$\times$$0$$\times$$0$$\mathcal{S}_{c_{I+1}}^{\mathbf{(1,0)}}$$\mathcal{S}_{c_{I}}^{\mathbf{(1,0)}}$$\mathcal{L}_{0,0,1}$$\mathcal{L}_{0,0,1}$$\mathbb{C}^{\times}_{X}$$\mathbb{R}_{t}$

$R_{I}^{(c_{I})}$$R_{I+1}^{(c_{I+1})}$$\times$$0$$\times$$0$$\mathcal{S}_{c_{I}}^{\mathbf{(1,0)}}$$\mathcal{S}_{c_{I+1}}^{\mathbf{(1,0)}}$$\mathcal{L}_{0,0,1}$$\mathcal{L}_{0,0,1}$

Figure 12: Change of ordering in surface defect fusion, and product of
R-matrices related by a conjugation of the $q$-deformed Maulik-Okounkov
R-matrix.

####  3.3.5 R-matrices with multiple M2-branes

So far in this section, we have considered the intersections between a single
M2-brane and multiple M5-branes, leading to the Miura transformation for the
$q$-deformed $Y$-algebras. Here, we will consider the intersections between
multiple M2-branes and a single M5-brane.

Let us fix the support of the M5-brane to be
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{C}^{\times}_{X}$.
The support of the $i$-th M2-brane is determined according to the function
$c:i\mapsto c_{i}\in\\{1,2,3\\}$, to be
$\mathbb{R}^{2}_{c_{i}}\times\mathbb{R}_{t}$. In particular, we set
$|c^{-1}(1)|=l$, $|c^{-1}(2)|=m$, and $|c^{-1}(3)|=n$. The algebra of local
operators on the resulting line defect can be reconstructed by the composing
the tensor product of the respective representations and the coproduct,

$\displaystyle\rho_{\text{M2}_{l,m,n}}=\left(\rho_{\text{M2}_{c_{l+m+n}}}\otimes\cdots\otimes\rho_{\text{M2}_{c_{1}}}\right)\Delta^{l+m+n-1}:U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))\twoheadrightarrow\text{M2}_{l,m,n}.$
(129)

The resulting M2-brane algebra does not depend on the ordering of the
M2-branes in the tensor product, as we will show shortly. Thus, we will denote
this algebra as $\text{M2}_{l,m,n}$ without ambiguity.

The intersection between the fused line defect and the surface defect is
decomposed into multiple intersections between the individual line defects and
the surface defect (see Figure 13). Correspondingly, by repetitively applying
(99c) the R-matrix between the representations $\text{M2}_{l,m,n}$ and
$\text{M5}_{0,0,1}^{\mathbf{(1,0)}}$ is obtained by taking the product of the
basic R-matrices:

$\displaystyle
R_{\text{M2}_{l,m,n},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}=R_{\text{M2}_{c_{l+m+n}},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}\cdots
R_{\text{M2}_{c_{2}},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}R_{\text{M2}_{c_{1}},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}\in\text{M2}_{l,m,n}\,\widehat{\otimes}\,\text{M5}_{0,0,1}^{\mathbf{(1,0)}}.$
(130)

Thus, when the product on the right hand side is expanded, the terms recombine
into the elements in
$\text{M2}_{l,m,n}\,\widehat{\otimes}\,\text{M5}_{0,0,1}^{\mathbf{(1,0)}}$ by
construction. In the case of
$\mathcal{C}=\mathbb{C}\times\mathbb{C}^{\times}$, such a recombination of the
product of Miura operators was observed in Gaiotto:2020dsq  at the example of
two M2-branes passing through a single M5-brane. Our construction of the Miura
operators as R-matrices provides a natural explanation for this recombination.

Moreover, let us consider the R-matrices between the quantum torus algebras,

$\displaystyle
R_{\text{M2}_{c},\text{M2}_{c^{\prime}}}=\left(\rho_{\text{M2}_{c}}\otimes\rho_{\text{M2}_{c^{\prime}}}\right)\mathcal{R}\in\text{M2}_{c}\,\widehat{\otimes}\,\text{M2}_{c^{\prime}},\qquad
c,c^{\prime}\in\\{1,2,3\\}.$ (131)

Then, the universal Yang-Baxter equation (100) implies

$\displaystyle
R_{\text{M2}_{c_{i+1}},\text{M2}_{c_{i}}}R_{\text{M2}_{c_{i+1}},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}R_{\text{M2}_{c_{i}},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}=R_{\text{M2}_{c_{i}},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}R_{\text{M2}_{c_{i+1}},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}R_{\text{M2}_{c_{i+1}},\text{M2}_{c_{i}}},$
(132)

so that changing the order of any two adjacent basic R-matrices results in an
equivalent R-matrix, up to a conjugation by
$R_{\text{M2}_{c_{i+1}},\text{M2}_{c_{i}}}$.

$R_{1}$$R_{l+m+n}$$R_{2}$$\times$$0$$\mathcal{S}_{0,0,1}^{\mathbf{(1,0)}}$$\mathcal{L}_{c_{1}}$$\mathcal{L}_{c_{2}}$$\mathcal{L}_{c_{l+m+n}}$$\mathbb{C}^{\times}_{X}$$\mathbb{R}_{t}$
Figure 13: Intersection between multiple M2-branes and a single M5-brane, and
associated R-matrices. Here, we abbreviated $R_{i}\equiv
R_{\text{M2}_{c_{i}},\text{M5}_{0,0,1}^{\mathbf{(1,0)}}}$.

##  4 Miura transformation from IIB string theory

So far, we have analyzed the modules of the quantum toroidal algebra in terms
of branes in twisted M-theory. In this Section, we will give an alternate
derivation of our results in terms of Type IIB string theory. It is well known
that M-theory compactified on a 2-torus is the same theory as type IIB
compactified on a circle. Here, we have two natural tori in M-theory on which
we can perform the duality to Type IIB: one possibility is to consider the
torus of phases in the internal Calabi-Yau manifold
$\mathbb{R}^{2}_{{\varepsilon}_{1}}\times\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}$,
whose action on local coordinates is

$(z_{1},z_{2},z_{3})\mapsto(e^{i\alpha_{1}}\,z_{1},e^{i\alpha_{2}}\,z_{2},e^{-i(\alpha_{1}+\alpha_{2})}\,z_{3})\;.$
(133)

The Calabi-Yau geometry used in the M-theory compactification is mapped to a
$\mathbf{(p,q)}$ web of 5-branes in type IIB. In that picture, the M5-branes
become D3-branes supported on 4-manifolds with codimension-2 “corners”, where
the chiral algebras $Y_{L,M,N}$ live Gaiotto:2017euk , while the M2-branes
become strings.

Alternatively, we can view
$\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$ in the M-theory
background as a torus fibration:

$S^{1}_{X}(R_{X})\times
S^{1}_{Z}(R_{Z})\rightarrow\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}\rightarrow\mathbb{R}_{x}\times\mathbb{R}_{z}\;.$
(134)

In this picture, the complex structure $\tau=i\,R_{X}/R_{Z}$ of the M-theory
torus is identified with the type IIB coupling (dilaton field242424We are
assuming throughout that the axion field of Type IIB is turned off, or
equivalently that the torus in M-theory is rectangular.) $\tau=i\,g^{-1}_{s}$.
The duality turns the M5-branes into $\mathbf{(p,q)}$ 5-branes, and the
M2-branes into D3-branes, all wrapping a common circle $S^{1}(R)$ of radius

$R=\frac{1}{m^{3}_{pl}\,R_{x}\,R_{z}}\;,$ (135)

with $m_{pl}$ the Planck mass. The support of the branes is displayed in Table
6. This Type IIB frame makes it possible to understand the underlying quantum
toroidal symmetry in a very direct way. To our knowledge, the earliest example
comes from the work Awata:2011ce , which reinterpreted 5-brane junctions as
intertwiners of Fock modules in the toroidal algebra, see also
Bourgine:2016vsq ; Bourgine:2017jsi ; Mironov:2016yue . Since then, the
dictionary has been extended in a series of paper by Zenkevich
Zenkevich:2018fzl ; Zenkevich:2020ufs ; 2022arXiv221214808Z ;
Zenkevich:2023cza  and unpublished work by Aganagic-Zenkevich: the quantum
toroidal generators have been identified as $\mathbf{(p,q)}$ strings, the
vector representations as D3-branes, the Fock representations as
$\mathbf{(p,q)}$ 5-branes, and the MacMahon representations as
$\mathbf{(p,q)}$ 7-branes (though this last identification is still at the
conjectural stage). See also Awata:2017lqa ; Bourgine:2019phm  for extensions
to the quantum toroidal algebras of higher-rank.

For instance, back in section 2.1.1, we showed that the algebra of local
operators on a single M2-brane in the twisted M-theory is given by a quantum
torus algebra, furnishing a vector representation of
$U_{q_{1},q_{2},q_{3}}(\widehat{\widehat{\mathfrak{gl}}}(1))$; we can now
provide an alternative perspective for this statement in the dual IIB theory,
by mapping the M2-brane to a D3-brane: the vector representation has central
charges $(C,C^{\perp})=(1,1)$, so in particular is invariant under the Miki
automorphism action $C^{\perp}\rightarrow
C\rightarrow(C^{\perp})^{-1}\rightarrow C^{-1}\rightarrow C^{\perp}$, but this
is precisely the property of a D3-brane, the only $SL(2,\mathbb{Z})$-invariant
object of type IIB (regardless of whether one turns on an $\Omega$-background
or not). Furthermore, recall from (27) that this vector representation can be
realized via the action of $q_{c}$-difference operators on states
$[u]_{i}^{(c)}$, for $u\in\mathbb{C}^{\times}$ and $i\in\mathbb{Z}$, where
each state belongs in the vector space
$\mathcal{V}_{c}(u)=\bigoplus_{i\in\mathbb{Z}}\mathbb{C}[u]_{i}^{(c)}$. If we
label each state $[u]_{i}^{(c)}$ as a function $f(X)$ of the $X$-coordinate,
then note that the “raising” mode $E_{0}$ and the “Cartan” mode $H_{1}$ act as

$E_{0}\,f(X)\sim f(q_{c}\,X)\;,\qquad\qquad H_{1}\,f(X)\rightarrow X\,f(X)\;,$
(136)

according to (26). Since the grading of $E_{0}$ is $(d^{\perp},\,d)=(1,\,0)$
and the grading of $H_{1}$ is $(d^{\perp},\,d)=(0,\,1)$, and since $Z$ and $X$
are literally complex coordinates on the M-theory torus, these actions are
naturally interpreted as adding 1 unit of Kaluza-Klein momentum along
$S^{1}_{Z}$ and along $S^{1}_{X}$ on the torus, respectively.252525And more
generally, $Z^{m}\,X^{n}$ carries $m$ units of KK momentum along $S^{1}_{Z}$
and $n$ units of KK momentum along $S^{1}_{X}$ on the M-theory torus. In Type
IIB, this diagonal action of $H_{1}$ is precisely that of a fundamental string
wrapped around $S^{1}(R)$ and ending on a D3-brane, adding one unit of
electric charge.

Although it is not immediately apparent, the action of $E_{0}$ is likewise
that of a D-string wrapped around $S^{1}(R)$ ending on the D3-brane; this is
because a D-string turns on one unit of vortex flux on a D3-brane. One would
naively expect a soliton singularity to be present at the D-string endpoint
location, but this is not so in an $\Omega$-background Nekrasov:2010ka :
remarkably, turning on $k$ units of vortex flux there has the sole effect of
shifting $X\rightarrow q^{k}_{c}\,X$ . In more detail, consider a D3-brane
supported on $\mathbb{R}^{2}_{{\varepsilon}_{c}}\times\mathbb{R}_{t}\times
S^{1}(R)$. Its fluctuation in the $\mathbb{R}_{x}$ direction is a Coulomb
modulus, complexified by the holonomy around $S^{1}(R)$, and denoted as $X$.
In the $\Omega$-background, $X$ always appears as part of a product
$X\,e^{R\,{\varepsilon}_{c}\,y_{c}\,D_{y_{c}}}$ in the low energy effective
Lagrangian on the brane, with $D_{y_{c}}=\partial_{y_{c}}+A_{y_{c}}$ the
covariant derivative on $\mathbb{R}^{2}_{{\varepsilon}_{c}}$ of complex
coordinate $y_{c}$. A vortex of charge $k$ induces a gauge field profile
$A_{y_{c}}=k/y_{c}$, or equivalently a shift of the D3-brane modulus as
$X\rightarrow q^{k}_{c}\,X$, as claimed; for $k=1$, this is exactly the
$E_{0}$ action (136).

It is far more involved to see directly that $\mathbf{(p,q)}$ 5-branes are
Fock modules of the quantum toroidal algebra. The idea is that
$({\bf{p}},{\bf{q}})$ strings are now instantons in the 5-branes, meaning the
quantum toroidal algebra generators are acting on the instanton moduli space
of the 5-brane theory. More precisely, for 5-branes supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}\times
l^{({\bf{p}},{\bf{q}})}\times S^{1}(R)$, this is an action on the equivariant
K-theoretic moduli space of instantons on
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}$.
The equivariant action is the corresponding $U(1)\times U(1)$ on this part of
the $\Omega$-background, with parameters $q_{a}$ and $q_{b}$. In the limit
$R\rightarrow 0$, the Hilbert space of BPS states would be the cohomology of
the moduli space, that is the Higgs branch of the instanton quantum mechanics
on $l^{({\bf{p}},{\bf{q}})}$. For us, $S^{1}(R)$ has finite size, and the
instantons are strings wrapping that circle. The resulting Kaluza-Klein tower
of states implies that the Hilbert space of BPS states is the K-theory of the
moduli space rather than its cohomology.

It has been shown that this instanton moduli space admits a realization as a
Fock space 2009arXiv0904.1679F ; 2009arXiv0905.2555S .262626Note that for a
Fock representation of central charges $(C,C^{\perp})=(q_{c}^{N/2},\,1)$, this
realization implies, in the $R\rightarrow 0$ limit, a mathematical proof of
the Alday-Gaiotto-Tachikawa-Wyllard conjecture Alday:2009aq ; Wyllard:2009hg ,
for the algebra ${\cal W}(\text{sl}(N))\oplus u(1)$ 2012arXiv1202.2756S ;
2012arXiv1211.1287M . It follows from their construction that a
$({\bf{p}},{\bf{q}})$ 5-brane is a Fock representation with central charges
$(C,C^{\perp})=(q^{\mathbf{p}/2}_{c},q^{\mathbf{q}/2}_{c})$.

An elementary Miura operator is therefore realized as the intersection of an
NS5-brane and a D3-brane in the type IIB frame. We will now analyze how it can
be recovered from a gauge theory supersymmetric index.

###  4.1 Miura operators from supersymmetric index

A generic configuration of Type IIB fivebranes and D3-branes in table 6 will
preserve at least 2 supercharges, even in the presence of the
$\Omega$-background; the supersymmetry in question is the reduction of 2d
$\mathcal{N}=(0,2)$ supersymmetry to 1d. The spectrum of protected BPS
operators in such a brane system is captured by an index of the type IIB
string theory on $M_{10}$ in the presence of the branes, which is the index of
this supersymmetric quantum mechanics along the thermal circle $S^{1}(R)$.

The simplest configuration consists of a single NS5-brane, say a NS51,0,0
supported on
$\mathbb{R}^{2}_{{\varepsilon}_{2}}\times\mathbb{R}^{2}_{{\varepsilon}_{3}}$,
intersecting a single D3-brane in spacetime, say a D30,0,1 brane supported on
$\mathbb{R}^{2}_{{\varepsilon}_{3}}$. In section 3.2, we proposed that such an
intersection stood for the insertion of a Miura operator $R^{(1)}(x)$ at
position $x\in{\mathbb{R}}_{x}\times S^{1}(R)$, or equivalently an R-matrix
valued in the tensor product of modules ${\mathcal{V}}_{3}\otimes{\cal
F}_{1}^{(1,0)}$, with spectral parameter $x$.

Because this brane configuration preserves a full 3-dimensional
${\mathcal{N}}=4$ supersymmetry, it is somewhat more direct to count BPS
operators by defining a supersymmetric index directly in 3 dimensions on
$\mathbb{R}^{2}_{{\varepsilon}_{3}}\times S^{1}(R)$, instead of via the
quantum mechanics on $S^{1}(R)$. We can think of the NS5/D3 intersection as
two D3-branes, both semi-infinite along ${\mathbb{R}}_{u}$, ending on opposite
sides of the NS5-brane. The quantization of string zero modes at this locus
yields charged matter, a 3d ${\mathcal{N}}=4$ hypermultiplet in the
bifundamental representation $(1,-1)$ of $U(1)\times U(1)$, with both $U(1)$
groups understood as non-dynamical. When the two D3-branes break along the
NS5-brane in the ${\mathbb{R}}_{x}$ direction, the relative distance between
them is proportional to a real equivariant mass for this hypermultiplet.

When the two halves of the D3-brane coincide along ${\mathbb{R}}_{x}$ and
combine into a single brane, the hypermultiplet becomes massless. In that
case, the full D3-brane can instead be pulled away from the NS5-brane in the
${\mathbb{R}}_{z}$ direction without breaking supersymmetry, probing the Higgs
phase.

We explicitly break supersymmetry to $\mathcal{N}=2$ as follows by turning on
a real equivariant mass ${\varepsilon}_{2}$ for the $U(1)_{{\varepsilon}_{2}}$
flavor symmetry rotating $\mathbb{R}^{2}_{{\varepsilon}_{2}}$. Both the 3d
$\mathcal{N}=2$ bifundamental chiral multiplets and the adjoint chiral
multiplet sitting inside the $\mathcal{N}=4$ vector multiplet272727By adjoint
representation, we really mean the charge 0 irrep of $U(1)$ here, since there
is only one D3-brane present. are charged under this symmetry. After turning
on the mass, the resulting supersymmetry is often denoted as 3d
$\mathcal{N}=2^{*}$.

All the above global symmetry parameters are complexified by the D3-brane
$U(1)$ holonomy along the circle $S^{1}(R)$, with notation

$\displaystyle
q_{c}=e^{-R\left({\varepsilon}_{c}+i\,A^{\theta}_{{\varepsilon}_{c}}\right)}\;,\;\;\;\qquad
X_{d}=e^{-R\left(x_{d}+i\,A^{\theta}_{x_{d}}\right)}\;.$ (137)

For a single D3-brane, the Coulomb branch moduli space is that of a free
vector multiplet, with background gauge field $A^{\theta}$. In what follows,
we do not reduce this moduli space by factoring out the center of mass
position of D3-branes.282828Because our 3d theories are all supported on an
$\Omega$-background $S^{1}(R)\times\mathbb{R}^{2}_{{\varepsilon}_{c}}$, it is
customary to treat each $U(1)$ modulus as a physical equivariant Coulomb
parameter.

The 3-dimensional index has been defined and studied both by physicists and
mathematicians. In Physics, the BPS count is the partition function of a 3d
${\cal N}=2$ SCFT on the 3-manifold $S^{1}(R)\times\mathbb{R}^{2}$, which is
well-defined only after regularizing the non-compactness of $\mathbb{R}^{2}$.
One such regularization prescription is to introduce a massive deformation of
the background, denoted as
$M_{3}=S^{1}(R)\times\mathbb{R}^{2}_{{\varepsilon}_{3}}$, understood as a
$\mathbb{R}^{2}$ bundle over $S^{1}(R)$: let $(X_{S^{1}},X_{3})$ be
coordinates on $S^{1}(R)\times\mathbb{R}^{2}$, and identify

$\displaystyle(0,X_{3})\sim(2\pi R,q_{3}\,X_{3})\;.$ (138)

This background $M_{3}$ is known as a Melvin cigar, or 3-dimensional
$\Omega$-background with equivariant parameter
$q_{3}=e^{R\,{\varepsilon}_{3}}$; the fugacity $\log(q_{3})$ is conjugate to
the Cartan generator $J_{3}$ of the $U(1)_{J_{3}}$ action rotating
$\mathbb{R}^{2}_{{\varepsilon}_{3}}$. The BPS states counted by the partition
function are fixed points of this equivariant action, so they are
codimension-2 particles (vortices) localized at the origin of
$\mathbb{R}^{2}_{{\varepsilon}_{3}}$. A partition function on $M_{3}$ is known
as a holomorphic block Beem:2012mb . Because boundary conditions have to be
specified for the fields “at infinity” of
$\mathbb{R}^{2}_{{\varepsilon}_{3}}$, holomorphic blocks are intrinsically
defined in the IR.292929Such boundary conditions label the choice of a 3d
vacuum, and are often called Picard-Lefschetz thimbles in supersymmetric
Physics Cecotti:1991me ; Cecotti:1992rm ; Hori:2000ck ; Gaiotto:2015zna ;
Gaiotto:2015aoa .

In Mathematics, this index is called vertex function, or generating function
of quasimaps $\mathbb{CP}^{1}\rightarrow X$ of all degrees in equivariant
quantum K-theory, where $X$ is the Higgs branch of the 3d theory
2009arXiv0908.4446C ; 2010arXiv1005.4125K ; 2011arXiv1106.3724C ;
Okounkov:2015spn . Quasimaps are solutions to the vortex equations of the 3d
theory, subjected to certain regularity conditions on the gauge and matter
fields, and K-theory enters here instead of cohomology because we are studying
a 3-dimensional theory compactified on a circle instead of a purely
2-dimensional one.

It is also possible to define the index straight from a 3d ${\mathcal{N}}=2$
UV Lagrangian: one simply replaces $\mathbb{R}^{2}_{{\varepsilon}_{3}}$ with a
finite disk/hemisphere $D^{2}$, and imposes a specific set of 1/2-BPS
$\mathcal{N}=(0,2)$ boundary conditions at finite distance on
$T^{2}=S^{1}(R)\times S^{1}_{D^{2}}$, which will flow to a superconformal
point in the IR. One requires the $\mathcal{N}=(0,2)$ boundary theory to have
an unbroken $U(1)_{R}$ R-symmetry, so that an index can be defined in the
first place. Often, this $U(1)_{R}$ R-symmetry is identified with the one from
the 3d $\mathcal{N}=2$ bulk, but it does not need to be so in
general.303030The $U(1)_{R}$ charge is sometimes a linear combination of the
bulk R-symmetry charge and other $U(1)$ charges in the maximal torus of the UV
flavor symmetry group; for instance, this will be the case for the
distinguished Dirichlet boundary conditions of section 4.3.

###  4.2 Index as an integral and Neumann boundary conditions

This UV partition function, sometimes called half-index, is most easily
computed via equivariant localization, as a trace over the Hilbert space of
states on $D^{2}$ Yoshida:2014ssa . Alternatively, the UV half-index has yet
another definition, as a character over the vector space of local operators at
the boundary of 3d $\mathcal{N}=2$ Minkowski spacetime
$\mathbb{R}^{1,1}\times\mathbb{R}_{\leq 0}$, with a
$\mathcal{N}=(0,2)$-preserving boundary supported on
$\mathbb{R}^{1,1}\times\\{0\\}$. These two UV definitions of the half-index
happen to agree up to 1-loop contributions of certain boundary mixed ’t Hooft
anomalies, which will be safely ignored for our purposes Bullimore:2020jdq .

In the Minkowski formulation, one introduces coordinates $(x^{0},x^{1})$ on
the boundary $\mathbb{R}^{1,1}\times\\{0\\}$ and a bulk coordinate
$x^{\bot}\leq 0$ on the half-line $\mathbb{R}_{-}$, with boundary located at
$x^{\bot}=0$. The 3d $\mathcal{N}=2$ algebra has four real supercharges, or
two complex spinors $Q_{\alpha}$ and $\overline{Q}_{\alpha}$, where
$\alpha=+,-$. These supercharges
$(Q_{+},Q_{-},\overline{Q}_{+},\overline{Q}_{-})$ have $U(1)_{R}$ R-charge
$(-1,-1,1,1)$, and satisfy the relations

$\displaystyle\\{Q_{\pm},\overline{Q}_{\pm}\\}=\mp 2\,P_{\pm}\;,$
$\displaystyle\\{Q_{\pm},\overline{Q}_{\mp}\\}=\mp 2\,i\,(P_{\bot}\mp
i\,Z)\;,$ (139)

where $P_{\alpha}$ denotes the momentum on $\mathbb{R}^{1,1}$, and $Z$ is a
real central charge.

It directly follows from (4.2) that a 1/2-BPS $\mathcal{N}=(0,2)$ subalgebra
generated by the supercharges $Q_{+}$ and $\overline{Q}_{+}$ is preserved on
the boundary. Then, the half-index is defined in
$\overline{Q}_{+}$-cohomology, as a character over the vector space of local
operators at $x^{\bot}=0$:

${\mathcal{Z}}_{3d}={\rm
Tr}\left[(-1)^{F}\,{q_{1}}^{J_{1}}\,{q_{2}}^{J_{2}}\,{q_{3}}^{J_{3}}\;X^{\mathfrak{t}_{H}}\;v^{\mathfrak{t}_{C}}\right]\;\;.$
(140)

The fermion number operator is denoted as $F=2J_{3}$. The operator
$\mathfrak{t}_{H}$ stands for the Cartan generator of the flavor symmetries
$F_{H}$ preserved on the boundary; the conjugate fugacities are the
hypermultiplet masses, collectively denoted as $\log(x)$. The operator
$\mathfrak{t}_{C}$ stands for the Cartan generator of an additional global
symmetry $U(1)_{\cal J}$, called topological symmetry. The conjugate fugacity
is the Fayet-Iliopoulos (F.I.) parameter, denoted as $\log(v)$; we will have
more to say about this symmetry momentarily.

From the point of view of the quantum mechanics on $S^{1}(R)$, the fugacities
$\log(q_{c})$ for $c=1,2,3,$ are treated on an equal footing, and are all
conjugate to R-symmetries. From the 3-dimensional viewpoint, the symmetry
between them is broken, according to which
$\mathbb{R}^{2}_{{\varepsilon}_{c}}$ plane the D3-brane is supported on; in
our setup, we have fixed a D30,0,1 brane, so the fugacity $\log(q_{3})$ is a
geometric parameter for the $\Omega$-background, conjugate to the Cartan
generator $J_{3}$ of the $U(1)$ action rotating the boundary plane (or the
disk/hemisphere $D^{2}$ in the Euclidean formulation); only $\log(q_{1})$ and
$\log(q_{2})$ represent R-symmetries. In detail, our D3-NS5 brane
configuration preserves a full 3d $\mathcal{N}=4$ supersymmetry, with
R-symmetry $SU(2)_{H}\times SU(2)_{C}$. We denote the maximal torus as
$U(1)_{H}\times U(1)_{C}$, or as $U(1)_{V}\times U(1)_{A}$ on the
$\mathcal{N}=(2,2)$ boundary. Then, $\log(q_{2})$ is conjugate to the Cartan
generator $J_{2}$ of the $U(1)_{A}$ action on vector multiplet scalars at the
boundary; these scalars are the moduli describing the fluctuation of the
D30,0,1 brane along the NS51,0,0 brane worldvolume on
$\mathbb{R}^{2}_{{\varepsilon}_{2}}$. The fugacity $\log(q_{1})$ is conjugate
to the Cartan generator $J_{1}$ of the $U(1)_{V}$ action on hypermultiplet
scalars at the boundary.

In the above conventions, $J_{1,2,3}$ all have half-integer eigenvalues, so in
particular the index can be expressed as a Taylor series in ${q_{3}}^{1/2}$.

In 3d $\mathcal{N}=2$ notation, we can invoke the constraint
$q_{1}=q_{2}^{-1}\,q_{3}^{-1}$ and change notations to $J_{1}=-V/2$ and
$J_{2}=-A/2$, so that the index takes a more familiar form Gadde:2013wq ;
Gadde:2013sca ; Dimofte:2017tpi :

${\mathcal{Z}}_{3d}={\rm
Tr}\left[(-1)^{F}\,{q_{3}}^{J_{3}+\frac{V}{2}}\;{q_{2}}^{\frac{V-A}{2}}\;X^{\mathfrak{t}_{H}}\;v^{\mathfrak{t}_{C}}\right]\;\;.$
(141)

That is, the generator $J_{1}$ of $U(1)_{V}$ is now identified with the
generator of the surviving $U(1)_{R}$ R-symmetry. Moreover,
$J_{2}-J_{1}=(V-A)/2$ generates the $U(1)_{{\varepsilon}_{2}}$ flavor symmetry
rotating $\mathbb{R}^{2}_{{\varepsilon}_{2}}$. In this convention, $V$ and $A$
both have integer eigenvalues, and $R=V$ is the $U(1)_{R}$ R-symmetry
generator.

####  4.2.1 Index for non-transverse $\text{D3}_{0,0,1}$-$\text{NS5}_{1,0,0}$
intersection

Let $x_{1}$ and $x_{2}$ be the positions of the two semi-infinite D3-branes on
${\mathbb{R}}_{x}$, respectively, on opposite sides of the NS5-brane. The 3d
$\mathcal{N}=4$ hypermultiplet decomposes into a pair of $\mathcal{N}=2$
chiral multiplets $(\Phi^{+},\Phi^{-})$, which contribute

$\displaystyle\frac{\left(\sqrt{q_{3}/q_{2}}\;X_{0}/X_{1}\,;q_{3}\right)_{\infty}}{\left(\sqrt{q_{3}q_{2}}\;X_{0}/X_{1}\,;q_{3}\right)_{\infty}}\;.$
(142)

to the index. The denominator is the contribution of the $\mathcal{N}=2$
bifundamental chiral multiplet $\Phi^{+}$ with Neumann (N) boundary
conditions: boundary scalars (and their $n$-th derivatives) are counted in
$\overline{Q}_{+}$-cohomology with charge

$(\text{bif
chiral},\text{N}):\;\;\;\left(1,-1,n,1,\frac{1}{2}\right)\;\;\text{under}\;\;U(1)_{X_{0}}\times
U(1)_{X_{1}}\times U(1)_{J_{3}}\times U(1)_{R}\times
U(1)_{{\varepsilon}_{2}}.$ (143)

The numerator is the contribution of the $\mathcal{N}=2$ bifundamental chiral
multiplet $\Phi^{-}$ with Dirichlet (D) boundary conditions: boundary fermions
(and their $n$-th derivatives) are counted in $\overline{Q}_{+}$-cohomology
with charge

$(\overline{\text{bif}}\;\text{chiral},\text{D}):\;\;\;\left(1,-1,n+\frac{1}{2},0,\frac{-1}{2}\right)\;\;\text{under}\;\;U(1)_{X_{0}}\times
U(1)_{X_{1}}\times U(1)_{J_{3}}\times U(1)_{R}\times
U(1)_{{\varepsilon}_{2}}.$ (144)

In writing (142), we assumed a certain holomorphic Lagrangian splitting of the
$\mathcal{N}=4$ bifundamental hypermultiplet matter, called a _polarization_ :
in terms of the $\mathcal{N}=2$ chiral multiplets $(\Phi^{+},\Phi^{-})$, a
polarization is a choice of sign $\epsilon=\pm$, where $\epsilon=-$ labels
(D,N) boundary conditions for $(\Phi^{+},\Phi^{-})$, while $\epsilon=+$ labels
(N,D) boundary conditions for $(\Phi^{+},\Phi^{-})$. The index contribution
(142) therefore corresponds the polarization $(+)$.313131It is also possible
to flip boundary conditions to have both chiral multiplets with D, or both
with N, at the expense of shifting the Chern-Simons level to a nonzero value
at 1-loop.

So far, we have only analyzed the index of a bare D3-NS5 brane intersection;
in order to make contact with the integral form of the Miura operator, we now
argue that we need to further gauge the $U(1)$ symmetry on one of the two
half-infinite D3-branes located on either side of the NS5-brane. In string
theory, this is done by simply adding another NS5-brane for that D3 to end on,
separated from the first NS5-brane along the $\mathbb{R}_{t}$-direction. Then,
we consider two NS51,0,0 branes and one semi-infinite D30,0,1 brane, meaning
$(L,M,N)=(2,0,0)$ and $(l,m,n)=(0,0,1)$ in our M-theory notation. It is
convenient to think of this configuration as made up of two parts: a D30,0,1
brane segment, of finite extent between the two NS51,0,0 branes, and a semi-
infinite D30,0,1 brane ending on one of the NS51,0,0 branes; see the top of
figure (14).

In this Hanany-Witten setup Hanany:1996ie , the low energy theory on the
D3-branes is again a 3d $\mathcal{N}=4$ $U(1)$ gauge theory on
$\mathbb{R}^{2}_{{\varepsilon}_{3}}\times S^{1}(R)$, with a bifundamental
hypermultiplet in representation $(1,-1)$ of $U(1)\times U(1)$; this time
around, one of the two $U(1)$’s is gauged, say the first one. Correspondingly,
the motion of the D3-brane segment along ${\mathbb{R}}_{x}$ between the two
NS5-branes is a real Coulomb modulus ${\mathfrak{a}}$ for this $U(1)$ gauge
group. The position of the semi-infinite D3-brane on ${\mathbb{R}}_{x}$ is a
mass parameter, as before.

When the two parts of the D3-brane combine along ${\mathbb{R}}_{x}$ into a
single brane, the hypermultiplet becomes massless. In that case, a real F.I.
term can be turned on by pulling away the middle NS5-brane along the
${\mathbb{R}}_{z}$ direction. We denote the corresponding F.I. parameter as

$\xi=\xi_{1}-\xi_{2}\;,$ (145)

where $\xi_{i}$ denotes the position of the $i$-th NS5-brane along
${\mathbb{R}}_{z}$. In the 3d gauge theory, the F.I. term arises in the action
from coupling the $U(1)$ gauge field on the D3-brane to a global symmetry
group $U(1)_{\cal J}$ usually called topological symmetry. This symmetry
arises from the conserved current ${\cal J}=\frac{1}{2\pi}*F$, where $*F$ is
the Hodge dual of the $U(1)$ field strength. The charge associated to this
topological symmetry is called vortex number $k$, and the equivariant mass is
the F.I. parameter $\xi$. In what follows, we work with $\xi$ positive and
generic. The Coulomb modulus and F.I. parameter are once again complexified by
the holonomy along $S^{1}(R)$, with notation

$\displaystyle
X=e^{-R\left({\mathfrak{a}}+i\,A^{\theta}_{\mathfrak{a}}\right)}\;,\;\;\;\qquad
v=e^{-R\left(\xi+i\,A^{\theta}_{\xi}\right)}\;.$ (146)

The 3d index is evaluated as before, with the caveat that additional boundary
conditions need to be specified for the dynamical vector multiplet. Imposing
Neumann boundary conditions ${\bf N}$, the gauge symmetry is preserved at the
boundary, and the index trace becomes a finite-dimensional contour integral
over the gauge moduli, projecting to gauge-invariant operators. For a $U(1)$
gauge group, there is a single such modulus $X$, which is integrated over,
with contribution

$\frac{\left(q_{3}\,;q_{3}\right)_{\infty}}{\left(q^{-1}_{2}\,;q_{3}\right)_{\infty}}\oint\frac{dX}{2\pi
i\,X}\;.$ (147)

Indeed, the $\mathcal{N}=4$ vector multiplet decomposes as a $\mathcal{N}=2$
vector multiplet and a $\mathcal{N}=2$ adjoint chiral multiplet. The numerator
is the contribution of the vector multiplet with boundary conditions ${\bf
N}$: boundary gauginos (and their $n$-th derivatives) are counted in
$\overline{Q}_{+}$-cohomology with charge

$(\text{vector},{\bf
N}):\;\;\;\left(\text{adj},n+\dfrac{1}{2},1,0\right)\;\;\text{under}\;\;U(1)\times
U(1)_{J_{3}}\times U(1)_{R}\times U(1)_{{\varepsilon}_{2}}\;.$ (148)

The denominator is the contribution of the adjoint chiral multiplet with
boundary conditions N: boundary scalars (and their $n$-th derivatives) are
counted in $\overline{Q}_{+}$-cohomology with charge

$(\text{adj
chiral},\text{N}):\qquad\left(\text{adj},n,0,-1\right)\;\;\text{under}\;\;U(1)\times
U(1)_{J_{3}}\times U(1)_{R}\times U(1)_{{\varepsilon}_{2}}\;,$ (149)

Note that the above charge assignment is consistent with the existence of a
$\mathcal{N}=4$ superpotential: this is the familiar superpotential term
coupling the $\mathcal{N}=2$ bifundamental chiral multiplets (143) and (144)
to the adjoint chiral multiplet (149).

In type IIB, the contribution (147) is obtained from quantizing the zero modes
of strings starting and ending on the D3-brane segment, between the two
NS5-branes. Meanwhile, the contribution (142) is obtained from quantizing the
zero modes of strings starting on the D3-brane segment, and ending on the
semi-infinite part of the D3-brane. Note the fugacity $X_{0}$ there (or $X$
here) is now integrated over.

$v_{2}$$v_{1}$NS5NS5D3$1$$1$$=$$v_{1}$$v_{2}$$v_{2}$$S_{1,1}^{3}$$\Phi^{*}_{3,1}$$v_{2}$$v_{1}$$\Phi^{*}_{3,1}$$R^{(1)}$
Figure 14: Top: Two NS51,0,0 branes and a semi-infinite D30,0,1 brane drawn in
the $(\mathbb{R}_{x},\mathbb{R}_{t})$ plane, and the low energy effective 3d
$\mathcal{N}=4$ gauge theory: a $U(1)$ vector multiplet with one
hypermultiplet. Bottom Left: The 3d half-index is manifestly equal to the vev
of a screening charge composed with a dual intertwiner on the second Fock
space of the quantum toroidal algebra. Bottom Right: Equivalently, the 3d
half-index can be reinterpreted as the vev of a dual intertwiner on the first
Fock space composed with an R-matrix.

There are further contributions of boundary ’t Hooft anomalies at 1-loop.
Crucially, we must ensure there is no gauge-gauge anomaly, since the gauge
symmetry is preserved by the boundary condition ${\bf N}$ on the gauge
multiplet. Here, both the $\mathcal{N}=4$ vector multiplet and the
$\mathcal{N}=4$ hypermultiplet are separately gauge anomaly-free:

  * •

For the $\mathcal{N}=4$ vector multiplet, the anomaly polynomial is

$+\frac{1}{2}f^{2}_{R}-\frac{1}{2}f^{2}_{R}=0\;,$ (150)

where $f_{R}$ denotes the curvature of the $U(1)_{R}$ R-symmetry. The first
term is the contribution of the $\mathcal{N}=2$ gauginos, which cancels the
second term, the contribution of the fermions in the $\mathcal{N}=2$ adjoint
chiral multiplet.

  * •

For the $\mathcal{N}=4$ bifundamental chiral multiplet, the anomaly polynomial
is

$-\frac{1}{2}(f-f_{X_{1}}+f_{{\varepsilon}_{2}})^{2}+\frac{1}{2}(f-f_{X_{1}}-f_{{\varepsilon}_{2}})^{2}=-2\,f\,f_{{\varepsilon}_{2}}+2\,f_{X_{1}}\,f_{{\varepsilon}_{2}}\;,$
(151)

where $f,f_{X_{1}},f_{{\varepsilon}_{2}}$, are the curvatures of the $U(1)$
gauge symmetry, $U(1)_{X_{1}}$ and $U(1)_{{\varepsilon}_{2}}$ flavor
symmetries, respectively. The $\mathcal{N}=2$ chiral multiplet with boundary
condition N contributes the first term on the left-hand side, while the
$\mathcal{N}=2$ chiral multiplet with boundary condition D contributes the
second term. In particular, the gauge anomaly is canceled, but there are
remaining _mixed_ ’t Hooft anomalies, which indicate $U(1)_{X_{1}}$ and
$U(1)_{{\varepsilon}_{2}}$ are both broken at the boundary.323232The analysis
also applies to the non-abelian case: for $K$ D3-brane segments of the same
type instead of just 1, the gauge group will be $U(K)$, in which case the
gauginos contribute additional terms $+K\text{Tr}(f^{2})-(\text{Tr}f)^{2}$ to
the anomaly polynomial, but these are again exactly canceled by the fermions
in the adjoint chiral multiplet with boundary condition N. Similarly, the
gauge anomaly for the hypermultiplets is canceled as in the abelian case.

  * •

There is a mixed topological-gauge anomaly, with polynomial

$-2\,(f-f_{X_{1}})\,f_{v}\;,$ (152)

where $f_{v}$ is the curvature of the $U(1)_{{\cal J}}$ topological symmetry.

Correspondingly, inside the index integrand, the mixed anomalies manifest
themselves as the additional factors

$e^{-\frac{\ln(q_{2})\ln(X/X_{1})}{\ln(q_{3})}}\,e^{-\frac{\ln(v)\ln(X/X_{1})}{\ln(q_{3})}}\;.$
(153)

For our purposes, we only really need to keep track of the dynamical
contributions to the index, which are the ones _inside_ the integrand.
“Constant” $q$-Pochhammer symbols and boundary ’t Hooft anomaly contributions
of global symmetries factor out of the integral, and our index computations
will only be correct up to overall normalization by such factors.

Note that none of our expressions depend on moduli in the $\mathbb{R}_{t}$
direction. This is by design, since the $\mathbb{R}_{t}$ direction is
topological in our background.

All in all, the index comes out to be

${\mathcal{Z}}^{({\bf
N},+)}_{\text{D3}_{0,0,1}-\text{NS5}_{1,0,0}}=\frac{\left(q_{3}\,;q_{3}\right)_{\infty}}{\left(q^{-1}_{2}\,;q_{3}\right)_{\infty}}\oint\frac{dX}{2\pi
i\,X}\,e^{-\frac{\ln(q_{2})\ln(X/X_{1})}{\ln(q_{3})}}\,e^{-\frac{\ln(v)\ln(X/X_{1})}{\ln(q_{3})}}\frac{\left(\sqrt{q_{3}/q_{2}}\;X/X_{1}\,;q_{3}\right)_{\infty}}{\left(\sqrt{q_{3}q_{2}}\;X/X_{1}\,;q_{3}\right)_{\infty}}\;.$
(154)

We recognize the vev of a screening current (120) and a dual intertwiner,

${\mathcal{Z}}^{({\bf
N},+)}_{\text{D3}_{0,0,1}-\text{NS5}_{1,0,0}}=\left\langle\oint\frac{dX}{2\pi
i\,X}\left(1\otimes\Phi_{3,1}^{*}(X_{1})\right)\,{\mathcal{S}}^{3}_{1,1}(X)\,\right\rangle\;,$
(155)

where the bracket notation labels the vacuum

$\left\langle\ldots\right\rangle={[u]^{(3)*}_{0}}\otimes\langle
v_{1}\,q_{2}^{-1},\varnothing|\otimes\langle
v_{2},\varnothing|\;\;\ldots\;\;|v_{1},\varnothing\rangle\otimes|v_{2},\varnothing\rangle\;.$
(156)

In particular, the complexified F.I. parameter is $v=v_{1}/v_{2}$, and appears
in the integrand via the zero modes of the operators acting on the Fock vacua,
see (62). The integrand should be understood as the composition of two
homomorphisms: the screening current

${\mathcal{S}}^{3}_{1,1}\;:\;\;\mathcal{F}_{1}^{\mathbf{(1,0)}}(v_{1})\otimes\mathcal{F}_{1}^{\mathbf{(1,0)}}(v_{2})\rightarrow\mathcal{F}_{1}^{\mathbf{(1,0)}}(v_{1}\,q^{-1}_{2})\otimes\mathcal{F}_{1}^{\mathbf{(1,0)}}(v_{2}\,q_{2})\;,$
(157)

and the dual intertwiner

$\Phi_{3,1}^{*}\;:\;\;\mathcal{F}_{1}^{\mathbf{(1,0)}}(v_{2}\,q_{2})\rightarrow\mathcal{V}_{3}(u)\otimes\mathcal{F}_{1}^{\mathbf{(1,0)}}(v_{2})\;.$
(158)

Above, $\mathcal{F}_{1}^{\mathbf{(1,0)}}(v_{1})$ labels the Fock space of the
first $\text{NS5}_{1,0,0}$ brane (the one the D3-brane ends on), and
$\mathcal{F}_{1}^{\mathbf{(1,0)}}(v_{2})$ labels the Fock space of the second
intersecting $\text{NS5}_{1,0,0}$ brane (the notation was introduced above
equation (62)). Moreover, the state in $\mathcal{V}_{3}(u)$ is annihilated by
the dual state $[u]^{*}_{0}$ from the vacuum: if we expand $\Phi^{*}_{3,1}$ in
terms of basis elements as
$\Phi^{*}_{3,1}=\sum_{i\in\mathbb{Z}}(\Phi^{*}_{3,1})_{i}[u]_{i}^{(3)}$, then
the insertion of the vacuum state ${[u]^{(3)*}_{0}}$ results in
$\sum_{i\in\mathbb{Z}}(\Phi^{*}_{3,1})_{i}\,{[u]^{(3)*}_{0}}\,[u]_{i}^{(3)}=(\Phi^{*}_{3,1})_{0}$,
by orthonormality of the basis.

Equivalently, the half-index is also the vev of an R-matrix, as pictured on
the bottom of figure (14). Namely, the integrand can also be understood as the
composition of two different homomorphisms: the dual intertwiner

$\Phi_{3,1}^{*}\;:\;\;\mathcal{F}_{1}^{\mathbf{(1,0)}}(v_{1})\rightarrow\mathcal{V}_{3}(u)\otimes\mathcal{F}_{1}^{\mathbf{(1,0)}}(v_{1}\,q^{-1}_{2})\;.$
(159)

and the non-transverse R-matrix

$R^{(1)}\;:\;\;\mathcal{V}_{3}(u)\otimes\mathcal{F}_{2}^{\mathbf{(1,0)}}(v_{2})\rightarrow\mathcal{V}_{3}(u)\otimes\mathcal{F}_{2}^{\mathbf{(1,0)}}(v_{2})\;.$
(160)

Explicitly, define an integrand

$R^{(1)}(X_{1})=\oint\frac{dX}{2\pi i\,X}\,{\mathcal{R}}^{(1)}(X,X_{1})\;,$
(161)

for the integral presentation (89) of the R-matrix $R^{(1)}$; then the half-
index is the vev

${\mathcal{Z}}^{({\bf
N},+)}_{\text{D3}_{0,0,1}-\text{NS5}_{1,0,0}}=\left\langle\oint\frac{dX}{2\pi
i\,X}\,{\mathcal{R}}^{(1)}(X,X_{1})\,(\Phi_{3,1}^{*}(X)\otimes
1)\right\rangle\;.$ (162)

####  4.2.2 Index for non-transverse $\text{D3}_{0,0,1}$-$\text{NS5}_{0,1,0}$
intersection

We start again with a semi-infinite D30,0,1 brane ending on an NS51,0,0 brane.
This time, we make the D3-brane intersect a new NS50,1,0 brane, located some
distance away from NS51,0,0 along $\mathbb{R}_{t}$. We would like to
understand the half-index of the resulting configuration.

Quantizing the zero modes of strings at the D30,0,1-NS50,1,0 intersection
yields a 3d $\mathcal{N}=4$ hypermultiplet as before. In the index, this is
the contribution (142), with a swap of fugacities $q_{2}\leftrightarrow
q_{1}$:

$\displaystyle\frac{\left(\sqrt{q_{3}/q_{1}}\;X/X_{1}\,;q_{3}\right)_{\infty}}{\left(\sqrt{q_{3}q_{1}}\;X/X_{1}\,;q_{3}\right)_{\infty}}\;.$
(163)

Quantizing the zero modes of strings starting and ending on the D3-brane
segment, between the NS51,0,0 and NS50,1,0 branes, the supersymmetry is broken
from 8 to 4 supercharges, and the surviving degrees of freedom are those of a
3d $\mathcal{N}=2$ vector multiplet. Indeed, the NS50,1,0 brane is rotated in
the $\mathbb{R}^{2}_{{\varepsilon}_{2}}$ plane compared to the NS51,0,0 brane;
this is a complex mass deformation which explicitly breaks 3d $\mathcal{N}=4$
supersymmetry to $\mathcal{N}=2$ Kitao:1998mf ; Bergman:1999na . The $x^{3}$
and $x^{4}$ moduli for the position of the D3 brane on
$\mathbb{R}^{2}_{{\varepsilon}_{2}}$ are lifted. After the mass deformation,
the $\mathcal{N}=2$ adjoint chiral multiplet inside the $\mathcal{N}=4$ vector
multiplet no longer contributes to the index, and only the $\mathcal{N}=2$
vector multiplet remains.

Imposing boundary conditions ${\bf N}$ on this multiplet, the index
contribution is

$\left(q_{3};q_{3}\right)_{\infty}\oint\frac{dX}{2\pi i\,X}\;.$ (164)

Note the absence of the factor
$\left(q^{-1}_{2}\,;q_{3}\right)^{-1}_{\infty}$, which stood for the
contribution of the adjoint chiral multiplet in (147).

In our abelian setup, the 3d gauge theory has no gauge anomaly, but it has a
$U(1)_{R}$ R-symmetry anomaly: the $\mathcal{N}=2$ gaugino contribution to the
boundary anomaly polynomial is $+\frac{1}{2}f^{2}_{R}$, with $f_{R}$ the
curvature of $U(1)_{R}$.333333For $K$ D3-branes and a $U(K)$ gauge group,
there would now be a gauge anomaly $+K\text{Tr}(f^{2})-(\text{Tr}f)^{2}$ to
cancel: if we do not introduce additional matter (as new 3d $\mathcal{N}=2$
chiral multiplets and 2d $\mathcal{N}=(0,2)$ chiral multiplets at the
boundary), one possibility is to turn on bare Chern-Simons levels in the bulk.
In fact, for a $U(K)$ gauge group, we are allowed to turn on two levels; for
the $SU(K)$ part of the gauge group, a Chern-Simons level $k_{1}$ contributes
$k_{1}\text{Tr}(f^{2})$ to the anomaly polynomial. For the $U(1)$ part, an
effective Chern-Simons level $k_{1}+k_{2}$ contributes
$\frac{k_{2}}{K}(\text{Tr}f)^{2}$ to the anomaly polynomial. We would
therefore choose $k_{1}=-K$ and $k_{2}=K$. We will ignore this contribution as
it factors out of the index integral. The remaining mixed anomalies are as
before, again after swapping $q_{2}\leftrightarrow q_{1}$. It follows that the
index evaluates to

${\mathcal{Z}}^{({\bf
N},+)}_{\text{D3}_{0,0,1}\text{-}\text{NS5}_{0,1,0}}=\left(q_{3}\,;q_{3}\right)_{\infty}\oint\frac{dX}{2\pi
i\,X}\,e^{-\frac{\ln(q_{1})\ln(X/X_{1})}{\ln(q_{3})}}\,e^{-\frac{\ln(v)\ln(X/X_{1})}{\ln(q_{3})}}\frac{\left(\sqrt{q_{3}/q_{1}}\;X/X_{1}\,;q_{3}\right)_{\infty}}{\left(\sqrt{q_{3}q_{1}}\;X/X_{1}\,;q_{3}\right)_{\infty}}\;.$
(165)

This is the vev of a different screening current (124) and a dual intertwiner,

${\mathcal{Z}}^{({\bf
N},+)}_{\text{D3}_{0,0,1}\text{-}\text{NS5}_{0,1,0}}=\left\langle\oint\frac{dX}{2\pi
i\,X}\left(1\otimes\Phi_{3,1}^{*}(X_{1})\right)\,{\mathcal{S}}^{3}_{1,2}(X)\,\right\rangle\;,$
(166)

where the bracket notation labels the vacuum

$\left\langle\ldots\right\rangle={[u]^{(3)*}_{0}}\otimes\langle
v_{1}\,q_{2}^{-1},\varnothing|\otimes\langle
v_{2},\varnothing|\;\;\ldots\;\;|v_{1},\varnothing\rangle\otimes|v_{2},\varnothing\rangle\;.$
(167)

Equivalently, this is the vev of an R-matrix: if we define

$R^{(2)}(X_{1})=\oint\frac{dX}{2\pi i\,X}\,{\mathcal{R}}^{(2)}(X,X_{1})\;,$
(168)

where $R^{(2)}$ is the R-matrix integral (89), we obtain

${\mathcal{Z}}^{({\bf
N},+)}_{\text{D3}_{0,0,1}-\text{NS5}_{0,1,0}}=\left\langle\oint\frac{dX}{2\pi
i\,X}\,{\mathcal{R}}^{(2)}(X,X_{1})\,(\Phi_{3,2}^{*}(X)\otimes
1)\right\rangle\;.$ (169)

####  4.2.3 Index for transverse $\text{D3}_{0,0,1}$-$\text{NS5}_{0,0,1}$
intersection

The index is further able to capture the contribution of open strings at a
fully transverse D3-NS5-brane intersection. For concreteness, we consider a
D30,0,1 brane and a NS50,0,1 brane; this configuration is the Type IIB frame
dual to the M-theory setup of section 3.1. As before, it will be convenient to
make the D30,0,1 end on one end on a NS51,0,0 brane, separated from the
NS50,0,1 brane in the $\mathbb{R}_{t}$-direction.

When $q_{c}=1$ for $c=1,2,3$, the brane configuration only preserves a 1d
$\mathcal{N}=4$ quantum mechanics on the circle $S^{1}(R)$, by which we mean a
reduction of 2d $\mathcal{N}=(0,4)$ supersymmetry to one dimension. The
supersymmetry is not broken further by turning on the $\Omega$-background,
$q_{c}\neq 1$.343434Explicit expressions for the preserved supercharges can be
found in Hwang:2014uwa . A detailed string theory derivation of the preserved
supersymmetry is beyond the scope of this paper, but can be derived following
the analysis of NaveenNikita .

The BPS states counted by the index are those of this $\mathcal{N}=4$ quantum
mechanics. If we view the expression (140) as the Witten index of this quantum
mechanics, it is straightforward to count the degrees of freedom directly from
the relevant 1-dimensional BPS multiplets. In fact, we could have proceeded
that way already for the previous 3d $\mathcal{N}=4$ brane intersections,
after decomposing the 3d multiplets in terms of 1d multiplets, and identifying
the 3d supercharges $Q_{+}$ and $\overline{Q}_{+}$ as the ones contributing to
the quantum mechanical index. In the Type IIB picture, this is done by
explicitly introducing dynamical D1-branes, and quantizing the zero modes of
strings stretching between them and the remaining heavier D-branes.

Instead, we chose to interpret (140) from a 3-dimensional point of
view.353535For further details on the equivalence between the indices of the
3d theory and its vortex quantum mechanics, see for instance Hwang:2017kmk ;
Haouzi:2019jzk ; Haouzi:2020bso ; Crew:2020jyf . In this spirit, we will count
the 1d $\mathcal{N}=4$ multiplets of the D3-NS5 intersection as arising from a
coupled 3d/1d half-index, by which we mean the half-index of a 3d bulk theory
supported on $\mathbb{R}^{2}_{{\varepsilon}_{3}}\times S^{1}(R)$, coupled to a
codimension-2 line defect along $S^{1}(R)$.

From this perspective, the D30,0,1-NS51,0,0 and D30,0,1-NS50,1,0 intersections
from section 4.2.1 and 4.2.2 are both captured by a 3d/1d index whose sole
contributions come from the 3d bulk, while the defect is trivial and
contributes the identity. By contrast, the present D30,0,1-NS50,0,1
intersection will be captured instead by a 3d/1d index whose 1d contributions
are nontrivial, while the 3d bulk ones are essentially trivial (up to charge 0
$U(1)$ contributions and boundary mixed ’t Hooft anomalies).

We denote by $X$ and $X_{1}$ the (complexified) positions of the D30,0,1 brane
and the NS50,0,1 brane along $\mathbb{R}_{x}$, respectively. These are both
flavor symmetry masses from the point of view of the 1d quantum mechanics. We
couple this 1d theory to the bulk by gauging these flavor symmetries with 3d
background vector multiplets. Correspondingly, in the 3d/1d index, the 1d
masses are turned into the scalars of the corresponding 3d vector multiplets.
In what follows, we choose the scalar $X$ to be a modulus, meaning the
associated vector multiplet is dynamical. Meanwhile, we choose the scalar
$X_{1}$ to remain fixed as a mass even in 3d, meaning the associated vector
multiplet is non-dynamical.

The fastest way to identify the contribution of the D30,0,1-NS50,0,1 strings
is to note that because the intersection is fully transverse, the modulus $X$
should really be _frozen_ to the locus $X=X_{1}$. If we choose Neumann
boundary conditions on the 3d $U(1)$ gauge field, this is easily achieved by
considering the integral

$\oint{dX}\;\frac{1}{\left(1-X_{1}/X\right)}\,\;\ldots$ (170)

That is, we force $X=X_{1}$ via the contribution of a 1d $\mathcal{N}=2$
chiral multiplet. Because the preserved supersymmetry is $\mathcal{N}=4$, this
cannot be the full story: the 1d $\mathcal{N}=2$ chiral multiplet must pair up
with another 1d $\mathcal{N}=2$ chiral multiplet to restore $\mathcal{N}=4$
supersymmetry.

With the support of the D30,0,1 brane along
$\mathbb{R}^{2}_{{\varepsilon}_{3}}$, this second chiral multiplet will have
$U(1)_{J_{3}}$ charge -1 (for a derivation, see for instance Hwang:2014uwa ).
Then, after including the contribution of the bulk mixed topological-gauge
anomaly, we obtain the index of the D30,0,1-NS50,0,1 intersection:

${\mathcal{Z}}^{({\bf
N})}_{\text{D3}_{0,0,1}-\text{NS5}_{0,0,1}}=\oint{dX}\,e^{-\frac{\ln(v)\ln(X/X_{1})}{\ln(q_{3})}}\frac{1}{\left(1-X_{1}/X\right)\left(1-q^{-1}_{3}\,X_{1}/X\right)}\;.$
(171)

Correspondingly, the contour is chosen to enclose not only the simple pole at
$X=X_{1}$, as we argued classically, but also the second simple pole at
$X=X_{1}\,q^{-1}_{3}$, understood as a quantum contribution. For another
derivation of the contribution (171), see the work Nieri:2017ntx .

Up to overall normalization, we recognize the vev of the totally transverse
R-matrix: if we define ${\mathcal{R}}^{(3)}(X,X_{1})$ as the integrand,

$R^{(3)}(X_{1})=\oint\frac{dX}{2\pi i\,X}\,{\mathcal{R}}^{(3)}(X,X_{1})\;,$
(172)

of the integral expression (94) for the R-matrix $R^{(3)}$, we obtain363636We
stress that $\Phi_{3,3}^{*}(X)$ is _not_ an intertwiner, as explained in
section 3.2.2.

${\mathcal{Z}}^{({\bf
N})}_{\text{D3}_{0,0,1}\text{-}\text{NS5}_{0,0,1}}=\left\langle\oint\frac{dX}{2\pi
i\,X}\,{\mathcal{R}}^{(3)}(X,X_{1})\,(\Phi_{3,3}^{*}(X)\otimes
1)\right\rangle\;.$ (173)

The bracket notation labels the vacuum

$\left\langle\ldots\right\rangle=[u]_{0}^{(3)*}\otimes\langle
v_{1}\,q_{2}^{-1},\varnothing|\otimes\langle
v_{2},\varnothing|\;\;\ldots\;\;|v_{1},\varnothing\rangle\otimes|v_{2},\varnothing\rangle\;.$
(174)

##### Remark

In an S-dual frame, our D30,0,1-NS50,0,1 configuration turns into a
D30,0,1-D50,0,1 configuration, with 8 Dirichlet-Neumann directions. This is
T-dual to the well-studied D0-D8 system with (0,8) supersymmetry, where the
massless open string modes are known to be chiral fermions Banks:1997zs ;
Bachas:1997kn ; Hung:2006nn ; Dijkgraaf:2007sw ; Tong:2014yna ; NaveenNikita .
The index for this intersection would therefore count degrees of freedom found
in $\mathcal{N}=2$ Fermi multiplets, instead of our chiral multiplets. It
would be interesting to generalize this observation to more general brane
configurations, and systematically analyze the behavior of the 3d/1d half-
index under S-duality, in the spirit of Okazaki:2019bok .

###  4.3 Index as a sum and Dirichlet boundary conditions

In the above, we imposed Neumann boundary conditions ${\bf N}$ on the 3d
$\mathcal{N}=2$ gauge multiplet, and showed that the resulting half-index was
a vev of the Miura R-matrix (or a product of such R-matrices in general),
presented as a contour integral. In the present section, we will instead
impose Dirichlet boundary conditions ${\bf D}$ on the gauge multiplet, and
show that the half-index is again the same vev, but presented as a sum over
monopole charges.

First, recall that in the Neumann case, the gauge symmetry was unbroken at the
boundary, which is why the half-index took the form of an integral, projecting
to gauge-invariant operators on the boundary of the 3-manifold. In contrast,
the Dirichlet boundary condition sets $A_{\mu|\partial}=0$ for the gauge
field, which implies the local gauge symmetry on the boundary is broken to a
global symmetry $G_{|\partial}=U(1)_{\partial}$. We introduce a fugacity
$u\in\mathbb{C}^{\times}$ and a generator $\mathfrak{t}_{\partial}$ for this
$U(1)_{\partial}$ symmetry. Explicitly, the index now reads

${\mathcal{Z}}^{({\bf D})}={\rm
Tr}\left[(-1)^{F}\,{q_{3}}^{J_{3}+\frac{V}{2}}\;{q_{2}}^{\frac{V-A}{2}}\;u^{\mathfrak{t}_{\partial}}\;X^{\mathfrak{t}_{H}}\;v^{\mathfrak{t}_{C}}\right]\;\;.$
(175)

In fact, we will seek to break $U(1)_{\partial}$ entirely, as follows: we will
generate a flow along the boundary to a new boundary condition by giving one
of the 3d chiral fields a vev $\mathfrak{c}\neq 0$. At the level of the index,
the weight of the corresponding field must be set to 1 to preserve
supersymmetry. In the terminology of Dimofte:2017tpi , we say that the chiral
field is given deformed Dirichlet boundary condition
$\text{D}_{\mathfrak{c}}$.

To see how this works in practice, consider again the non-transverse brane
configuration of section 4.2.1, with two NS51,0,0 branes of the same type and
a semi-infinite D30,0,1 brane ending on one of them while intersecting the
other. Recall that the supersymmetry preserved by this configuration is 3d
$\mathcal{N}=4$, and that the low energy effective theory is a $U(1)$ gauge
theory with a single hypermultiplet. Imposing the Dirichlet boundary
conditions on the $\mathcal{N}=4$ vector multiplet, the index contribution is

$\frac{\left(q_{3}\,q_{2}\,;q_{3}\right)_{\infty}}{\left(q_{3}\,;q_{3}\right)_{\infty}}\;.$
(176)

The denominator is the contribution of the $\mathcal{N}=2$ vector multiplet
with boundary conditions ${\bf D}$. The gauge symmetry is broken along the
boundary, but there is a residual gauge symmetry orthogonal to it, and the
gauge-invariant operators there (and their $n$-th derivatives) generate the
$\overline{Q}_{+}$-cohomology.373737In detail, the 3d gauge-invariant linear
multiplet decomposes on the boundary as a $(0,2)$ gauge multiplet, whose top
component is the gaugino, and a $(0,2)$ chiral multiplet, whose top component
is $\sigma+i\,A_{\perp}$, where $\sigma$ is a real scalar and $A_{\perp}$ is
the gauge field component perpendicular to the boundary. The index with
boundary condition ${\bf N}$ counts the contributions of the gauge-invariant
operators built out of the gaugino, while the index with boundary condition
${\bf D}$ counts the contributions of gauge-invariant operators built out of
$\sigma+i\,A_{\perp}$. They contribute to the index with charge

$(\text{vector},{\bf
D}):\;\;\;\left(\text{adj},n+1,0,0\right)\;\;\text{under}\;\;U(1)_{\partial}\times
U(1)_{J_{3}}\times U(1)_{R}\times U(1)_{{\varepsilon}_{2}}\;.$ (177)

The numerator is the contribution of the $\mathcal{N}=2$ adjoint chiral
multiplet with boundary conditions D: the gauginos (and their $n$-th
derivatives) are counted in $\overline{Q}_{+}$-cohomology with charge

$(\text{adj chiral},{\bf
D}):\;\;\;\left(\text{adj},n+\dfrac{1}{2},1,1\right)\;\;\text{under}\;\;U(1)_{\partial}\times
U(1)_{J_{3}}\times U(1)_{R}\times U(1)_{{\varepsilon}_{2}}\;.$ (178)

The $\mathcal{N}=4$ hypermultiplet decomposes into a pair of $\mathcal{N}=2$
chiral multiplets $(\Phi^{+},\Phi^{-})$. Here, we decide to impose Dirichlet
boundary conditions for both $\Phi^{+}$ and $\Phi^{-}$, with index
contribution

$\displaystyle\left(\sqrt{q_{3}/q_{2}}\;X_{1}/u\,;q_{3}\right)_{\infty}\,\left(\sqrt{q_{3}/q_{2}}\;u/X_{1}\,;q_{3}\right)_{\infty}\;.$
(179)

We will justify this choice below. The first factor is the contribution of
$\Phi^{+}$ with boundary conditions D: boundary fermions (and their $n$-th
derivatives) are counted in $\overline{Q}_{+}$-cohomology with charge

$(\text{bif
chiral},\text{D}):\;\;\;\left(-1,1,n+\frac{1}{2},0,\frac{-1}{2}\right)\;\;\text{under}\;\;U(1)_{\partial}\times
U(1)_{X_{1}}\times U(1)_{J_{3}}\times U(1)_{R}\times
U(1)_{{\varepsilon}_{2}}.$ (180)

The second factor is the contribution of $\Phi^{-}$ with boundary conditions
D: boundary fermions (and their $n$-th derivatives) are counted in
$\overline{Q}_{+}$-cohomology with charge

$(\overline{\text{bif}}\;\text{chiral},\text{D}):\;\;\;\left(1,-1,n+\frac{1}{2},0,\frac{-1}{2}\right)\;\;\text{under}\;\;U(1)_{\partial}\times
U(1)_{X_{1}}\times U(1)_{J_{3}}\times U(1)_{R}\times
U(1)_{{\varepsilon}_{2}}.$ (181)

Crucially, the contribution (179) receives non-perturbative corrections at the
boundary coming from BPS monopole operators. Given a 3d $\mathcal{N}=2$
theory, a monopole operator in the bulk is a disorder operator whose profile
is a singular solution to

$F=\star D\sigma\,,\qquad\;D\star\sigma=0\;,$ (182)

where $\sigma$ is the real scalar of footnote 37, and $F$ is the field
strength of the gauge field. A monopole operator at the origin will be
surrounded by a hemisphere $D^{2}$, with boundary
$\partial(D^{2})=S^{1}_{D^{2}}$. Because the scalar $\sigma$ and field
strength component $F_{z\bot}$ and are left unconstrained at the boundary ($z$
is a holomorphic coordinate on $D^{2}$), this monopole is compatible with the
Dirichlet boundary condition ${\bf D}$, which we impose to trivialize the
$U(1)$-bundle at the boundary $S^{1}_{D^{2}}$. After this trivialization, the
Dirac quantization condition takes the form:

$\frac{1}{2\pi}\oint_{D^{2}}F+\frac{1}{2\pi}\oint_{S^{1}_{D^{2}}}A=m\in\mathbb{Z}\;.$
(183)

But the boundary condition $A_{\mu|\partial}=0$ implies
$\oint_{S^{1}_{D^{2}}}A=0$, so the monopole charge $m$ is nothing but the
quantized flux through the hemisphere:383838More generally, for a non-abelian
gauge group $G$, we embed the Dirac monopole solution into $G$, meaning $m$ is
an element of the cocharacter lattice
$m\in\Lambda_{cochar}=\text{Hom}(U(1),T_{G})$, the space of maps to the
maximal torus of $G$. In our abelian setup, $\text{Hom}(U(1),U(1))=\mathbb{Z}$
and $m$ is just an integer.

$\frac{1}{2\pi}\oint_{D^{2}}F=m\in\mathbb{Z}\;.$ (184)

In the end, the half-index takes the form of a sum over all magnetic fluxes
$m$ through the hemisphere $D^{2}$. Moreover, all electrically charged states
acquire spin, meaning the fugacity $u$ for the boundary symmetry
$U(1)_{\partial}$ is shifted everywhere as $u\rightarrow q_{3}^{m}\,u$
Dimofte:2017tpi :

${\mathcal{Z}}^{({\bf
D},\text{D},\text{D})}_{\text{D3}_{0,0,1}-\text{NS5}_{1,0,0}}=\frac{\left(q_{3}\,q_{2}\,;q_{3}\right)_{\infty}}{\left(q_{3}\,;q_{3}\right)_{\infty}}\sum_{m\in\mathbb{Z}}\;\left(q_{3}^{-m}\sqrt{q_{3}/q_{2}}\;X_{1}/u\,;q_{3}\right)_{\infty}\,\left(q_{3}^{m}\sqrt{q_{3}/q_{2}}\;u/X_{1}\,;q_{3}\right)_{\infty}\;\ldots\;.$
(185)

The $``\ldots"$ stands for the 1-loop contributions of boundary ’t Hooft
anomalies, which we will come back to shortly.

In order to implement the deformed Dirichlet boundary condition
$\text{D}_{\mathfrak{c}}$, we generate an RG flow along the boundary to a new
boundary condition, where a chiral field acquires a vev $\mathfrak{c}\neq 0$
in the supersymmetric vacuum. If this chiral field has charge $(1,-1,1)$ under
$U(1)_{V}\times U(1)_{\partial}\times U(1)_{X_{1}}$, then the linear
combinations $V+\mathfrak{t}_{\partial}\equiv V^{\prime}$ and
$\mathfrak{t}_{X_{1}}+\,\mathfrak{t}_{\partial}\equiv\mathfrak{t}^{\prime}_{X_{1}}$
are preserved along this flow; the generators of the new vector R-symmetry and
flavor symmetry on the boundary are denoted by $V^{\prime}$ and
$\mathfrak{t}^{\prime}_{X_{1}}$ in the index (175), respectively. Concretely,
the weight of the corresponding chiral field is set to 1, which translates to
a constraint $u^{-1}\,q_{3}^{1/2}\,q_{2}^{1/2}\,X_{1}\,=1$ on $u$. Put
differently, we simply substitute $u=q_{3}^{1/2}\,q_{2}^{1/2}\,X_{1}$ in the
entire summand (185). We obtain

${\mathcal{Z}}^{({\bf
D},\text{D},\text{D}_{\mathfrak{c}})}_{\text{D3}_{0,0,1}-\text{NS5}_{1,0,0}}=\frac{\left(q_{3}\,q_{2}\,;q_{3}\right)_{\infty}}{\left(q_{3}\,;q_{3}\right)_{\infty}}\sum^{\infty}_{m=0}\;\left(q_{3}^{-m}q^{-1}_{2}\;\,;q_{3}\right)_{\infty}\,\left(q_{3}^{1+m}\,;q_{3}\right)_{\infty}\;\ldots\;,$
(186)

Note the summation index $m$ only runs over the positive integers now, because
of the factor $\left(q_{3}^{1+m}\,;q_{3}\right)_{\infty}$.

##### Remark

An alternative way to motivate the deformed Dirichlet boundary condition is as
follows: consider again the hypermultiplet contribution to the index (179),
where we rescale $u$ by the fugacity $q^{1/2}_{3}$ out of convenience:

$\left(q_{3}^{-m}\sqrt{1/q_{2}}\;X_{1}/u\,;q_{3}\right)_{\infty}\,\left(q_{3}^{1+m}\sqrt{1/q_{2}}\;u/X_{1}\,;q_{3}\right)_{\infty}\;.$
(187)

In the $U(1)$ 3d gauge theory, the flavor symmetries acts by Hamiltonian
isometries on the complex scalars of the chiral multiplets
$(\Phi^{+},\Phi^{-})$ as

$U(1)_{X_{1}}:\;(\phi^{+},\phi^{-})\;\rightarrow(e^{+i\theta}\phi^{+},e^{-i\theta}\phi^{-})\;,\qquad
U(1)_{{\varepsilon}_{2}}:\;(\phi^{+},\phi^{-})\;\rightarrow(e^{+i\theta/2}\phi^{+},e^{+i\theta/2}\phi^{-})\;,$
(188)

and the gauge symmetry acts as

$U(1):\;(\phi^{+},\phi^{-})\;\rightarrow(e^{-i\theta}\phi^{+},e^{+i\theta}\phi^{-})\;.$
(189)

Correspondingly, the complex and real superpotentials are

$\displaystyle\widetilde{W}=\varphi\,\phi^{+}\,\phi^{-}$ $\displaystyle
W=\left(-\sigma+x_{1}+\frac{{\varepsilon}_{2}}{2}\right)|\phi^{+}|^{2}+\left(\sigma-
x_{1}+\frac{{\varepsilon}_{2}}{2}\right)|\phi^{-}|^{2}+\xi\,\sigma\;,$ (190)

where $\sigma$ and $\phi$ are the real and complex scalars of the
$\mathcal{N}=4$ vector multiplet, while $\xi$ and $m$ are the real F.I.
parameter and hypermultiplet mass, respectively. The (classical) vacua are
critical points of these superpotentials, which are solutions of the moment
map equations

$\displaystyle\left(|\phi^{+}|^{2}-|\phi^{-}|^{2}\right)$
$\displaystyle=\xi\;,$ $\displaystyle\phi^{+}\,\phi^{-}$ $\displaystyle=0\;,$
(191) $\displaystyle(-\sigma+x_{1}+\frac{{\varepsilon}_{2}}{2})\,\phi^{+}$
$\displaystyle=0\;,$ $\displaystyle\varphi\,\phi^{+}$ $\displaystyle=0\;,$
(192) $\displaystyle(+\sigma-x_{1}+\frac{{\varepsilon}_{2}}{2})\,\phi^{-}$
$\displaystyle=0\;,$ $\displaystyle\varphi\,\phi^{-}$ $\displaystyle=0\;.$
(193)

We are interested in the Higgs vacuum where either $\phi^{+}$ or $\phi^{-}$
acquires a nonzero vev. When $\xi>0$, the vev is $|\phi^{+}|^{2}=\xi$ (with
$\phi^{-}=0=\varphi$), and $-\sigma+x_{1}+\frac{{\varepsilon}_{2}}{2}=0$. When
$\xi<0$, the vev is $|\phi^{-}|^{2}=-\xi$, with $\sigma-
x_{1}+\frac{{\varepsilon}_{2}}{2}=0$. We choose to work with $\xi>0$
throughout393939With this convention, the index has a positive radius of
convergence as a power series in the complexified F.I. parameter
$v=e^{-R\left(\xi+i\,A^{\theta}_{\xi}\right)}$ around 0., and therefore set
$\sigma=x_{1}+\frac{{\varepsilon}_{2}}{2}$.

Because the 3d theory is compactified on $S^{1}(R)$, we likewise impose
$A^{\theta}_{\sigma}=A^{\theta}_{x_{1}}+\frac{A^{\theta}_{{\varepsilon}_{2}}}{2}$
for the holonomies. In the end, substituting $u=q^{1/2}_{2}\,X_{1}$ for the
complexified variables in (187), we recover the summand of the index (194).

Imposing Dirichlet boundary conditions on both $\mathcal{N}=2$ chiral
multiplets of the hypermultiplet has introduced a boundary anomaly at 1-loop,
meaning there is now an _effective_ Chern-Simons coupling at level 1.
Correspondingly, a bare monopole operator of charge $m$ will induce a
nontrivial electric charge $m$ and spin $\frac{1}{2}m^{2}$ in the theory,
meaning an insertion $u^{m}\,q^{\frac{m^{2}}{2}}_{3}$ inside the summand of
the index. After performing the deformed Dirichlet substitution
$u=q_{3}^{1/2}\,q_{2}^{1/2}\,X_{1}$, and redefining the topological fugacity
$v\rightarrow-v\,X^{-1}_{1}$, we find:

${\mathcal{Z}}^{({\bf
D},\text{D},\text{D}_{\mathfrak{c}})}_{\text{D3}_{0,0,1}-\text{NS5}_{1,0,0}}=\frac{\left(q_{3}\,q_{2}\,;q_{3}\right)_{\infty}}{\left(q_{3}\,;q_{3}\right)_{\infty}}\sum^{\infty}_{m=0}(-1)^{m}\,q^{m}_{2}\,q^{\frac{m(m+1)}{2}}_{3}\;v^{m}\;\left(q_{3}^{-m}q^{-1}_{2}\;\,;q_{3}\right)_{\infty}\,\left(q_{3}^{1+m}\,;q_{3}\right)_{\infty}\;.$
(194)

Up to normalization by perturbative contributions outside the summand, we
recognize the vev of the R-matrix (88) (for $c=1$), composed with a dual
intertwiner:

${\mathcal{Z}}^{({\bf
D},\text{D},\text{D}_{\mathfrak{c}})}_{\text{D3}_{0,0,1}-\text{NS5}_{1,0,0}}=\left\langle{R}^{(1)}\,(\Phi_{3,1}^{*}\otimes
1)\right\rangle\;.$ (195)

Above,
$\Phi^{*}_{3,1}=\sum_{i\in\mathbb{Z}}\Phi^{*}_{3,1}(u\,q^{i}_{3})\,[u]_{i}^{(3)}$
is understood as an expansion in the basis elements $[u]_{i}^{(3)}$. The
bracket once again denotes the vacuum (156), so the sum over $i\in\mathbb{Z}$
reduces to a single term, by orthonormality of the basis.

In fact, up to the perturbative normalization, the above Dirichlet index
coincides with the residue sum of the Neumann index (155), where the contour
encloses “$\infty$” and the infinite number of poles at

$X=q^{-m}_{3}\,X_{1}\;,\qquad\;\;\;m=0,1,2,\ldots$ (196)

For instance, the residues of the mixed topological-gauge anomaly contribution
are

$e^{-\frac{\ln(v)\ln(q^{-m}_{3}\,X_{1}/X_{1})}{\ln(q_{3})}}=v^{m}\;,$ (197)

which we recognize as the correct factor in the summand of the Dirichlet
index. See also section 3.4.2 of Dimofte:2017tpi .

###  4.4 Arbitrary D3-NS5 intersections

In summary, for all configurations considered so far, a D30,0,1 brane ends on
a NS51,0,0 brane on one end, and intersects another NS5-brane in a
supersymmetric way:

  * •

A non-transverse D30,0,1-NS51,0,0 intersection, preserving 3d $\mathcal{N}=4$
supersymmetry.

  * •

A non-transverse D30,0,1-NS50,1,0 intersection, preserving 3d $\mathcal{N}=2$
supersymmetry.

  * •

A transverse D30,0,1-NS50,0,1 intersection, preserving 1d $\mathcal{N}=4$
supersymmetry.

These building blocks can be combined to study more general configurations of
D3- and NS5-branes. Namely, consider again a semi-infinite D30,0,1 brane
ending on a NS51,0,0 brane on one end, and intersecting $L$ NS51,0,0, $M$
NS50,1,0, and $N$ NS50,0,1 branes on the other end. The low energy effective
theory is now an abelian quiver gauge theory Hanany:1996ie , of 3d gauge group
$U(1)^{L+M+N}$, with $L+M$ 3d hypermultiplets and $N$ 1d hypermultiplets
(coupled via line defects).

It follows that if we give boundary conditions ${\bf N}$ for the vector
multiplets, and polarization “$+$” to all $L+M$ hypermultiplets, the 3d/1d
half index of this configuration is the integral

$\displaystyle\begin{split}&{\mathcal{Z}}^{({\bf
N},+,\ldots,+)}_{\text{D3}_{0,0,1},\text{NS5}_{L,M,N}}=\oint\prod_{I=1}^{L+M+N}\frac{dX^{(I)}}{2\pi
iX^{(I)}}e^{-\sum_{I\in c^{-1}\\{1,2\\}}\frac{\log q_{\bar{c}_{I}}}{\log
q_{3}}\log\frac{X^{(I)}}{X^{(I+1)}}}\;e^{-\sum_{I=1}^{L+M+N}\frac{\log
X^{(I)}}{\log q_{3}}\log\frac{v_{I}}{v_{I+1}}}\\\
&\qquad\qquad\qquad\qquad\times\prod_{I\in
c^{-1}\\{1,2\\}}\frac{\left(q_{\bar{c}_{I}}^{-1}\frac{X^{(I)}}{X^{(I+1)}};q_{3}\right)_{\infty}}{\left(\frac{X^{(I)}}{X^{(I+1)}};q_{3}\right)_{\infty}}\prod_{I\in
c^{-1}(3)}\frac{1}{\left(1-\frac{X^{(I+1)}}{X^{(I)}}\right)\left(1-\frac{q_{3}^{-1}X^{(I+1)}}{X^{(I)}}\right)}\;,\end{split}$
(198)

up to overall normalization by perturbative contributions outside the
integral. We used the notation
$(|c^{-1}(1)|,|c^{-1}(2)|,|c^{-1}(3)|)=(L,M,N)$, and declare
$X^{(L+M+N+1)}\equiv X_{1}$, as well as $v_{L+M+N+1}=v_{X_{1}}$.

We recognize the vev of a product of R-matrices (111), or equivalently the vev
of the Miura transformation:

${\mathcal{Z}}^{({\bf
N},+,\ldots,+)}_{\text{D3}_{0,0,1},\text{NS5}_{L,M,N}}=\left\langle\oint\prod_{I=1}^{L+M+N}\frac{dX^{(I)}}{2\pi
iX^{(I)}}\,{\mathcal{R}}^{(c_{L+M+N})}_{L+M+N}\ldots\,{\mathcal{R}}^{(c_{2})}_{2}\,{\mathcal{R}}^{(c_{1})}_{1}\,(\Phi_{3,1}^{*}(X^{(0)})\otimes
1)\right\rangle\;.$ (199)

In the bracket $\left\langle\ldots\right\rangle$ for the vev, the in-state on
the right stands for

$|v_{0},\varnothing\rangle\otimes|v_{1},\varnothing\rangle\otimes|v_{2},\varnothing\rangle\otimes\ldots\otimes|v_{L+M+N},\varnothing\rangle\;.$
(200)

and the out-state on the left stands for

${[u]^{(3)*}_{0}}\otimes\langle v_{0}\,q_{2}^{-1},\varnothing|\otimes\langle
v_{1},\varnothing|\otimes\langle v_{2},\varnothing|\otimes\ldots\otimes\langle
v_{L+M+N},\varnothing|\;.$ (201)

To close this section, recall that the brane configurations we studied all had
in common a semi-infinite D30,0,1 brane ending on a NS51,0,0 brane. Choosing
different orientations for this D3-brane and NS5-brane is easily achieved by
simply permuting the parameters $q_{1}$, $q_{2}$ and $q_{3}$ in the index we
have already computed. We can also relax the semi-infinite requirement for the
D3-brane: making the brane infinite simply increases the number of
intersections with NS5-branes by 1, which results in one additional
hypermultiplet in the gauge theory, and one additional R-matrix insertion in
the index. In that way, any arbitrary configuration of D3-branes intersecting
NS5-branes in $\Omega$-deformed Type IIB can be studied via our R-matrix
formalism.

##  5 $qq$-characters from M2-branes on web of M5-branes

So far, we have restricted our discussion to the M5-branes with the
holomorphic part of the support being
$C^{\mathbf{(1,0)}}=\mathbb{C}^{\times}_{X}$. In this section, we introduce
the M5-branes supported on more general holomorphic curves
$C^{\mathbf{(p,q)}}$. We explain the gauge-invariant non-transverse
intersections of M5-branes provide intertwiners between the $q$-boson
representations. We elucidate the M2-branes, which transversally intersect the
M5-brane web, give rise to the $qq$-characters of the 5d $\mathcal{N}=1$
effective gauge theory on the worldvolume of the M5-brane web.

###  5.1 Non-transverse intersections of M5-branes

We discuss the M5-branes whose holomorphic part of the support is generic
holomorphic curve $C^{\mathbf{(p,q)}}$. We find $q$-boson representations
associated to them, and study their intersections in terms of the intertwiners
of the $q$-boson representations.

####  5.1.1 M5-branes on holomorphic curves and $SL(2,\mathbb{Z})$-dual
$q$-bosons

In section 2.2.1, we have argued that the worldvolume theory on single
M5-brane, supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}\times\mathbb{C}^{\times}_{X}$,
gets localized to a free $q$-boson on $\mathbb{C}^{\times}_{X}$. For the
single M5-brane supported on
$\mathbb{R}^{2}_{{\varepsilon}_{c+1}}\times\mathbb{R}^{2}_{{\varepsilon}_{c-1}}\times
C^{\mathbf{(p,q)}}$, the only modification is the holomorphic support, so that
the algebra of local operators should again be the $q$-boson algebra supported
on the holomorphic curve $C^{\mathbf{(p,q)}}$.

In fact, there exists a symmetry that exchanges the M5-branes supported on
different holomorphic curves $C^{\mathbf{(p,q)}}$ with one another: the
$SL(2,\mathbb{Z})$ symmetry of reparametrization of the holomorphic surface
$\mathbb{C}^{\times}_{X}\times\mathbb{C}^{\times}_{Z}$. Specifically, let us
reparametrize this complex two-dimensional space by $X^{\prime}=X^{a}Z^{b}$
and $Z^{\prime}=X^{c}Z^{d}$. If $ad-bc=1$, the 3-form background of the
twisted M-theory is invariant under the reparametrization since