can be made safe from corruption (part (i)), and that most states are completely non-corrupt (part (ii)). Other sources of reward corruption, such as an irrational human in CIRL or misevaluations in SSRL, are likely better analysed under different assumptions. For these cases, we note that in standard CRMDPs the source of the corruption is unimportant. Thus, techniques suitable for standard CRMDPs are still applicable, including quantilisation described in \[sec:quant\] below.

How \[as:lim-cor-df\] helps agents in CRMDPs with decoupled feedback is illustrated in the following example, and stated more generally in \[th:irf-learnability,th:cr-sublinear\] below.

Let $\S=\{s_1,s_2\}$ and $\R=\{0,1\}$. We represent true reward functions $\irf$ with pairs $\langle\irf(s_1), \irf(s_2)\rangle\in \{0,1\}^2$, and observed reward functions $\orf_s$ with pairs $\langle\orf_{s}(s_1),\orf_{s}(s_2)\rangle\in\{0,1,\#\}^2$.

Assume that a Decoupled RL agent observes the same rewards from both states $s_1$ and $s_2$, $\orf_{s_1}=\orf_{s_2} = \langle 0,1 \rangle$. What can it say about the true reward $\irf$, if it knows that at most $q=1$ state is corrupt? By \[as:lim-cor-df\], an observed pair $\langle\orf_{s}(s_1),\orf_{s}(s_2)\rangle$ disagrees with the true reward $\langle\irf(s_1), \irf(s_2)\rangle$ only if $s$ is corrupt. Therefore, any hypothesis other than $\irf=\langle 0,1 \rangle$ must imply that *both* states $s_1$ and $s_2$ are corrupt. If the agent knows that at most $q=1$ 