```python
import sympy as sy
import control as pc
```

Part of the big question is whether we are simulating the dynamic response, or the steady-state mathematics iterations. Here below is the steady state reponse mathamtical system design, rather than a dynamic diffferential changes with time. Hence, Python's control toolbox cannot solve a steady-state response of a already stable system. However, attempting to perform a fully stable model means that if we want to perform feedback, there's a factor of recalculating stability from that until differential changes become 0 at different time-steps. So for example, if you were to perform feedback after an initial state, you can get a steady state reponse, but this is not the dynamic response, this is solving for the right answer after a number of iterations, and all the initial steady-state answers are wrong for a feedback system. So this gives a steady state response. If we change an input signal also at another point of time, then there is a new steady state response, but the question is whether you want to account for the time dynamics. Because currently, the system is accounting for the digital computing dynamics rather than the actual transient model, so you can't account for how the system dynamically responds and results in a real-time steady-state. So if an input changes in time, you can actually model the time differential change. If we begin considering delays and inputs into the rest of the system, we need to model time as time and computations as computations - and hence we need a time variable input.

There's also the aspect that optics is fundamentally linear and 3D, so how do we account for differential changes in it? What is its dynamic response? There are only dynamic changes on dynamic inputs. What are time changes in optics? You could argue it is near instantaneous. So there's group delay as the time it takes for light to travel through the waveguide, specially if it has been affected in any way. So there is a need to model transient and steady-state responses, but each of these models should be kept separate.

Now, getting a time domain model of a thermo-optic phase shifter response is non-trivial nor necessarily accurate. So, time is a completely new thing in the optical domain for these type of modelled systems. Yes, there is propagation time effects.

Let's consider an MZI, in one arm there's light going over a certain length which incurrs a certain propagation time. Assuming we're only in steady-state mode, when does change occur? You can differtiate phase shift against length for example, but you would be discretizing space-wise. So imagine Pac-Mans moving in a grid of time, according to their position, their properties change, such as phase and this is also dependent in temperature at a given position, but then there must be a differential in temperature and length to calculate a steady state response. So there's this big aspect of time differentials. Could we work based purely on steady-state time differentials? Actually, that could work. There's also the factor of where to add delays, but if we consider every wavelength arm as a system, then we can simply follow the basic principles of control theory.

Hence, the time discretization must occur at a fraction of the time of propagation of the shortest path in the model.

So our inputs into the control system are the electromagnetic signals, that are time-domain constrained

#### Beamsplitter
\begin{equation}
\begin{bmatrix} E_{out0}(t) \\ E_{out1}(t) \end{bmatrix} =
\begin{bmatrix} r_{ac}& t_{bc} \\  t_{ad}& r_{bd} \end{bmatrix}
\begin{bmatrix} E_{in0}(t) \\ E_{in1}(t) \end{bmatrix} =
\begin{bmatrix} \sqrt{r} &  i\sqrt{1-r} \\  i \sqrt{1-r} & \sqrt{r} \end{bmatrix}
\begin{bmatrix} E_{in0}(t) \\ E_{in1}(t) \end{bmatrix}  =
\textbf{B}\textbf{E(t)}
\end{equation}

#### Phase Shifter on one arm

Assuming a phase shifter on top arm
\begin{equation}
\textbf{ $\Phi$ } = \begin{bmatrix} e^{i\phi} & 0 \\ 0 & 1\end{bmatrix}
\end{equation}

Note we're not yet taking into account Linear and LinearGroup delay, or the quantum probability variable within the design of the device.

Hence the MZI configuration is:

\begin{equation}
M\textbf{E(t)}  = \textbf{B} \textbf{ $\Phi$ } \textbf{B}\textbf{E(t)}
\end{equation}

\begin{equation}
M\textbf{E(t)}  = \begin{bmatrix} \sqrt{r} &  i\sqrt{1-r} \\  i \sqrt{1-r} & \sqrt{r} \end{bmatrix} \begin{bmatrix} e^{i\phi} & 0 \\ 0 & 1\end{bmatrix} \begin{bmatrix} \sqrt{r} &  i\sqrt{1-r} \\  i \sqrt{1-r} & \sqrt{r} \end{bmatrix}\begin{bmatrix} E_{in0}(t) \\ E_{in1}(t) \end{bmatrix} = \begin{bmatrix} E_{out0}(t) \\ E_{out1}(t) \end{bmatrix}
\end{equation}

Now we could just crunch some matrices together and input our time signals, but what if we wanted to leverage classical control system theory with regards to MIMO inputs and outputs, so that it is easier to design this type of systems togther and include feedback and nonlinearity back into the classical control models?

The standard way to start is to create the differential equations for this type of system and we can use the fast linear properties of SymPy to create a single MZI component, and the quantum probability variable could be added into this later. Right now let´s consider that we've only got differential equations, with equal reflectivity on both sides of the matrix:

```python
t = sy.Symbol("t")
E0 = sy.Function("E_0")(t)
E0_s = sy.Symbol("E_0")
dE0_s = sy.Symbol("dE_0")
E1 = sy.Function("E_1")(t)
E1_s = sy.Symbol("E_1")
dE1_s = sy.Symbol("dE_1")
phi = sy.Function("\phi")(t)
r_0, r_1 = sy.symbols("r_0 r_1")
B_0  = sy.Matrix([[sy.sqrt(r_0), sy.I*sy.sqrt(1 - r_0)],
				[sy.I*sy.sqrt(1 - r_0), sy.sqrt(r_0)]])
B_1  = sy.Matrix([[sy.sqrt(r_1), sy.I*sy.sqrt(1 - r_1)],
				[sy.I*sy.sqrt(1 - r_1), sy.sqrt(r_1)]])
Phi = sy.Matrix([[sy.exp(sy.I * phi), 0],
				[0, 1]])
E = sy.Matrix([[E0], [E1]])

out = B_0 * Phi * B_1 * E
out
```

$\displaystyle \left[\begin{matrix}\left(\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}\right) \operatorname{E_{0}}{\left(t \right)} + \left(i \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + i \sqrt{r_{1}} \sqrt{1 - r_{0}}\right) \operatorname{E_{1}}{\left(t \right)}\\\left(\sqrt{r_{0}} \sqrt{r_{1}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}}\right) \operatorname{E_{1}}{\left(t \right)} + \left(i \sqrt{r_{0}} \sqrt{1 - r_{1}} + i \sqrt{r_{1}} \sqrt{1 - r_{0}} e^{i \phi{\left(t \right)}}\right) \operatorname{E_{0}}{\left(t \right)}\end{matrix}\right]$

```python
differential_time_response = out.diff(t)
differential_time_response = differential_time_response.replace(E0.diff(t), dE0_s)
differential_time_response = differential_time_response.replace(E1.diff(t), dE1_s)
differential_time_response
```

$\displaystyle \left[\begin{matrix}dE_{0} \left(\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}\right) + dE_{1} \left(i \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + i \sqrt{r_{1}} \sqrt{1 - r_{0}}\right) + i \sqrt{r_{0}} \sqrt{r_{1}} \operatorname{E_{0}}{\left(t \right)} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} - \sqrt{r_{0}} \sqrt{1 - r_{1}} \operatorname{E_{1}}{\left(t \right)} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}\\dE_{0} \left(i \sqrt{r_{0}} \sqrt{1 - r_{1}} + i \sqrt{r_{1}} \sqrt{1 - r_{0}} e^{i \phi{\left(t \right)}}\right) + dE_{1} \left(\sqrt{r_{0}} \sqrt{r_{1}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}}\right) - \sqrt{r_{1}} \sqrt{1 - r_{0}} \operatorname{E_{0}}{\left(t \right)} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} - i \sqrt{1 - r_{0}} \sqrt{1 - r_{1}} \operatorname{E_{1}}{\left(t \right)} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}\end{matrix}\right]$

The problem becomes an equilibrium condition to solve. One could argue that when the output is in equilibrium or in steady state:

\begin{equation}
\begin{bmatrix} \frac{dE_{out0}}{dt}  \\ \frac{dE_{out1}}{dt} \end{bmatrix}  = 0
\end{equation}

```python
# Note we're only expecting one solution
dE0_dt = sy.solve(differential_time_response[0], dE0_s)[0]
dE0_dt = dE0_dt.replace(E0, E0_s)
dE0_dt = dE0_dt.replace(E1, E1_s)
dE0_dt
```

$\displaystyle \frac{- i E_{0} \sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} + E_{1} \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} - i dE_{1} \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} - i dE_{1} \sqrt{r_{1}} \sqrt{1 - r_{0}}}{\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}}$

```python
# Note we're only expecting one solution
dE1_dt = sy.solve(differential_time_response[0], dE1_s)[0]
dE1_dt = dE1_dt.replace(E0, E0_s)
dE1_dt = dE1_dt.replace(E1, E1_s)
# dE1_dt = dE1_dt.replace(dE0_s, dE0_dt) # Cannot be solved canonically as it is a complementary system.
dE1_dt
```

$\displaystyle \frac{i \left(i E_{0} \sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} - E_{1} \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)} + dE_{0} \sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - dE_{0} \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}\right)}{\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{r_{1}} \sqrt{1 - r_{0}}}$

Note that this is a complemenetary function with regards to the other state variable.

Hence, we can write in the nonlinear state-space equations:

\begin{equation}
\dot{\textbf{x}} = \textbf{A} \textbf{x} +  \textbf{B} \textbf{u}
\end{equation}

\begin{equation}
\textbf{y} = \textbf{C} \textbf{x} + \textbf{D} \textbf{u}
\end{equation}

```python
A, B = sy.linear_eq_to_matrix([dE0_dt, dE1_dt], [E0_s, E1_s])
B = sy.simplify(B)
A = sy.simplify(A)
```

```python
A
```

$\displaystyle \left[\begin{matrix}- \frac{i \sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}}{\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}} & \frac{\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}}{\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}}\\- \frac{\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}}{\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{r_{1}} \sqrt{1 - r_{0}}} & - \frac{i \sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} \frac{d}{d t} \phi{\left(t \right)}}{\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{r_{1}} \sqrt{1 - r_{0}}}\end{matrix}\right]$

```python
B
```

$\displaystyle \left[\begin{matrix}\frac{i dE_{1} \left(\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{r_{1}} \sqrt{1 - r_{0}}\right)}{\sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} - \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}}\\\frac{i dE_{0} \left(- \sqrt{r_{0}} \sqrt{r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{1 - r_{0}} \sqrt{1 - r_{1}}\right)}{\sqrt{r_{0}} \sqrt{1 - r_{1}} e^{i \phi{\left(t \right)}} + \sqrt{r_{1}} \sqrt{1 - r_{0}}}\end{matrix}\right]$

Note that the differential state inputs are coupled into the state space system for the opposite original state variables, hence are coupled together and this is the nonlinear represenatation. We can no longer set the differntials inputs to 0, since they are our effective state changes. However, we know have a nonlinear system we have to deal with. It is both nonlinear in $\phi$ and in $\frac{dE_0}{dt} * e^{\phi}$. Note that this system should always be oscillating, that is the nature of light. So let us redefine what a steady state of a system is, because there will be none using standard control theory in optics. But is there a steady state in an MZI? That is a different question. But there is a steady state in an MZI for a given wavelength and is fully dependent on the phase input.

We're also dealing with complementary systems.

### Thermo-optic Phase Shifter Model

```python

```

```python

```


# Fixed Cell Analysis – Multi-channel spot detection

***

```python
import glob, os, random, re, sys, math, warnings
import numpy as np
import pandas as pd
import seaborn as sns
import scipy.ndimage as ndi
import matplotlib.pyplot as plt
import ipywidgets as widgets
import trackpy as tp
import tensorflow as tf

import pims
import skimage.io
import skimage.feature
import skimage.filters
import skimage.segmentation
import skimage.morphology
import skimage.measure

import stardist.models
import utils.model_builder
import utils.predictions

model_unet = utils.model_builder.standard_unet()
model_unet.load_weights('./models/UNet.h5')
model_star = stardist.models.StarDist2D(None, name='Stardist', basedir='./models/')
```

    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      _np_qint8 = np.dtype([("qint8", np.int8, 1)])
    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      _np_qint16 = np.dtype([("qint16", np.int16, 1)])
    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      _np_qint32 = np.dtype([("qint32", np.int32, 1)])
    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      np_resource = np.dtype([("resource", np.ubyte, 1)])
    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      _np_qint8 = np.dtype([("qint8", np.int8, 1)])
    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      _np_qint16 = np.dtype([("qint16", np.int16, 1)])
    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      _np_qint32 = np.dtype([("qint32", np.int32, 1)])
    /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
      np_resource = np.dtype([("resource", np.ubyte, 1)])

    WARNING:tensorflow:From /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
    Instructions for updating:
    Call initializer instance with the dtype argument instead of passing it to the constructor

    Using TensorFlow backend.

    WARNING:tensorflow:From /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

    WARNING:tensorflow:From /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

    WARNING:tensorflow:From /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

    WARNING:tensorflow:From /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:3976: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

    WARNING:tensorflow:From /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:2018: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.

    Loading network weights from 'weights_best.h5'.
    WARNING:tensorflow:From /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.

    WARNING:tensorflow:From /Users/beichenberger/anaconda3/envs/image/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:181: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

    Loading thresholds from 'thresholds.json'.
    Using default values: prob_thresh=0.455612, nms_thresh=0.3.

## 1. Image import

```python
root = './data'
channels = 3
```

```python
def get_root_files(root, extension='nd'):
    files = sorted(glob.glob(f'{root}/*.{extension}'))
    files = [f.split('/')[-1] for f in files]
    files = [f.split('.')[0] for f in files]
    return files

def get_stk_files(root, file):
    files = glob.glob(f'{root}/{file}*.stk')
    return files
```

```python
# Nd / stk files
files = get_root_files(root, 'nd')

@widgets.interact(file=widgets.Dropdown(options=files, description='File: '),
                  sharp=widgets.Checkbox(value=False, description='Use sharpest slice'))
def f(file, sharp):
    global img_file, img, var_sharp
    img_file = file
    img_files = get_stk_files(root, file)
    var_sharp = sharp
    img = list(map(skimage.io.imread, img_files))
    img = np.array(img)

    @widgets.interact(frame = widgets.IntSlider(value=img.shape[1]//2, min=1, max=img.shape[1]-1, description='Slice: '))
    def g(frame):
        img_plot = img[:, frame]
        fig, ax = plt.subplots(1, 3, figsize=(15,10))
        for i, image in enumerate(img_plot):
            ax[i].set_title(f'Channel {i}')
            ax[i].imshow(img_plot[i])
        plt.show()
```

    interactive(children=(Dropdown(description='File: ', options=('DCP1a_6h_1',), value='DCP1a_6h_1'), Checkbox(va…

## 2. Cellular segmentation

To do a cellular analysis, all cells are segmented. Segmentation is performed on a acquisition channel with labeled nuclei. This allows for simpler distinguishing between individual cells. The parts of this multi-step process are described below. Alternatively, the speedrun can be used which combines all individual steps performed below.

#### Preprocessing

```python
def sharp(img, axis=0, slice_only=False):
    """Returns index of the sharpest slice in an image array of shape z, x, y.

    Args:
        img (arr): Image array (len(img.shape)) == 3) of which sharpest slice is to be determined.
        return_slice (bool): Return index of sharpest slice.

    Returns:
        sharpest (int): Index of sharpest slice.
        img (arr): Image of sharpest slice.
    """
    assert img.ndim==3

    sharpness = []
    array = np.asarray(img, dtype=np.int32)
    for i in range(array.shape[0]):
        y, x = np.gradient(array[i])
        norm = np.sqrt(x**2 + y**2)
        sharpness.append(np.average(norm))
    sharpest = sharpness.index(max(sharpness))

    if slice_only:
        return sharpest
    return img[sharpest]
```

```python
c_nucleus = 0
c_cytoplasm = 1
c_granules = 2

sharp_slice = sharp(img[c_nucleus], slice_only=True) if var_sharp else None

assert img.ndim==4
img = np.array([img[i, sharp_slice]
                if var_sharp
                else np.max(img[i], axis=0)
                for i in range(img.shape[0])])

img_nucleus = img[c_nucleus]
img_cytoplasm = img[c_cytoplasm]
img_granules = img[c_granules]
```

#### Nucleus – automated

```python
def fill_label_holes(lbl_img, **kwargs):
    """Fill small holes in label image."""
    # TODO: refactor 'fill_label_holes' and 'edt_prob' to share code
    def grow(sl,interior):
        return tuple(slice(s.start-int(w[0]),s.stop+int(w[1])) for s,w in zip(sl,interior))
    def shrink(interior):
        return tuple(slice(int(w[0]),(-1 if w[1] else None)) for w in interior)
    objects = ndi.measurements.find_objects(lbl_img)
    lbl_img_filled = np.zeros_like(lbl_img)
    for i,sl in enumerate(objects,1):
        if sl is None: continue
        interior = [(s.start>0,s.stop<sz) for s,sz in zip(sl,lbl_img.shape)]
        shrink_slice = shrink(interior)
        grown_mask = lbl_img[grow(sl,interior)]==i
        mask_filled = ndi.morphology.binary_fill_holes(grown_mask, **kwargs)[shrink_slice]
        lbl_img_filled[sl][mask_filled] = i
    return lbl_img_filled
```

```python
img_seg_nucleus = utils.predictions.starnet_single(model_star, model_unet, img_nucleus)
img_seg_nucleus = fill_label_holes(img_seg_nucleus)

fig, ax = plt.subplots(1, 2, figsize=(10, 5))
ax[0].set_title('Nuclear image')
ax[0].imshow(img_nucleus)
ax[1].set_title('Nuclear segmentation')
ax[1].imshow(img_seg_nucleus)
plt.show()
```

#### Cytoplasm – quantile based

```python
@widgets.interact(quantile = widgets.FloatSlider(value=0.75, min=0, max=1, step=0.01, description='Quantile: '))
def g(quantile):
    global img_thresh_cytoplasm

    img_smooth = skimage.filters.gaussian(img_cytoplasm, 2)
    img_quantile = np.quantile(img_smooth, quantile)
    img_thresh_cytoplasm = img_smooth > img_quantile
    img_thresh_cytoplasm = ndi.binary_fill_holes(img_thresh_cytoplasm)

    fig, ax = plt.subplots(1, 2, figsize=(10, 5))
    ax[0].set_title('Cytoplasmic image')
    ax[0].imshow(img_cytoplasm)
    ax[1].set_title('Cytoplasmic threshold')
    ax[1].imshow(img_thresh_cytoplasm)
    plt.show()
```

    interactive(children=(FloatSlider(value=0.75, description='Quantile: ', max=1.0, step=0.01), Output()), _dom_c…

```python
img_seg_cytoplasm = skimage.segmentation.watershed(~img_thresh_cytoplasm, img_seg_nucleus, mask=img_thresh_cytoplasm)
img_seg_cytoplasm = np.where(img_seg_nucleus, 0, img_seg_cytoplasm)

fig, ax = plt.subplots(1, 2, figsize=(10, 5))
ax[0].set_title('Cytoplasmic image')
ax[0].imshow(img_cytoplasm)
ax[1].set_title('Cytoplasmic segmentation')
ax[1].imshow(img_seg_cytoplasm)
plt.show()
```

#### Granules – quantile based

```python
@widgets.interact(quantile = widgets.FloatSlider(value=0.95, min=0, max=1, step=0.01, description='Quantile: '))
def g(quantile):
    global var_quantile_granules, quantile_dist_transform

    img_smooth = skimage.filters.gaussian(img_granules, 2)
    var_quantile_granules = np.quantile(img_smooth, quantile)

    def quantile_dist_transform(img, quantile, return_both=False):
        img_seg = img > quantile
        img_seg_dist = ndi.distance_transform_edt(img_seg)
        if return_both:
            return img_seg, img_seg_dist
        return img_seg_dist

    img_seg_granules, img_seg_granules_dist = quantile_dist_transform(img_smooth, var_quantile_granules, True)

    fig, ax = plt.subplots(1, 3, figsize=(15, 10))
    ax[0].set_title('Granular image')
    ax[0].imshow(img_granules)
    ax[1].set_title('Granular segmentation')
    ax[1].imshow(img_seg_granules)
    ax[2].set_title('Granular distance transform')
    ax[2].imshow(img_seg_granules_dist)
    plt.show()
```

    interactive(children=(FloatSlider(value=0.95, description='Quantile: ', max=1.0, step=0.01), Output()), _dom_c…

## 3. Spot detection

To detect spots, one has to input the corresponding channels. Note, the 'first' channel is defined as channel 0.

Parameters:
- Image – frame in time series of images.
- Diameter – expected blob diameter (actual size still measured). When in doubt, round up.
- Minmass – the minimum integrated brightness.
- Maxsize – maximum radius of gyration.
- Percentile – spots must be above percentile. Default is 64.

```python
c_spots = [1, 2]
```

```python
dfs = [None] * len(c_spots)

@widgets.interact(channel = widgets.Dropdown(options=c_spots, description='Channel: '),
    diameter = widgets.IntSlider(min=1, max=101, step=2, value=15, description='Diameter: '),
    minmass = widgets.IntSlider(min=0, max=1_000_000, step=1000, value=500_000, description='Minmass: '),
    maxsize = widgets.FloatSlider(min=0, max=10, step=0.5, value=5, description='Maxsize: '),
    percentile = widgets.IntSlider(min=0, max=100, step=1, value=64, description='Percentile: ')
)
def g(channel, diameter, minmass, maxsize, percentile):
    global dfs

    f = pims.Frame(img[channel])
    df = tp.locate(f, diameter=diameter, minmass=minmass, maxsize=maxsize, percentile=percentile, engine='numba')

    df['channel'] = channel
    dfs[c_spots.index(channel)] = df

    plt.figure(figsize=(15, 10))
    plt.title(f'Channel {channel}')
    tp.annotate(df, f, color='b', imshow_style={'cmap':'viridis'})
    plt.show()
```

    interactive(children=(Dropdown(description='Channel: ', options=(1, 2), value=1), IntSlider(value=15, descript…

## 4. Pearson correlation coefficient analysis

The pearson correlation coefficient (PCC) can be used to quantify colocalization with a intensity weight. The equation is shown below.

\begin{align}
r = \frac{\sum_i\left(I_{1,i} - \bar{I}_1\right)\left(I_{2,i} - \bar{I}_2\right)}{\sqrt{\left(\sum_i \left(I_{1,i} - \bar{I}_1\right)^2\right)\left(\sum_i \left(I_{2,i} - \bar{I}_2\right)^2\right)}}.
\end{align}

The image set is tested for statistical relevance i.e. if the colocalization is greater than simple variance. This is done via scrambling small blocks of the image. The blocks' height and width are equal to the point spread function (PSF). The PCC is then calculated on the scrambled image blocks. The PSF can be approximated using the rayleigh criterion, which in case of these images / microscopes is as follows.

\begin{align}
R_{\text{Confocal}} = \frac{0.4 \times \lambda}{\text{NA}} = \frac{0.4 \times 640\text{ nm}}{1.45} = 278\text{ nm} \rightarrow \text{PSF} \approx 3\text{ px}
\end{align}

Due to the fact that the PSF is close to one pixel, every pixel will be scrambled inside the segmented cellular area. Alternatively – if the PSF were larger – the edges would be mirrored to obtain an image that is a multiple of the PSF. This, however, due to the cellular segmentation (resulting in uneven shapes) is not possible. The widget below can be used to visualize the effect of scrambling on the r-value.

For this analysis, the segmented cytoplasm will give the 'real' PCC value. A box around the segmented mask will be scrambled and will provide information on whether the current cell has a statistically accurate PCC value.

```python
def coloc_cellbox(img1, img2, roi, psf=3, scrambles=10):
    # Generate cell boxes
    img1_box = extract_mask(roi, img1, psf)
    img2_box = extract_mask(roi, img2, psf)

    # Mirror edges (in case box isn't used)
    #img1_mirror = mirror_edges(img1, psf)
    #img2_mirror = mirror_edges(img2, psf)

    # Generate blocks of both channels
    img1_blocks = img_to_blocks(img1_box, psf)
    img2_blocks = img_to_blocks(img1_box, psf)

    # Store blocks of channel 2 as flattened array (not scrambled)
    img2_blocks_flat = np.array(img2_blocks).flatten()

    # Scamblin' and obtain R value
    img1_scr = scramble(img1_blocks, img2_blocks_flat, scrambles)

    # Unscrambled R value
    img1_unscr, p = stats.pearsonr(np.array(img1_blocks).ravel(), img2_blocks_flat)

    # Probablity
    img1_prob = sum(i > img1_unscr for i in img1_scr) / len(img1_scr)

    return img1_scr, img1_prob
```

```python
def coloc_cellmask(img1, img2):
    r_unscr, _ = stats.pearsonr(img1.flatten(), img2.flatten())
    return r_unscr
```

```python
@interact(t_scrambles = widgets.IntSlider(min=0, max=2000, step=10, value=10, description='Scrambles: '),
          t_cell = widgets.IntSlider(min=1, max=len(np.unique(img0_seg)), value=1, description='Cell: '))

def g(t_scrambles, t_cell):
    cell_mask = img0_seg==t_cell

    t_scr, t_prob = coloc_cellbox(img[1], img[2], cell_mask, scrambles=t_scrambles)
    t_unscr = coloc_cellmask(img[1]*cell_mask, img[2]*cell_mask)

    # Visualization
    _ = sns.distplot(t_scr, bins=int(np.sqrt(t_scrambles)), label='Scrambles')
    plt.plot([t_unscr, t_unscr], plt.gca().get_ylim(), '-', label='Real image')

    plt.title(f'Scrambled histogram vs. "real" image – Prob: {t_prob}')
    plt.xlabel('Pearson correlation coefficient')
    plt.xlim(-1, 1)
    plt.yticks([])
    plt.ylabel(None)
    plt.legend(frameon=False)
    sns.despine(trim=True, bottom=False, left=True)
    plt.show()
```

## 5. Measure other channels and cell number

As currently only one channel is tracked, one has to calculate the (potential) spot intensity in the second – untracked – channel. In addition, the tracks are assigned to their belonging cells. If a track is detected in the background it is sacrificed.

```python
# Cellular assignment
def assign_cell(df, img_seg):
    def _get_value(img, y, x):
        return img[int(round(y)), int(round(x))]

    df['cell'] = df.apply(lambda row: _get_value(img_seg, row['y'], row['x']), axis=1)
    print('Before:', df.index.nunique())
    df = df[df['cell']!=0]
    print('After:', df.index.nunique())
    return df

def assign_granule(df, img, quantile):
    def _get_value(img, y, x, quantile):
        return img[int(y), int(x)]

    img = skimage.filters.gaussian(img, 2)
    img = quantile_dist_transform(img, var_quantile_granules)

    df['granule'] = df.apply(lambda row: _get_value(img, row['y'], row['x'], quantile), axis=1)
    return df
```

```python
dfs[0].index.nunique()
```

    32

```python
dfs = [assign_cell(df, img_seg_cytoplasm) for df in dfs]
```

    Before: 53
    After: 41
    Before: 15
    After: 12

## 6. Output metrics

```python
columns = ['file', 'channel', 'cell',
           'y', 'x',
           'mass', 'size', 'ecc',
           'signal', 'raw_mass', 'ep']

df = pd.concat(dfs)
df['file'] = img_file
df = df[columns]
```

```python
df
```

      file
      channel
      cell
      y
      x
      mass
      size
      ecc
      signal
      raw_mass
      ep

      19
      DCP1a_6h_1
      1
      2
      21.986593
      408.006515
      4.959346e+05
      4.047838
      0.551052
      16490.126012
      2655920.0
      0.023936

      115
      DCP1a_6h_1
      1
      29
      98.153682
      803.605896
      4.960438e+05
      3.264503
      0.469882
      19431.677764
      2299092.0
      0.028040

      134
      DCP1a_6h_1
      1
      29
      113.149143
      778.706352
      6.717762e+05
      2.306807
      0.008327
      34997.039318
      2594134.0
      0.024558

      186
      DCP1a_6h_1
      1
      29
      154.924164
      765.342966
      3.837611e+05
      2.952081
      0.210195
      15925.063930
      2278329.0
      0.028323

      230
      DCP1a_6h_1
      1
      13
      190.416011
      461.490896
      8.631197e+05
      3.326110
      0.350877
      35294.746312
      2428434.0
      0.026399

      262
      DCP1a_6h_1
      1
      29
      215.571533
      866.643434
      1.340092e+06
      2.944455
      0.014501
      33120.387421
      4602251.0
      0.013309

      269
      DCP1a_6h_1
      1
      29
      215.572205
      763.061428
      5.187406e+05
      3.837039
      0.293809
      11559.555732
      3149390.0
      0.019906

      287
      DCP1a_6h_1
      1
      29
      233.714052
      830.921313
      4.810939e+05
      2.178578
      0.153058
      26253.107216
      1946249.0
      0.033765

      322
      DCP1a_6h_1
      1
      29
      257.884861
      792.829671
      6.375295e+05
      2.926910
      0.254461
      23479.459626
      5382228.0
      0.011299

      336
      DCP1a_6h_1
      1
      29
      273.026500
      787.270019
      7.299478e+05
      3.871896
      0.190478
      20872.424627
      5919187.0
      0.010235

      380
      DCP1a_6h_1
      1
      29
      301.799247
      792.071423
      5.545461e+05
      2.851131
      0.232748
      24931.184837
      4416624.0
      0.013898

      414
      DCP1a_6h_1
      1
      29
      327.102551
      800.436123
      8.607949e+05
      3.527838
      0.394150
      30857.555952
      3971961.0
      0.015544

      439
      DCP1a_6h_1
      1
      27
      347.281680
      910.425820
      6.091757e+05
      2.470010
      0.027636
      24626.374206
      2453882.0
      0.026099

      466
      DCP1a_6h_1
      1
      12
      366.670673
      798.281519
      4.229306e+05
      3.986234
      0.314392
      8951.229162
      1942547.0
      0.033838

      499
      DCP1a_6h_1
      1
      24
      393.383957
      337.707661
      4.636841e+05
      2.628690
      0.098802
      17091.352067
      1699650.0
      0.039386

      521
      DCP1a_6h_1
      1
      27
      406.503210
      931.506252
      8.578682e+05
      2.553693
      0.132010
      34837.530364
      2568360.0
      0.024827

      540
      DCP1a_6h_1
      1
      25
      421.492103
      555.913993
      8.244688e+05
      2.431557
      0.042727
      38824.608414
      1673466.0
      0.040094

      547
      DCP1a_6h_1
      1
      32
      426.878605
      249.887518
      3.866794e+05
      3.140500
      0.258324
      15529.197580
      1686252.0
      0.039745

      642
      DCP1a_6h_1
      1
      35
      500.301921
      283.818922
      4.269751e+05
      2.482090
      0.239131
      19757.799308
      1513373.0
      0.045050

      662
      DCP1a_6h_1
      1
      12
      513.779010
      780.970308
      4.701207e+05
      2.229875
      0.217296
      24736.157696
      1537903.0
      0.044213

      703
      DCP1a_6h_1
      1
      32
      540.671892
      125.125175
      5.537776e+05
      2.356300
      0.187407
      26987.365030
      2347098.0
      0.027408

      720
      DCP1a_6h_1
      1
      19
      555.686552
      114.646132
      7.434241e+05
      2.635275
      0.126216
      29578.255399
      3105066.0
      0.020212

      722
      DCP1a_6h_1
      1
      32
      557.826414
      134.789249
      5.365959e+05
      2.997773
      0.272824
      21135.259218
      2443885.0
      0.026216

      730
      DCP1a_6h_1
      1
      19
      562.045872
      92.655609
      4.891345e+05
      2.814089
      0.163494
      22360.959596
      3241305.0
      0.019301

      731
      DCP1a_6h_1
      1
      32
      561.556380
      171.602239
      6.777406e+05
      4.011885
      0.461113
      24189.823386
      2620687.0
      0.024287

      755
      DCP1a_6h_1
      1
      19
      581.125231
      102.480439
      4.562912e+05
      2.424332
      0.104600
      24340.291346
      2816742.0
      0.022454

      768
      DCP1a_6h_1
      1
      19
      595.030710
      74.337951
      5.250473e+05
      2.485831
      0.107448
      26659.951915
      3744524.0
      0.016547

      840
      DCP1a_6h_1
      1
      19
      655.674659
      52.631661
      9.971563e+05
      2.634556
      0.145432
      34306.694900
      4023194.0
      0.015335

      884
      DCP1a_6h_1
      1
      22
      691.920158
      18.015535
      7.095113e+05
      2.461103
      0.034535
      30785.228006
      2149314.0
      0.030215

      916
      DCP1a_6h_1
      1
      19
      712.178753
      152.192456
      5.406779e+05
      2.868411
      0.228184
      22699.351060
      2789609.0
      0.022691

      934
      DCP1a_6h_1
      1
      33
      729.717196
      519.601131
      6.271337e+05
      2.476121
      0.122964
      28353.846590
      2584487.0
      0.024658

      937
      DCP1a_6h_1
      1
      19
      731.027923
      155.283518
      1.563553e+06
      3.159656
      0.230297
      34587.611478
      4654480.0
      0.013153

      939
      DCP1a_6h_1
      1
      19
      737.074149
      106.652197
      4.427607e+05
      2.193673
      0.071090
      26904.704520
      2909877.0
      0.021677

      966
      DCP1a_6h_1
      1
      19
      753.022577
      124.062079
      8.729073e+05
      2.304305
      0.048042
      38256.963191
      3156250.0
      0.019860

      984
      DCP1a_6h_1
      1
      19
      767.999735
      151.574799
      1.896518e+06
      3.657548
      0.292338
      32329.946291
      5350514.0
      0.011369

      1024
      DCP1a_6h_1
      1
      33
      798.707689
      456.155303
      6.035683e+05
      2.482923
      0.095433
      31359.976866
      2015271.0
      0.032468

      1046
      DCP1a_6h_1
      1
      33
      816.625121
      439.991004
      4.165838e+05
      2.091447
      0.003387
      26082.619914
      1564502.0
      0.043339

      1124
      DCP1a_6h_1
      1
      33
      879.483008
      442.441194
      4.444972e+05
      2.464668
      0.111063
      23198.543048
      2336996.0
      0.027538

      1168
      DCP1a_6h_1
      1
      33
      912.299718
      477.764154
      9.829039e+05
      2.441677
      0.117493
      38216.278721
      3360162.0
      0.018571

      1242
      DCP1a_6h_1
      1
      21
      965.529912
      818.967317
      1.150305e+06
      2.514452
      0.070597
      42321.535469
      2860010.0
      0.022087

      1247
      DCP1a_6h_1
      1
      21
      969.982648
      760.998711
      4.051508e+05
      2.150705
      0.046753
      22773.616363
      1631013.0
      0.041299

      370
      DCP1a_6h_1
      2
      34
      281.592687
      189.993637
      4.569511e+05
      2.042142
      0.037724
      29494.153482
      1202085.0
      0.001575

      480
      DCP1a_6h_1
      2
      32
      359.644034
      225.075003
      4.128311e+05
      2.271704
      0.072352
      20906.681681
      1120370.0
      0.001701

      492
      DCP1a_6h_1
      2
      23
      367.379957
      487.231534
      3.967112e+05
      3.176525
      0.275614
      13053.827795
      1232805.0
      0.001533

      554
      DCP1a_6h_1
      2
      27
      405.701073
      930.876095
      5.672502e+05
      2.228268
      0.130963
      28933.062598
      1625577.0
      0.001139

      600
      DCP1a_6h_1
      2
      12
      439.227993
      829.561921
      4.116025e+05
      2.090805
      0.065562
      26211.650884
      1093219.0
      0.001747

      916
      DCP1a_6h_1
      2
      15
      670.497682
      845.062911
      6.541945e+05
      2.083518
      0.041020
      39624.020591
      1326019.0
      0.001416

      1049
      DCP1a_6h_1
      2
      28
      777.787180
      216.075714
      4.681711e+05
      3.679861
      0.190542
      11565.848629
      1830078.0
      0.001004

      1077
      DCP1a_6h_1
      2
      28
      798.569403
      180.719492
      4.588176e+05
      3.566078
      0.320399
      12223.679321
      1983578.0
      0.000923

      1079
      DCP1a_6h_1
      2
      28
      800.723650
      234.832028
      1.092463e+06
      4.540427
      0.094714
      14725.612596
      3791188.0
      0.000471

      1094
      DCP1a_6h_1
      2
      28
      807.525812
      218.143161
      5.568603e+05
      2.490096
      0.035429
      31050.455138
      2008813.0
      0.000911

      1115
      DCP1a_6h_1
      2
      28
      821.642696
      248.339114
      4.715473e+05
      2.931100
      0.086841
      16008.624295
      1985561.0
      0.000922

      1234
      DCP1a_6h_1
      2
      33
      913.120655
      477.299417
      5.994972e+05
      2.353010
      0.041246
      28519.499920
      1525966.0
      0.001218

```python
root_out = './out'

os.makedirs(root_out, exist_ok=True)
df.to_csv(f'./out/{img_file}.csv', index=False)
```


### RCNN与SPPnet的不足
- 1. 训练的pipeline步骤多,
- 2. 训练代价高, 占用的空间与时间多
- 3. 推理过程慢, 难以用于实际应用

### Fast RCNN
- 1. 端到端训练, 多任务损失
- 2. 训练能更新整个网络的参数
- 3. 无需缓存特征

### 网络结构
输入为原始图片及RoI

先将原图经过卷积网络得到特征图,

再通过RoI pooling(单个的SPP层)从特征图提取RoI特征向量

SPPnet无法更新SPP层下的权重,
由于RoI取自整张图片,感受野大,梯度在不同图片的RoI中的传播十分低效,

### 多任务损失
$L(p,u,t^u,v) = L_{cls}(p,u)+\lambda [u \geq 1]L_{loc}(t^u,v)$

$类别损失L_{cls}(p,u)=-log p_u$

boundingbox损失
\begin{equation}
L_{loc}(t^u,v)=\sum _{i\in {x,y,w,h}}smooth_{L_1}(t_i^u-v_i)
\end{equation}

$
smooth_{L_1}(x)=
\left\{
\begin{aligned}
   0.5x_2& & if |x|<1  \\
   |x|-0.5& & otherwise,
 \end{aligned}
 \right.
$
$文章中的L_1损失对轮廓的不那么敏感,相较于RCNN与SPPnet中的L_2损失； 当回归的目标值没有限制的时候, L_2u损失对学习率要特别小心, 容易梯度爆炸$

### mini-batch sampling
全排列数据集,随机取其中一对, RoI大小为128,

其中四分之一正样本RoI,IoU大于0.5,

剩下的为背景,IoU在0.1至0.5之间

### SVD加快检测
对整个分类任务,卷积上花的时间比FC层要多,

但RoI数量的检测上,FC层上花费的时间占了近整个前匮网络的一半,
一个参数较多的FC层能在SVD后大大的加速

$W\approx U\Sigma_tV^T$

能将参数量有uv降至t(u+v)

$具体做法是将一层FC层换成两层,第一层权重为\Sigma_tV^T, 第二层权重为U$

当RoI数量很大时能加速很多


# メモ

手を動かしてまなぶ 線形代数 藤岡敦

を読む。

http://www.shokabo.co.jp/

に詳細な解答があるって。

# 行列

1. 行列とは数を長方形状に並べたものである。
1. すべての成分が 0 の行列を零行列という。
1. 行と列の個数が等しい行列を正方行列という。
1. 特別な正方行列として、単位行列、対角行列、スカラー行列、上三角行列、下三角行列があげられる。
1. 単位行列はクロネッカーのデルタを用いて、簡単に表すことができる。
1. 転置をとっても変わらない正方行列を対称行列という。
1. 行ベクトル、列ベクトルをあわせて数ベクトルという。
1. すべての成分が 0 の数ベクトルを零ベクトルという。

自然数 $i=1,2,\cdots,m$ および $j=1,2,\cdots,n$ に対して、数 $a_{ij}$ が対応しているとする。

このとき、$mn$ 個の数 $a_{11},a_{12},\cdots,a_{mn}$ を丸括弧  ( ) や角括弧 [ ] を用いて長方形状に並べたものを $m \times n$ 行列または$m$行$n$列の行列という。

$a_{ij}$ の $i$ や $j$ を添字(そえじ)という。

```latex
%%latex
\begin{pmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{pmatrix}, \quad
\begin{bmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{bmatrix}
```

\begin{pmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{pmatrix}, \quad
\begin{bmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{bmatrix}

例

次の行列は $2 \times 3$ 行列である。

```latex
%%latex
\begin{pmatrix}
    1 & 2 & 3 \\
    4 & 5 & 6
\end{pmatrix}

```

\begin{pmatrix}
    1 & 2 & 3 \\
    4 & 5 & 6
\end{pmatrix}

```
from sympy import *
A=Matrix([[1,2,3],[4,5,6]])
display(A)
```

$\displaystyle \left[\begin{matrix}1 & 2 & 3\\4 & 5 & 6\end{matrix}\right]$

行列 $A$ について
$A = (a_{ij})$
と書く。

$\quad a_{ij},\quad (a_{i1} a_{i2} \cdots a_{in}), \quad

\begin{pmatrix}
    a_{1j} \\
    a_{2j} \\
    a_{2j} \\
    a_{mj}
\end{pmatrix}$

を$A$の$(i,j)$成分、第$i$行、第$j$列という。

```latex
%%latex
\begin{pmatrix}
    a_{1j} \\
    a_{2j} \\
    a_{2j} \\
    a_{mj}
\end{pmatrix}
```

\displaystyle
\begin{pmatrix}
    a_{1j} \\
    a_{2j} \\
    a_{2j} \\
    a_{mj}
\end{pmatrix}

 $ A=(a_{ij})_{m \times n} $ を $ m \times n $ 行列、 $ B=(b_{kl})_{p \times q} $ を $ p \times q $ 行列とする。

$ A $ と $ B $ が同じ型で、対応する成分が等しいとき、すなわち、 $ m=p $ かつ $ n=q $ で、
任意の $ i=1,2,\cdots,m $ および $ j=1,2,\cdots,n $ に対して $ a_{ij}=b_{ij} $ が成り立つとき

$$ A = B$$

と記し、A $ と $ B $ は等しい、という。A = B $ でないときは

$$ A \neq B $$

$$
\begin{array}{rcl}
\begin{pmatrix}
    a & b & c \\
    d & e & f
\end{pmatrix}
&=&
\begin{pmatrix}
    p & q & r \\
    s & t & u
\end{pmatrix} \\[0.8em]

&\Updownarrow& \\[0.8em]

a = p,\quad  b = q,\quad  c = r&,& d = s,\quad  e = t,\quad  f = u

\end{array}
$$

```latex
%%latex
\begin{array}{rcl}
\begin{pmatrix}
    a & b & c \\
    d & e & f
\end{pmatrix} &=&
\begin{pmatrix}
    p & q & r \\
    s & t & u
\end{pmatrix} \\[0.8em]

&\Updownarrow& \\[0.8em]

a = p,\quad  b = q,\quad  c = r&,& d = s,\quad  e = t,\quad  f = u
\end{array} \\

```

\begin{array}{rcl}
\begin{pmatrix}
    a & b & c \\
    d & e & f
\end{pmatrix} &=&
\begin{pmatrix}
    p & q & r \\
    s & t & u
\end{pmatrix} \\[0.8em]

&\Updownarrow& \\[0.8em]

a = p,\quad  b = q,\quad  c = r&,& d = s,\quad  e = t,\quad  f = u
\end{array} \\

```latex
%%latex
すべての成分が~0~の~m \times n~行列を~O_{m,n}~または~O~と書き、零行列という。
\\[0.8em]
\quad \quad \quad \quad
O_{3,2} =
\begin{pmatrix}
    0 & 0 \\
    0 & 0 \\
    0 & 0
\end{pmatrix}
\\[0.8em]
2~列、~3~行
```

すべての成分が~0~の~m \times n~行列を~O_{m,n}~または~O~と書き、零行列という。
\\[0.8em]
\quad \quad \quad \quad
O_{3,2} =
\begin{pmatrix}
    0 & 0 \\
    0 & 0 \\
    0 & 0
\end{pmatrix}
\\[0.8em]
2~列、~3~行

```latex
%%latex
一般に、2~つの命題~P,~Q~に対して、P~ならば~Q~であることを~P \Longrightarrow Q~と表し、P~と~Q~が同値であることを
~P \Longleftrightarrow Q~と表す。
```

一般に、2~つの命題~P,~Q~に対して、P~ならば~Q~であることを~P \Longrightarrow Q~と表し、P~と~Q~が同値であることを
~P \Longleftrightarrow Q~と表す。

```latex
%%latex
A = (a_{ij})_{n \times n}~を~n \times n~ 行列とするとき、A~を~n~次の正方行列という。\\
成分~a_{11}, a_{22}, \cdots a_{nn}~を~A~の対角成分という。\\[0.8em]

正方行列 ~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i \neq j) \\[0.8em]

を満たすとき、A~を対角行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i \neq j), \quad a_{11} = a_{22} = \cdots = a_{nn} \\[0.8em]

を満たすとき、A~をスカラー行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i > j) \\[0.8em]

を満たすとき、A~を上三角行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i < j) \\[0.8em]

を満たすとき、A~を下三角行列という。
```

A = (a_{ij})_{n \times n}~を~n \times n~ 行列とするとき、A~を~n~次の正方行列という。\\
成分~a_{11}, a_{22}, \cdots a_{nn}~を~A~の対角成分という。\\[0.8em]

正方行列 ~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i \neq j) \\[0.8em]

を満たすとき、A~を対角行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i \neq j), \quad a_{11} = a_{22} = \cdots = a_{nn} \\[0.8em]

を満たすとき、A~をスカラー行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i > j) \\[0.8em]

を満たすとき、A~を上三角行列という。 正方行列~A~が\\[0.8em]

\quad \quad \quad \quad a_{ij} = 0 \quad (i < j) \\[0.8em]

を満たすとき、A~を下三角行列という。

```latex
%%latex
2~次の正方行列
~A~ =
\begin{pmatrix}
    a & b \\
    c & d
\end{pmatrix}
~とする。\\[0.8em]

A~の対角成分は~
\begin{pmatrix}
    a & 0 \\
    0 & d
\end{pmatrix}\\[0.8em]

A~が対角行列、上三角行列、下三角行列となるときそれぞれ\\[0.8em]

\begin{pmatrix}
    a & 0 \\
    0 & a
\end{pmatrix}, \quad

\begin{pmatrix}
    a & b \\
    0 & d
\end{pmatrix}, \quad

\begin{pmatrix}
    a & 0 \\
    c & d
\end{pmatrix}
```

2~次の正方行列
~A~ =
\begin{pmatrix}
    a & b \\
    c & d
\end{pmatrix}
~とする。\\[0.8em]

A~の対角成分は~
\begin{pmatrix}
    a & 0 \\
    0 & d
\end{pmatrix}\\[0.8em]

A~が対角行列、上三角行列、下三角行列となるときそれぞれ\\[0.8em]

\begin{pmatrix}
    a & 0 \\
    0 & a
\end{pmatrix}, \quad

\begin{pmatrix}
    a & b \\
    0 & d
\end{pmatrix}, \quad

\begin{pmatrix}
    a & 0 \\
    c & d
\end{pmatrix}

```latex
%%latex
対角成分がすべて~1~の~n~次スカラー行列を~E_n~または~E~と書き、~n~次単位行列という。 ~I_n~や~I~と書くこともある。\\[0.8em]

E_1 = (1) = 1, \quad E_2 =
\begin{pmatrix}
    1 & 0 \\
    0 & 1
\end{pmatrix}, \quad E_3 =
\begin{pmatrix}
    1 & 0 & 0 \\
    0 & 1 & 0 \\
    0 & 0 & 1
\end{pmatrix}
```

対角成分がすべて~1~の~n~次スカラー行列を~E_n~または~E~と書き、~n~次単位行列という。 ~I_n~や~I~と書くこともある。\\[0.8em]

E_1 = (1) = 1, \quad E_2 =
\begin{pmatrix}
    1 & 0 \\
    0 & 1
\end{pmatrix}, \quad E_3 =
\begin{pmatrix}
    1 & 0 & 0 \\
    0 & 1 & 0 \\
    0 & 0 & 1
\end{pmatrix}

```
# クロネッカーのデルタ
%%latex
クロネッカーのデルタ\\[0.8em]

i,j = 1,2, \cdots, n~ に対して\\[0.8em]

\quad \quad \quad \quad \delta =

\left \{
\begin{array}{cl}
    1 & (i = j) \\
    0 & (i \neq j)
\end{array}
\right .
\\[0.8em]
により、~0~または~1~をとる記号~\delta_{ij}~を定める。 \\
~\delta_{ij}~をクロネッカーのデルタという。
```

クロネッカーのデルタ\\[0.8em]

i,j = 1,2, \cdots, n~ に対して\\[0.8em]

\quad \quad \quad \quad \delta =

\left \{
\begin{array}{cl}
    1 & (i = j) \\
    0 & (i \neq j)
\end{array}
\right .
\\[0.8em]
により、~0~または~1~をとる記号~\delta_{ij}~を定める。 \\
~\delta_{ij}~をクロネッカーのデルタという。

```latex
%%latex
i,j = 1,2 のとき、クロネッカーのデルタ~\delta_{ij}~の値を求める。\\[0.8em]

\quad \quad \quad \quad \delta_{11} = \delta_{22} = 1, \quad \delta_{12} = \delta_{21} = 0
```

i,j = 1,2 のとき、クロネッカーのデルタ~\delta_{ij}~の値を求める。\\[0.8em]

\quad \quad \quad \quad \delta_{11} = \delta_{22} = 1, \quad \delta_{12} = \delta_{21} = 0

```latex
%%latex
クロネッカーのデルタを用いると、~n~次単位行列~E_n~は~E_n = (\delta_{ij})_{n \times n}~ と表すことができる。\\
例えば、~2~次単位行列は\\[0.8em]

\quad \quad \quad \quad E_2 =

\begin{pmatrix}
  \delta_{11} & \delta_{12} \\
  \delta_{21} & \delta_{22}
\end{pmatrix} =

\begin{pmatrix}
  1 & 0 \\
  0 & 1
\end{pmatrix} \\[0.8em]

となる。
```

クロネッカーのデルタを用いると、~n~次単位行列~E_n~は~E_n = (\delta_{ij})_{n \times n}~ と表すことができる。\\
例えば、~2~次単位行列は\\[0.8em]

\quad \quad \quad \quad E_2 =

\begin{pmatrix}
  \delta_{11} & \delta_{12} \\
  \delta_{21} & \delta_{22}
\end{pmatrix} =

\begin{pmatrix}
  1 & 0 \\
  0 & 1
\end{pmatrix} \\[0.8em]

となる。

```latex
%%latex
転置行列\\[0.8em]

m \times n~ 行列~A~の行と列を入れ替えて得られる~n \times m~ 行列を~{}^t A, A^t~ または~{}^T A~ などと書き、\\
~A~の転置行列という。 すなわち、\\[0.8em]
\quad \quad \quad \quad
A =
\begin{pmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{pmatrix} \\[0.8em]

のとき、\\[0.8em]

\quad \quad \quad \quad
{}^t A =
\begin{pmatrix}
    a_{11} & a_{21} & \ldots & a_{m1} \\
    a_{12} & a_{22} & \ldots & a_{m2} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{1n} & a_{2m} & \ldots & a_{mn}
\end{pmatrix} \\[0.8em]

である。定義より、\\[0.8em]

\quad \quad \quad \quad
{}^t({}^t A) = A \\[0.8em]

が成り立つ。 なお転置行列を作ることを、転置を取る、ともいう。
```

転置行列\\[0.8em]

m \times n~ 行列~A~の行と列を入れ替えて得られる~n \times m~ 行列を~{}^t A, A^t~ または~{}^T A~ などと書き、\\
~A~の転置行列という。 すなわち、\\[0.8em]
\quad \quad \quad \quad
A =
\begin{pmatrix}
    a_{11} & a_{12} & \ldots & a_{1n} \\
    a_{21} & a_{22} & \ldots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{m1} & a_{m2} & \ldots & a_{mn}
\end{pmatrix} \\[0.8em]

のとき、\\[0.8em]

\quad \quad \quad \quad
{}^t A =
\begin{pmatrix}
    a_{11} & a_{21} & \ldots & a_{m1} \\
    a_{12} & a_{22} & \ldots & a_{m2} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{1n} & a_{2m} & \ldots & a_{mn}
\end{pmatrix} \\[0.8em]

である。定義より、\\[0.8em]

\quad \quad \quad \quad
{}^t({}^t A) = A \\[0.8em]

が成り立つ。 なお転置行列を作ることを、転置を取る、ともいう。

```latex
%%latex
例題 \\[0.8em]

2 \times 3~ 行列~
\begin{pmatrix}
    1 & 2 & 3 \\
    4 & 5 & 6
\end{pmatrix}~
の転置行列を求める。 \\[0.8em]
```

例題 \\[0.8em]

2 \times 3~ 行列~
\begin{pmatrix}
    1 & 2 & 3 \\
    4 & 5 & 6
\end{pmatrix}~
の転置行列を求める。

```
from sympy import *
init_printing()
A = Matrix([[1,2,3],[4,5,6]])
display(A)
print()
display (A.T)
```

$\displaystyle \left[\begin{matrix}1 & 2 & 3\\4 & 5 & 6\end{matrix}\right]$

$\displaystyle \left[\begin{matrix}1 & 4\\2 & 5\\3 & 6\end{matrix}\right]$

```latex
%%latex
対称行列\\[0.8em]

~n~次の正方行列の転置行列は再び~n~次の正方行列となる。よって、次のような正方行列を考えることができる。\\[0.8em]

定義 \quad {}^t A = A が成り立つ正方行列~A~を対称行列という。\\[0.8em]

例 \\[0.8em]

\quad \quad \quad \quad
(1), \quad
\begin{pmatrix}
  1 & 2 \\
  2 & 3
\end{pmatrix} , \quad
\begin{pmatrix}
  1 & 2 & 3\\
  2 & 4 & 5\\
  3 & 5 & 6
\end{pmatrix}
```

対称行列\\[0.8em]

~n~次の正方行列の転置行列は再び~n~次の正方行列となる。よって、次のような正方行列を考えることができる。\\[0.8em]

定義 \quad {}^t A = A が成り立つ正方行列~A~を対称行列という。\\[0.8em]

例 \\[0.8em]

\quad \quad \quad \quad
(1), \quad
\begin{pmatrix}
  1 & 2 \\
  2 & 3
\end{pmatrix} , \quad
\begin{pmatrix}
  1 & 2 & 3\\
  2 & 4 & 5\\
  3 & 5 & 6
\end{pmatrix}

```latex
%%latex
定義より、対称行列について、次の定理が成り立つ。\\[0.8em]

A=(a_{ij})~を~n~次の正方行列とする。 ~A~が対称行列であるための必要十分条件は\\[0.8em]

\quad \quad \quad \quad
a_{ij} = a_{ji} \quad (i,j = 1,2,\cdots, n)\\[0.8em]

である。
```

定義より、対称行列について、次の定理が成り立つ。\\[0.8em]

A=(a_{ij})~を~n~次の正方行列とする。 ~A~が対称行列であるための必要十分条件は\\[0.8em]

\quad \quad \quad \quad
a_{ij} = a_{ji} \quad (i,j = 1,2,\cdots, n)\\[0.8em]

である。

```latex
%%latex
ベクトルと名のつくものを簡単に書くときは太文字、ボールド体を用いる。\\[0.8em]

\quad \quad \quad \quad \mathbf{0,a,b,c,d,e,n,v,x,y,z,R,C}\\[0.8em]

手書きするときは、文字の左側を~2~重にする。\\[0.8em]

\quad \quad \quad \quad \mathbb{0,a,b,c,d,e,n,v,x,y,z,R,C}\\[0.8em]

大文字しか~2~重にならない。
```

ベクトルと名のつくものを簡単に書くときは太文字、ボールド体を用いる。\\[0.8em]

\quad \quad \quad \quad \mathbf{0,a,b,c,d,e,n,v,x,y,z,R,C}\\[0.8em]

手書きするときは、文字の左側を~2~重にする。\\[0.8em]

\quad \quad \quad \quad \mathbb{0,a,b,c,d,e,n,v,x,y,z,R,C}\\[0.8em]

大文字しか~2~重にならない。 => いまの考えテキストが~2~重文字のとき、テキスト重視の考え方から\\
~\mathbb~

```latex
%%latex
問題\\[0.8em]

~3~次の正方行列~A~を~A =
\begin{pmatrix}
    a_{11} & a_{12} & a_{13} \\
    a_{21} & a_{22} & a_{23} \\
    a_{31} & a_{32} & a_{33}
\end{pmatrix}~と表す。

1.~A~の対角成分を答える。
2.~A~が対角行列、スカラー行列、上三角行列、下三角行列になるとき、~A~をそれぞれ具体的に示す。
```

問題\\[0.8em]

~3~次の正方行列~A~を~A =
\begin{pmatrix}
    a_{11} & a_{12} & a_{13} \\
    a_{21} & a_{22} & a_{23} \\
    a_{31} & a_{32} & a_{33}
\end{pmatrix}

```latex
%%latex
1.~a_{11},a_{12},a_{13} \\[0.8em]
2.~
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    0 & a_{22} & 0 \\
    0 & 0 & a_{33}
\end{pmatrix}, \quad
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    0 & a_{11} & 0 \\
    0 & 0 & a_{11}
\end{pmatrix}, \quad
\begin{pmatrix}
    a_{11} & a_{12} & a_{13} \\
    0 & a_{22} & a_{23} \\
    0 & 0 & a_{33}
\end{pmatrix}~, \quad
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    a_{21} & a_{22} & 0 \\
    a_{31} & a_{32} & a_{33}
\end{pmatrix}
```

1.~a_{11},a_{12},a_{13} \\[0.8em]
2.~
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    0 & a_{22} & 0 \\
    0 & 0 & a_{33}
\end{pmatrix}, \quad
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    0 & a_{11} & 0 \\
    0 & 0 & a_{11}
\end{pmatrix}, \quad
\begin{pmatrix}
    a_{11} & a_{12} & a_{13} \\
    0 & a_{22} & a_{23} \\
    0 & 0 & a_{33}
\end{pmatrix}~, \quad
\begin{pmatrix}
    a_{11} & 0 & 0 \\
    a_{21} & a_{22} & 0 \\
    a_{31} & a_{32} & a_{33}
\end{pmatrix}

```latex
%%latex
問題 \\[0.8em]

クロネッカーのデルタ~\delta_{ij} \quad (i,j = 1,2, \cdots, n)~ について、~i,j=1,2,3~のとき、~\delta_{ij}~の値を求める。 \\[0.8em]

解答 \\[0.8em]

\delta_{11}=\delta_{22}=\delta_{33}=1, \quad

\delta_{12}=\delta_{13}=\delta_{21}=\delta_{23}=\delta_{31}=\delta_{32}=0

```

問題 \\[0.8em]

クロネッカーのデルタ~\delta_{ij} \quad (i,j = 1,2, \cdots, n)~ について、~i,j=1,2,3~のとき、~\delta_{ij}~の値を求める。 \\[0.8em]

解答 \\[0.8em]

\delta_{11}=\delta_{22}=\delta_{33}=1, \quad

\delta_{12}=\delta_{13}=\delta_{21}=\delta_{23}=\delta_{31}=\delta_{32}=0

```latex
%%latex
問題 \\[0.8em]

~3 \times 2~行列~
\begin{pmatrix}
    5 & 4 \\
    3 & 2 \\
    1 & 0
\end{pmatrix}~の転置行列を求める。
```

問題 \\[0.8em]

~3 \times 2~行列~
\begin{pmatrix}
    5 & 4 \\
    3 & 2 \\
    1 & 0
\end{pmatrix}

```
from sympy import *
from IPython.display import Markdown
init_printing()
display(Markdown("解答"))
print()
display(Matrix([[5,4],[3,2],[1,0]]))
print()
display(Matrix([[5,4],[3,2],[1,0]]).T)
```

解答

$\displaystyle \left[\begin{matrix}5 & 4\\3 & 2\\1 & 0\end{matrix}\right]$

$\displaystyle \left[\begin{matrix}5 & 3 & 1\\4 & 2 & 0\end{matrix}\right]$

```latex
%%latex
問題 \\[0.8em]
\begin{pmatrix}
    a^2+b^2 & ab + bc \\
    ab + bc & b^2 + c^2
\end{pmatrix}
=
\begin{pmatrix}
    1 & 0 \\
    0 &  4
\end{pmatrix}
```

問題 \\[0.8em]
\begin{pmatrix}
    a^2+b^2 & ab + bc \\
    ab + bc & b^2 + c^2
\end{pmatrix}
=
\begin{pmatrix}
    1 & 0 \\
    0 &  4
\end{pmatrix}

```
from sympy import *
from IPython.display import Markdown
init_printing()
display(Markdown("問題"))
a,b,c = symbols('a,b,c')
A = Matrix([[a**2+b**2, a*b+b*c],[a*b+b*c, b**2+c**2]])
display(A)
B = Matrix([[1,0],[0,4]])
display(B)
display(Eq(A,B))
display(Markdown("解答"))
display(solve((A-B),a,b,c))
display(solve((A-B)))
```

```
from sympy import *
from IPython.display import Markdown
init_printing()
display(Markdown("問題"))
a,b,c = symbols('a,b,c')
A = Matrix([[a**2+b**2, a*b+b*c],[a*b+b*c, b**2+c**2],[]])
display(A)
```

問題

$\displaystyle \left[\begin{matrix}a^{2} + b^{2} & a b + b c\\a b + b c & b^{2} + c^{2}\end{matrix}\right]$

# いまここ p.10


```python
from sympy import *
from math import *
import sys

from lib_rungekutta import *
```

# Phương pháp Runge - Kutta hiện giải bài toán Cauchy cho phương trình vi phân thường

## 1. Nhập dữ liệu

Nhập biểu thức $ y' = f(x, y)$. Lưu ý PT phải đặt trong dấu nháy kép

```python
expr = "x"
```

Nhập giá trị ban đầu $x_{0}$ và $y(x_{0})$

```python
x_0 = 0
y_0 = 0
```

Nhập khoảng cách giữa 2 điểm liên tiếp $h$

```python
h = 0.1
```

Nhập số mốc của lưới điểm $n$

```python
n = 150
```

Nhập số nấc của Runge-Kutta. Các giá trị bao gồm 1, 2, 3, 4 hoặc kiểu hệ số của Heun

```python
s = 4
```

## 2. Giải PTVP và in ra kết quả
Cấp chính xác:

```python
precision_x   = 3;
precision_y   = 7;
precision_eps = 12;
```

Giải và in kết quả:

```python
s  = s if(s == "heun" or s == "Heun") else int(s);
uu = rungekutta_oop(expr, x_0, y_0, h, n, s);
g  = (uu.Solve());
```

```python
if(s == "heun" or s == "Heun"):
#{
    print(f"Phương pháp Runge-Kutta 3 nấc với cấu hình hệ số kiểu Heun hoàn tất với lưới điểm sau, sai số toàn cục O(h^3) = {round(h**3, precision_eps)}:");
#}
else:
#{
    print(f"Phương pháp Runge-Kutta {s} nấc với hoàn tất với lưới điểm sau, sai số toàn cục O(h^{s}) = {round(h**s, precision_eps)}:");
#}
for x in g: print(round(x[0], precision_x), round(x[1], precision_y));
```

    Phương pháp Runge-Kutta 4 nấc với hoàn tất với lưới điểm sau, sai số toàn cục O(h^4) = 0.0001:
    0 1
    0.1 1.005
    0.2 1.02
    0.3 1.045
    0.4 1.08
    0.5 1.125
    0.6 1.18
    0.7 1.245
    0.8 1.32
    0.9 1.405
    1.0 1.5
    1.1 1.605
    1.2 1.72
    1.3 1.845
    1.4 1.98
    1.5 2.125
    1.6 2.28
    1.7 2.445
    1.8 2.62
    1.9 2.805
    2.0 3.0
    2.1 3.205
    2.2 3.42
    2.3 3.645
    2.4 3.88
    2.5 4.125
    2.6 4.38
    2.7 4.645
    2.8 4.92
    2.9 5.205
    3.0 5.5
    3.1 5.805
    3.2 6.12
    3.3 6.445
    3.4 6.78
    3.5 7.125
    3.6 7.48
    3.7 7.845
    3.8 8.22
    3.9 8.605
    4.0 9.0
    4.1 9.405
    4.2 9.82
    4.3 10.245
    4.4 10.68
    4.5 11.125
    4.6 11.58
    4.7 12.045
    4.8 12.52
    4.9 13.005
    5.0 13.5
    5.1 14.005
    5.2 14.52
    5.3 15.045
    5.4 15.58
    5.5 16.125
    5.6 16.68
    5.7 17.245
    5.8 17.82
    5.9 18.405
    6.0 19.0
    6.1 19.605
    6.2 20.22
    6.3 20.845
    6.4 21.48
    6.5 22.125
    6.6 22.78
    6.7 23.445
    6.8 24.12
    6.9 24.805
    7.0 25.5
    7.1 26.205
    7.2 26.92
    7.3 27.645
    7.4 28.38
    7.5 29.125
    7.6 29.88
    7.7 30.645
    7.8 31.42
    7.9 32.205
    8.0 33.0
    8.1 33.805
    8.2 34.62
    8.3 35.445
    8.4 36.28
    8.5 37.125
    8.6 37.98
    8.7 38.845
    8.8 39.72
    8.9 40.605
    9.0 41.5
    9.1 42.405
    9.2 43.32
    9.3 44.245
    9.4 45.18
    9.5 46.125
    9.6 47.08
    9.7 48.045
    9.8 49.02
    9.9 50.005
    10.0 51.0
    10.1 52.005
    10.2 53.02
    10.3 54.045
    10.4 55.08
    10.5 56.125
    10.6 57.18
    10.7 58.245
    10.8 59.32
    10.9 60.405
    11.0 61.5
    11.1 62.605
    11.2 63.72
    11.3 64.845
    11.4 65.98
    11.5 67.125
    11.6 68.28
    11.7 69.445
    11.8 70.62
    11.9 71.805
    12.0 73.0
    12.1 74.205
    12.2 75.42
    12.3 76.645
    12.4 77.88
    12.5 79.125
    12.6 80.38
    12.7 81.645
    12.8 82.92
    12.9 84.205
    13.0 85.5
    13.1 86.805
    13.2 88.12
    13.3 89.445
    13.4 90.78
    13.5 92.125
    13.6 93.48
    13.7 94.845
    13.8 96.22
    13.9 97.605
    14.0 99.0
    14.1 100.405
    14.2 101.82
    14.3 103.245
    14.4 104.68
    14.5 106.125
    14.6 107.58
    14.7 109.045
    14.8 110.52
    14.9 112.005
    15.0 113.5

Đồ thị:

```python
uu.getPlot()
```

## 3. Nghiệm kiểm chứng
Nhập PT nghiệm kiểm chứng hoặc "NONE" nếu không có

```python
sol_expr = "NONE"
```

```python
if(sol_expr != "NONE"):
#{
    sol = lambdify(symbols("x"), sympify(sol_expr), "math");
    print("So sánh với giá trị đúng của nghiệm:");
    for x in g: print(f"y*({round(x[0], precision_x)}) = {round(x[1], precision_y)}, so sánh với giá trị chuẩn y({round(x[0], precision_x)}) = {round(sol(x[0]), precision_y)}, sai số {round(abs(sol(x[0]) - x[1]), precision_eps)}");
#}
```

Topic 5 - Solving Differential Equations/28.Runge_Kutta/main_rungekutta.ipynb


```python
import torch
import torch.nn as nn
import torch.distributions as td
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
```

## Environment
Here, we are going to use a simple environment to test our model. Suppose we control a particle (represented by white rectangular) that moves in a 2D plane. We can directly control the velocity of the particle along $x$ and $y$ axis. The task is to control the particle moving towards a goal position (represented by red rectangular) at the center of the plane (see video below).

Now, assume we can not directly observe the position of the particle and the goal, but we can access to some visual observation that is represented as images (see figure below).

Now we are going to learn a state space model of this environment. Given the learnt model, we will be able to forward simulate trajectoris in latent space and plan some actions that minimize the distance to the goal position.

```python
class ParticleEnv:
    def __init__(self):
        self.state = np.array([0, 0])
        self.goal_state = np.array([0.5, 0.5])

        self.action_scale = 0.025
        self.step_count = 0

    def reset(self):
        self.step_count = 0
        self.state = 0.8 * (np.random.rand(2) - 0.5) + 0.5

        reward = -np.sqrt(np.square(self.state - self.goal_state).sum())
        state_image = self.generate_image()

        # potential_goal_states = [[0.0, 0.0], [0.9, 0.9], [0.9, 0.0], [0.0, 0.9]]
        # self.goal_state = np.array(potential_goal_states[random.choice([0, 1, 2, 3])])
        return self.state.copy(), state_image, reward

    def step(self, action):
        self.state += action * self.action_scale
        self.state = np.clip(self.state, a_min=0.0, a_max=1.0)

        reward = -np.sqrt(np.square(self.state - self.goal_state).sum())

        if np.sqrt(np.square(self.state - self.goal_state).sum()) < 0.01:
            done = True
        else:
            done = False

        state_image = self.generate_image()
        self.step_count += 1

        if self.step_count > 10:
            done = True
        return self.state.copy(), state_image, reward, done

    def generate_image(self):
        resolution = 32
        radius = 3
        image_canvas = np.zeros(shape=[3, resolution, resolution])

        pixel_x = int(self.state[0].item() * (resolution - 1))
        pixel_y = int(self.state[1].item() * (resolution - 1))

        for i in range(radius):
            for j in range(radius):
                image_canvas[:, np.clip(pixel_x + i, 0, resolution - 1), np.clip(pixel_y + j, 0, resolution - 1)] = 1.0
                image_canvas[:, np.clip(pixel_x - i, 0, resolution - 1), np.clip(pixel_y - j, 0, resolution - 1)] = 1.0
                image_canvas[:, np.clip(pixel_x + i, 0, resolution - 1), np.clip(pixel_y - j, 0, resolution - 1)] = 1.0
                image_canvas[:, np.clip(pixel_x - i, 0, resolution - 1), np.clip(pixel_y + j, 0, resolution - 1)] = 1.0

        pixel_x = int(self.goal_state[0].item() * (resolution - 1))
        pixel_y = int(self.goal_state[1].item() * (resolution - 1))

        for i in range(radius):
            for j in range(radius):
                image_canvas[0, np.clip(pixel_x + i, 0, resolution - 1), np.clip(pixel_y + j, 0, resolution - 1)] = 1.0
                image_canvas[0, np.clip(pixel_x - i, 0, resolution - 1), np.clip(pixel_y - j, 0, resolution - 1)] = 1.0
                image_canvas[0, np.clip(pixel_x + i, 0, resolution - 1), np.clip(pixel_y - j, 0, resolution - 1)] = 1.0
                image_canvas[0, np.clip(pixel_x - i, 0, resolution - 1), np.clip(pixel_y + j, 0, resolution - 1)] = 1.0

        return image_canvas
```

Here, we define a ReplayBuffer to collect data when interacting with the environment. The ReplayBuffer will record the visual observations (images), actions, reward (negative distance to the goal) and terminal flag at each time step. We also define some neural network template that will be used later.

```python
class ReplayBuffer(object):
    """Buffer to store and replay environment transitions."""

    def __init__(self, obs_shape, action_shape, reward_shape, capacity, batch_size, length, device='cpu'):
        self.capacity = capacity
        self.batch_size = batch_size
        self.length = length
        self.device = device
        # Initialize all the buffers
        self.obs_buffer = np.empty(shape=(capacity, *obs_shape), dtype=np.float32)
        self.action_buffer = np.empty(shape=(capacity, *action_shape), dtype=np.float32)
        self.reward_buffer = np.empty(shape=(capacity, *reward_shape), dtype=np.float32)
        self.done_buffer = np.empty(shape=(capacity, *reward_shape), dtype=np.float32)
        self.idx = 0

    def add(self, obs, action, reward, done):
        if self.idx < self.capacity:
            self.obs_buffer[self.idx] = obs
            self.action_buffer[self.idx] = action
            self.reward_buffer[self.idx] = reward
            self.done_buffer[self.idx] = done
            self.idx += 1
        else:
            self.obs_buffer = self.obs_buffer[1:]
            self.obs_buffer = np.append(self.obs_buffer,
                                        obs.reshape((1, obs.shape[0], obs.shape[1], obs.shape[2])),
                                        axis=0)
            self.action_buffer = self.action_buffer[1:]
            self.action_buffer = np.append(self.action_buffer,
                                           action.reshape((1, action.shape[0])),
                                           axis=0)
            self.reward_buffer = self.reward_buffer[1:]
            self.reward_buffer = np.append(self.reward_buffer,
                                           reward.reshape((1, 1)),
                                           axis=0)
            self.done_buffer = self.done_buffer[1:]
            self.done_buffer = np.append(self.done_buffer,
                                         done.reshape((1, done.shape[0])),
                                         axis=0)

    def sample(self):
        idxs = np.random.randint(
            0, self.capacity - self.length + 1 if self.idx == self.capacity else self.idx - self.length + 1,
            size=self.batch_size)
        obses = torch.as_tensor(self.obs_buffer[idxs], device=self.device).unsqueeze(1).float()
        actions = torch.as_tensor(self.action_buffer[idxs], device=self.device).unsqueeze(1).float()
        rewards = torch.as_tensor(self.reward_buffer[idxs], device=self.device).unsqueeze(1).float()
        dones = torch.as_tensor(self.done_buffer[idxs], device=self.device).unsqueeze(1).float()

        for i in range(1, self.length):
            next_obses = torch.as_tensor(self.obs_buffer[idxs + i], device=self.device).unsqueeze(1).float()
            next_actions = torch.as_tensor(self.action_buffer[idxs + i], device=self.device).unsqueeze(1).float()
            next_rewards = torch.as_tensor(self.reward_buffer[idxs + i], device=self.device).unsqueeze(1).float()
            next_dones = torch.as_tensor(self.done_buffer[idxs + i], device=self.device).unsqueeze(1).float()
            obses = torch.cat((obses, next_obses), 1)
            actions = torch.cat((actions, next_actions), 1)
            rewards = torch.cat((rewards, next_rewards), 1)
            dones = torch.cat((dones, next_dones), 1)

        return obses, actions, rewards, dones

class CNNDenseModel(nn.Module):
    def __init__(self, embed_dim: int, layers: int, h_dim: int,
                 activation=nn.ReLU, min=1e-4, max=10.0):
        super().__init__()
        self._embed_size = embed_dim
        self._layers = layers
        self._hidden_size = h_dim
        self.activation = activation
        self.model = self.build_model()
        self.soft_plus = nn.Softplus()
        self._min = min
        self._max = max
        self.conv_channels = 4

        self.conv_in = nn.Sequential(torch.nn.Conv2d(in_channels=3,
                                                     out_channels=self.conv_channels,
                                                     kernel_size=3,
                                                     stride=3))
        self.fc_out = self.build_model()

    def build_model(self):
        model = [nn.Linear(400, self._hidden_size)]
        model += [self.activation()]
        for i in range(self._layers - 1):
            model += [nn.Linear(self._hidden_size, self._hidden_size)]
            model += [self.activation()]
        model += [nn.Linear(self._hidden_size, self._embed_size)]
        return nn.Sequential(*model)

    def forward(self, obs_visual):
        x_visual = self.conv_in(obs_visual)
        x_visual = x_visual.contiguous()
        x_visual = x_visual.view(-1, self.conv_channels * 10 * 10)
        x = self.fc_out(x_visual)
        return x

class CNNDecoder(torch.nn.Module):
    def __init__(self, z_dim=10, h_dim=64):
        super().__init__()
        self.conv_channels = 4

        self.fc = nn.Sequential(torch.nn.Linear(z_dim, h_dim),
                                torch.nn.ReLU(),
                                torch.nn.Linear(h_dim, self.conv_channels * 10 * 10))
        self.deconv = nn.Sequential(
            torch.nn.ConvTranspose2d(in_channels=self.conv_channels, out_channels=3, kernel_size=5, stride=3))

    def forward(self, x):
        x = self.fc(x)
        h = x.view(-1, self.conv_channels, 10, 10)
        h = self.deconv(h)  # , output_size=(x.size(0), 3, 28, 28))
        return h

class DenseModelNormal(nn.Module):
    def __init__(self, feature_dim: int, output_shape: tuple, layers: int, h_dim: int, activation=nn.ELU,
                 min=1e-4, max=10.0):
        super().__init__()
        self._output_shape = output_shape
        self._layers = layers
        self._hidden_size = h_dim
        self.activation = activation
        # For adjusting pytorch to tensorflow
        self._feature_size = feature_dim
        # Defining the structure of the NN
        self.model = self.build_model()
        self.soft_plus = nn.Softplus()

        self._min = min
        self._max = max

    def build_model(self):
        model = [nn.Linear(self._feature_size, self._hidden_size)]
        model += [self.activation()]
        for i in range(self._layers - 1):
            model += [nn.Linear(self._hidden_size, self._hidden_size)]
            model += [self.activation()]
        model += [nn.Linear(self._hidden_size, 2 * int(np.prod(self._output_shape)))]
        return nn.Sequential(*model)

    def forward(self, features):
        dist_inputs = self.model(features)
        reshaped_inputs_mean = torch.reshape(dist_inputs[..., :np.prod(self._output_shape)],
                                             features.shape[:-1] + self._output_shape)
        reshaped_inputs_std = torch.reshape(dist_inputs[..., np.prod(self._output_shape):],
                                            features.shape[:-1] + self._output_shape)

        reshaped_inputs_std = torch.clamp(self.soft_plus(reshaped_inputs_std), min=self._min, max=self._max)
        return td.independent.Independent(td.Normal(reshaped_inputs_mean, reshaped_inputs_std), len(self._output_shape))
```

## State Space Model
Here, we define our State Space Model (SSM). Intuitively, the SSM models an agent that is sequentially taking actions in a world and receiving rewards and visual observations. The observation $x_t$ at time $t$ visual observations are generated from the latent state $z_t$. The model assumes Markovian transitions where the next state is conditioned upon the current state and the action $a_t$ taken by the agent. Upon taking an action, the agent receives reward $r_t$. Given the graphical structure in the figure below, the SSM's joint distribution factorizes as:

\begin{align}
    &p_\theta(x_{1:T},r_{1:T},z_{0:T},a_{1:T-1}) = \prod_{t=1}^{T} p_\theta(x_{t}|z_{t})p_\theta(r_{t}|z_{t})p_\theta(z_{t}|z_{t-1},a_{t-1})p_\psi(a_{t-1}|z_{t-1})p(z_0)
\end{align}

Each of the factorized distributions are modelled using nonlinear functions:

* Transitions: $p_\theta(z_{t}|z_{t-1},a_{t-1}) = p(z_{t}| f_\theta(z_{t-1},a_{t-1}))$
* Observations: $p_\theta(x_{t}|z_{t}) = p(x_{t}| d_\theta(z_{t})) $
* Rewards: $p_\theta(r_{t}|z_{t}) = p(r_{t}| r_\theta(z_{t})) $

where $f_\theta$, $d^m_\theta$, $r_\theta$, and $\pi_\psi$ are neural networks parameterized by $\theta$. Given nonlinearity of these factorized distributions, the posterior distribution $p(z_{1:T}|x_{1:T},a_{1:T-1})$ is intractable. Thus, we approximate it by

$$p(z_{1:T}|x_{1:T},a_{1:T-1})\approx\hat{q}_{\phi}(z_{1:T}|x_{1:T},a_{1:T-1})=\prod_{t=2}^T q_\phi(z_{t}|g_\phi(x_{t}, z_{t-1}, a_{t-1}))p(z_1)$$

where $q_\phi$ is modeled as a Gaussian distribution and $g_\theta(x_{t}, z_{t-1}, a_{t-1})$ is a neural network parameterized by $\phi$, which is typically called inference network.

Given all these distributions and trajectories of the form $\tau = \left\{(x_{t}, a_{t}, r_{t})\right\}_{t=1}^{T}$ that samples from the ReplayBuffer, we seek to learn the parameters $\theta$ and $\phi$. Because maximum likelihood estimation is intractable in this setting, we optimize the evidence lower bound (ELBO) under the data distribution $p_d$ using a variational distribution $q$ over the latent state variables $z_t$

\begin{align}
\mathbb{E}_{p_d}[\mathrm{ELBO}] \leq \mathbb{E}_{p_d}[\log p_\theta(x_{1:T},r_{1:T}|a_{1:T-1})]
\end{align}

where
\begin{align}
     \mathrm{ELBO} = & \sum_{t=1}^{T}\Big(\displaystyle  \mathop{\mathbb{E}}_{\hat{q}_{\phi}(z_{t})}\left[\log p_{\theta}(x_{t}|z_{t})\right] +\displaystyle\mathop{\mathbb{E}}_{\hat{q}_{\phi}(z_{t})}\left[\log p_{\theta}(r_{t}|z_{t})\right]  \nonumber\\
    &- \displaystyle\mathop{\mathbb{E}}_{\hat{q}_{\phi}(z_{t-1})}\left[\mathrm{KL}\left[\hat{q}_{\phi}(z_{t}) \| p_{\theta}(z_{t}|z_{t-1},a_{t-1})\right]\right]\Big)
\end{align}

```python
class SequentialVAE(nn.Module):
    def __init__(self, z_dim=8, action_dim=2, embed_dim=32):
        super().__init__()
        self.image_embed_net = CNNDenseModel(embed_dim=embed_dim, h_dim=64, layers=2)
        self.inference_net = DenseModelNormal(feature_dim=embed_dim + z_dim + action_dim,
                                              output_shape=(z_dim,),
                                              layers=2,
                                              h_dim=32)

        self.transition_net = DenseModelNormal(feature_dim=z_dim + action_dim,
                                               output_shape=(z_dim,),
                                               layers=2,
                                               h_dim=32)

        self.emission_net = CNNDecoder(z_dim=z_dim, h_dim=32)
        self.reward_net = DenseModelNormal(feature_dim=z_dim,
                                           output_shape=(1,),
                                           layers=2,
                                           h_dim=32)

        self.z_dim = z_dim
        self.action_dim = action_dim

        self.optimizer = torch.optim.Adam(params=self.parameters(), lr=0.002)

        self.anneal_factor = 2.0

    def get_z_post_dist(self, obs, z=None, action=None):

        obs_embed = self.image_embed_net(obs)

        if len(obs.shape) == 3:
            obs_embed = obs_embed[0]

        if z is None:
            z = torch.zeros(obs_embed.shape[:-1] + (self.z_dim,))

        if action is None:
            action = torch.zeros(obs_embed.shape[:-1] + (self.action_dim,))
        z_post_dist = self.inference_net(torch.cat([obs_embed, z, action], dim=-1))
        return z_post_dist

    def cross_entropy_planning(self, z):
        horizon = 4
        sample_size = 100
        discount_factor = 0.8

        action_samples = torch.randint(low=-1, high=2, size=(sample_size, horizon, self.action_dim))

        accumulated_reward = torch.zeros((1, 1))
        z_temp = z.clone().unsqueeze(0).repeat(sample_size, 1)
        for t in range(horizon):
            z_temp_dist = self.transition_net(torch.cat([z_temp, action_samples[:, t]], dim=-1))
            z_temp = z_temp_dist.mean

            reward = self.reward_net(z_temp).mean
            accumulated_reward = accumulated_reward + reward * (discount_factor ** t)

        opti_id = torch.argmax(accumulated_reward, dim=0)
        opti_action_sample = action_samples[opti_id.item()]
        return opti_action_sample[0]

    def learn(self, replay_buffer, episode_count):
        max_epoch = 200
        for i in range(max_epoch):
            obs, action, reward, done = replay_buffer.sample()

            batch_t = obs.shape[1]

            # get latent_state
            z_post_rsample = [[]] * batch_t
            z_post_mean = [[]] * batch_t
            z_post_std = [[]] * batch_t

            for t in range(batch_t):
                if t == 0:
                    z_post_dist = self.get_z_post_dist(obs[:, t])
                else:
                    z_post_dist = self.get_z_post_dist(obs[:, t], z_post_rsample[t - 1], action[:, t - 1])
                z_post_rsample[t] = z_post_dist.rsample()
                z_post_mean[t] = z_post_dist.mean
                z_post_std[t] = z_post_dist.stddev
            z_post_rsample = torch.stack(z_post_rsample, dim=1)
            z_post_mean = torch.stack(z_post_mean, dim=1)
            z_post_std = torch.stack(z_post_std, dim=1)

            z_trans_mean = [[]] * batch_t
            z_trans_std = [[]] * batch_t
            for t in range(batch_t):
                if t == 0:
                    z_trans_mean[t] = z_post_mean[:, t]
                    z_trans_std[t] = z_post_std[:, t]
                else:
                    z_trans_dist = self.transition_net(torch.cat([z_post_rsample[:, t - 1], action[:, t]], dim=-1))
                    z_trans_mean[t] = z_trans_dist.mean
                    z_trans_std[t] = z_trans_dist.stddev
            z_trans_mean = torch.stack(z_trans_mean, dim=1)
            z_trans_std = torch.stack(z_trans_std, dim=1)

            obs_rec = self.emission_net(z_post_rsample)
            obs_rec_loss = 100 * torch.square(obs_rec - obs.view(-1, 3, 32, 32)).mean()

            reward_dist = self.reward_net(z_post_mean.detach())
            reward_rec_loss = -reward_dist.log_prob(reward).mean()

            kl_loss = (z_trans_std.log() - z_post_std.log()
                       + (z_post_std.pow(2) + (z_trans_mean - z_post_mean).pow(2))
                       / (2 * z_trans_std.pow(2) + 1e-5) - 0.5).mean()

            self.optimizer.zero_grad()
            loss = obs_rec_loss + reward_rec_loss + 1.0 * torch.max(kl_loss, torch.ones(1) * self.anneal_factor)
            loss.backward()
            self.optimizer.step()

        self.anneal_factor = np.clip(self.anneal_factor * 0.9, 0.1, 10.0)

        print(
            f'{episode_count}: obs_loss:{obs_rec_loss.item()}, reward_loss:{reward_rec_loss.item()}, kl_loss:{kl_loss.item()}')
```

## Main Loop
Without loss of generality, we begin at time-step $t-1$ where we have a sample of $z_{t-1}$ and taken a action $a_{t-1}$. At time step $t$, we first recieve an visual observation $x_t$ and reward $r_t$. Then,we pass $z_{t-1}$, $a_{t-1}$ and $x_t$ into the variational distribution $q_\phi(z_t|z_{t-1}, a_{t-1}, x_t)$ and sample a $z_t$ from $q_\phi$. Given the sample $z_t$, we optimize the following objective:

\begin{align}
\mathrm{argmax}_{a_{t:t+H-1}} \ \ J = \mathrm{E}_{p(z_{t+1:t+H}\ \ \ \ |a_{t:t+H-1}\ \ \ \ ,z_t)}\left[\sum_{k=t+1}^{t+H} {\gamma^tr(z_t)} \right]
\end{align}

where $p(z_{t+1:t+H}|a_{t:t+H-1},z_t)=\prod_{k=t+1}^{t+H}p_\theta(z_{k}|f_\theta(z_{k-1},a_{k-1}))$ We choose action $a_t$ to execute and replan at each time step. After that,environment fowards to the $t+1$.

```python
def main(test=False):
    env = ParticleEnv()
    replay_buffer = ReplayBuffer(obs_shape=(3, 32, 32),
                                 action_shape=(2,),
                                 reward_shape=(1,),
                                 capacity=1000,
                                 batch_size=50,
                                 length=10)

    model = SequentialVAE(z_dim=5, action_dim=2)
    if test:
        model.load_state_dict(torch.load('./model.pt'))

    model.eval()
    _, obs, reward = env.reset()

    z_post_mean = None
    action = None

    max_episode = 100
    episode_count = 0
    episode_data_size = 200

    visualize_freq = 5

    ims = []
    fig, ax = plt.subplots()
    while True:
        avg_reward = 0.0
        for i in range(episode_data_size):
            z_post_mean = model.get_z_post_dist(torch.as_tensor(obs).float(), z_post_mean, action).mean

            action = model.cross_entropy_planning(z_post_mean)
            _, obs, reward, done = env.step(action.detach().numpy())

            avg_reward += reward
            visualize_image = obs.transpose((1, 2, 0))
            if episode_count % visualize_freq == 0:

                im = ax.imshow(visualize_image, animated=True)
                ims.append([im])

            replay_buffer.add(obs, action, reward, np.array([done]))

            if done:
                _, obs, reward = env.reset()
                z_post_mean = None
                action = None

        avg_reward /= episode_data_size
        print(f'avg_reward:{avg_reward}')

        if episode_count % visualize_freq == 0:
            ani = animation.ArtistAnimation(fig, ims, interval=50, blit=True,
                                            repeat_delay=1000)
            writergif = animation.PillowWriter(fps=30)

            if test:
                file_name = f'./test_episode_{episode_count}.gif'
            else:
                file_name = f'./episode_{episode_count}.gif'
            ani.save(file_name, writer=writergif)
            ims.clear()

        if not test:
            # train model using collected data
            model.train()
            model.learn(replay_buffer, episode_count)
            model.eval()
            torch.save(model.state_dict(), './model.pt')

        episode_count += 1
        if episode_count > max_episode:
            break
```

```python
main(False)
```

```python

```

```python

```