# Transform points from pitch to texture and vice-versa
### Matrix P - Projection
The P matrix is a 3x4 matrix that given a 3D point in the world reference frame, it is projected into the 2D image in **texture coordinate** reference frame, i.e:

\begin{align}
\mathbf{pt_{world}} = (x, y, z, 1) \\
\mathbf{pt_{pitch}} = (a, b, c) = \mathbf{P_{texture}} * \mathbf{pt_{world}} \\
\mathbf{pt_{texture}} = (i, j) = (a/c, b/c)
\end{align}

**Texture coordinate range** is [0.0, 1.0]



### Matrix H - Homography
The H matrix is a 3x3 Matrix that transforms points from one plane to another. In our case from pitch to texture and from texture to pitch which is the inverse of the first (pitch to texture homography). **Pitch is the world plane where z=0.** Hence, get matrix **H*pith2texture*** is as simple as discard 3th matrix P column:




### How to convert from pitch (world) to texture coordinate system
\begin{align}
\mathbf{pt_{texture}} = \mathbf{H_{pitch2texture}} * \mathbf{pt_{pitch}}
\end{align}

### How to convert from texture to pitch (world) coordinate system
\begin{align}
\mathbf{pt_{pitch}} = \mathbf{H_{texture2pitch}} * \mathbf{pt_{texture}}
\end{align}

Where,

\begin{align}
\mathbf{H_{texture2pitch}} = \mathbf{H_{pitch2texture}^{-1}}
\end{align}

### How to convert from texture to video image coordinate system

\begin{equation*}
\mathbf{P_{image}} = \begin{vmatrix}
Width & 0 & 0 \\ 0 & Height & 0 \\ 0 & 0 & 1
\end{vmatrix} * \mathbf{P_{texture}}
\end{equation*}

Where **`Width`** and **`Height`** refer to the video frame size. 

### How to transform points from an image A to an image B




```python
# import what we need
import numpy as np
import math
```


```python
def rodrigues(r):
    '''
    Rodrigues formula
    :param r: 1x3 array of rotations about x, y, and z
    :return: the 3x3 rotation matrix
    '''
    def S(n):
        Sn = np.array([
            [0.0, -n[2], n[1]],
            [n[2], 0.0, -n[0]],
            [-n[1], n[0], 0]])
        return Sn
    theta = np.linalg.norm(r)
    if theta > 1e-30:
        n = r/theta
        Sn = S(n)
        R = np.eye(3) + np.sin(theta) * Sn + (1.0 - np.cos(theta)) * np.dot(Sn, Sn)
    else:
        Sr = S(r)
        theta2 = theta ** 2.0
        R = np.eye(3) + (1.0 - theta2 / 6.0)*Sr + (0.5 - theta2 / 24.0) * np.dot(Sr, Sr)
    return np.mat(R)
```


```python
def world_to_texture(pw, P):
    """
    Projects a world point to texture projection plane
    :param pw: world point (x, y, z)
    :param P: projection matrix P
    :return: texture point (i, j)
    """
    pw_h = np.append(pw, 1.0)
    pp = P.dot(pw_h)
    return np.array([pp[0]/pp[2], pp[1]/pp[2]])
```


```python
def homography_pitch_to_texture(P):
    """
    Returns the homografy to transform points from the pitch (world Z=0) to texture image
    :param matrixP: Full matrix P
    :return: homography_pitch_to_texture matrix
    """
    # delete the 3th column, the z component
    return np.delete(P, 2, axis=1) 
```


```python
def pitch_to_texture(pitch, P):
    """
    Project a point from pitch plane to texture plane
    :param pitch: pitch point (x, y)
    :param P: projection matrix P
    :return: texture point (i, j)
    """
    pitch_h = np.append(pitch, 1.0)
    H = homography_pitch_to_texture(P)
    transformed = H.dot(pitch_h)
    #print(projected)
    return np.array([transformed[0]/transformed[2], transformed[1]/transformed[2]])
    
```


```python
def texture_to_pitch(texture, P):
    """
    Project a point from texture plane to pitch plane
    :param texture: texture point (i, j)
    :param P: projection matrix P
    :return: pitch point (x, y)
    """
    texture_h = np.append(texture, 1.0)
    H = homography_pitch_to_texture(P)
    Hinv = np.linalg.inv(H)
    transformed = Hinv.dot(texture_h)
    return np.array([transformed[0]/transformed[2], transformed[1]/transformed[2]])
```


```python
def texture_to_image(texture, width, height):
    """
    Converts a texture coordinate to image coordinate system
    """
    return  np.array([int(float(width)*texture[0]), int(float(height)*texture[1])])

def image_to_texture(image, width, height):
    """
    Converts a image coordinate to texture coordinate system
    """
    return  np.array([float(image[0])/float(width), float(image[1])/float(height)])
```

# Projection matrix P from an individual camera
The funciton `camera_full_projection_matrix` is only valid for individual cameras. 

## How to get camera parameters: *aspect_ratio, zoom, skew, pan, tilt, roll, Tx, Ty, Tz* 
You can find camera parameters **`aspect_ratio, zoom, skew, pan, tilt, roll, Tx, Ty, Tz`** in `C:\AutomaticTV\data\cameras\{id}.xml`. In XML `<modelcalibration>` tag, for instance:

```
<modelsCalibration>
    <modelCalibration computed="true" sportFieldName="Football11">
      <Zoom>1.2124214</Zoom>
      <AspectRatio>1.7777778</AspectRatio>
      <Skew>0</Skew>
      <Pan>-28.826538</Pan>
      <Tilt>110.37401</Tilt>
      <Roll>-10.530287</Roll>
      <Tx>34.07756</Tx>
      <Ty>-3.4855517</Ty>
      <Tz>74.498503</Tz>
    </modelCalibration>
  </modelsCalibration>
```


```python
def camera_full_projection_matrix(aspect_ratio, zoom, skew, pan, tilt, roll, Tx, Ty, Tz):
    """
    Creates projection matrix P from camera model parameters
    :param aspect_ratio: camera aspect ratio = image width / image height
    :param zoom: camera focal
    :param skew:
    :param pan:
    :param tilt:
    :param roll:
    :param Tx:
    :param Ty:
    :param Tz:
    :return: the projection Matrix P
    """
    K = np.array([[zoom, skew, 0.5], [0.0, zoom * aspect_ratio, 0.5], [0.0, 0.0, 1.0]])

    # Rotation matrix
    Rpan = rodrigues(np.array([0.0, 0.0, pan*math.pi/180.0]))
    Rtilt = rodrigues(np.array([tilt*math.pi/180.0, 0.0, 0.0]))
    Rroll = rodrigues(np.array([0.0, 0.0, roll*math.pi/180.0]))
    Mrot = Rroll * Rtilt * Rpan

    # Translation vector
    t = np.array([Tx, Ty, Tz])
    KR = K * Mrot
    Kt = np.dot(K, t)

    # Projection Martix P
    P = np.zeros((3, 4))
    P[:, 0] = KR[:, 0].T
    P[:, 1] = KR[:, 1].T
    P[:, 2] = KR[:, 2].T
    P[:, 3] = Kt
    return P
```

### Individual camera matrix P Testing


```python
# test

# ML
camera_model = {
    "aspect_ratio": 1.7777778,
    "zoom": 1.1596733,
    "skew": 0.0,
    "pan": 0.3709719,
    "tilt": 112.66264,
    "roll": 0.85281113,
    "Tx": 0.32114744,
    "Ty": 0.32114744,
    "Tz": 68.733923}
P = camera_full_projection_matrix(**camera_model)

print(P)
# project world point
x = -10.0
y = 3.0
world = np.array([x, y, 0.0])
print("world: {}".format(world))
texture = world_to_texture(world, P)
print("texture: {}".format(texture))
"""
if (texture > 1.0).any() or (texture < 0.0).any():
    print("point is out the texture limits")
else:
    print("point is in the texture limits")
"""

print()
# transform point from pitch to texture plane
pitch = np.array([x, y])
print("pitch: {}".format(pitch))
texture = pitch_to_texture(pitch, P)
print("texture: {}".format(texture))

print()
# transform point from texture to pitch
print("texture: {}".format(texture))
pitch = texture_to_pitch(texture, P)
print("pitch: {}".format(pitch))


```

    [[  1.16255096e+00   4.60527814e-01  -1.76724528e-01   3.47393876e+01]
     [  2.85291856e-02  -3.33068495e-01  -2.09490260e+00   3.50290524e+01]
     [  5.97472614e-03   9.22770183e-01  -3.85304415e-01   6.87339230e+01]]
    world: [-10.   3.   0.]
    texture: [ 0.34286967  0.47233176]
    
    pitch: [-10.   3.]
    texture: [ 0.34286967  0.47233176]
    
    texture: [ 0.34286967  0.47233176]
    pitch: [-10.  3.]


## Matrix P for Panorama

## How to get Panorama camera parameters: *src_width, src_height, zoom, skew, pan, tilt, roll, Tx, Ty, Tz* 
You can find camera parameters **`zoom, skew, pan, tilt, roll, Tx, Ty, Tz`** in `C:\AutomaticTV\data\virtual_cameras\{id}.xml`. In XML `<CameraModel name="PANORAMA">` tag, for instance:

```
    <CameraModel name="PANORAMA">
      <Width>5760</Width>
      <Height>1080</Height>
      <Zoom>0.45361389</Zoom>
      <AspectRatio>5.3333333</AspectRatio>
      <Skew>0</Skew>
      <Pan>-2.0953152</Pan>
      <Tilt>108.76381</Tilt>
      <Roll>10</Roll>
      <Tx>0.48063888</Tx>
      <Ty>-0.30635475</Ty>
      <Tz>87.349004</Tz>
    </CameraModel>
```
* **src_width** is `5760`
* **src_height** is `1080`

**NOTE: Do not use this `AspectRatio`.**

## How to get Panorama camera *offset_x* and *offset_y*
You can find **`offset_x`** and **`offset_y`** in `C:\AutomaticTV\data\virtual_cameras\{id}.xml` file tag `<PanoramaOffsetX>` and `<PanoramaOffsetY>`
```
    <PanoramaOffsetX>0</PanoramaOffsetX>
    <PanoramaOffsetY>0</PanoramaOffsetY>
```

## How to get Panorama camera *src_width* and *src_height*
It can be found in `C:\AutomaticTV\data\productions\{id}.xml`. In XML `<panorama><realization>` tag, for instance:`
```
    <width>3840</width>
    <height>1080</height>
```

## How to get Panorama camera *aspect_ratio, state_x, state_y, state_zoom*
It can be found in `C:\AutomaticTV\data\productions\{id}.xml`. In XML `<panorama><realization><operators><operator name="panorama"><currentState>` tag, for instance:

```
          <currentState>
            <elem>0.500000</elem>
            <elem>0.500000</elem>
            <elem>1.000000</elem>
            <elem>0.453614</elem>
            <elem>3.555556</elem>
            <elem>0.000000</elem>
            <elem>-2.095315</elem>
            <elem>108.763810</elem>
            <elem>0.000000</elem>
            <elem>0.480639</elem>
            <elem>-0.306355</elem>
            <elem>87.349004</elem>
          </currentState>
```
where:
* ***aspect_ratio*** is the 4th elem `3.555556`
* ***state_x*** is the 1st elem `0.500000`
* ***state_y*** is the 2nd elem `0.500000`
* ***state_zoom*** is the 3rd elem `1.000000`


```python
def camera_full_projection_matrix_with_crop(aspect_ratio, zoom, skew, pan, tilt, roll, Tx, Ty, Tz, crop_tx, crop_ty, crop_zoom):
    """
    Creates projection matrix P from camera model parameters
    :param aspect_ratio: camera aspect ratio = dst_width / dst_height
    :param zoom: camera focal
    :param skew:
    :param pan:
    :param tilt:
    :param roll:
    :param Tx:
    :param Ty:
    :param Tz:
    :param crop_tx:
    :param crop_ty:
    :param crop_zoom:
    :return: the projection Matrix P
    """
    K = np.array([[crop_zoom * zoom, crop_zoom * skew, 0.5 + crop_tx], 
                  [0.0, crop_zoom * zoom * aspect_ratio, 0.5 + crop_ty],
                  [0.0, 0.0, 1.0]])

    # Rotation matrix
    Rpan = rodrigues(np.array([0.0, 0.0, pan*math.pi/180.0]))
    Rtilt = rodrigues(np.array([tilt*math.pi/180.0, 0.0, 0.0]))
    Rroll = rodrigues(np.array([0.0, 0.0, roll*math.pi/180.0]))
    Mrot = Rroll * Rtilt * Rpan

    # Translation vector
    t = np.array([Tx, Ty, Tz])
    KR = K * Mrot
    Kt = np.dot(K, t)

    # Projection Martix P
    P = np.zeros((3, 4))
    P[:, 0] = KR[:, 0].T
    P[:, 1] = KR[:, 1].T
    P[:, 2] = KR[:, 2].T
    P[:, 3] = Kt
    return P

def get_crop_params(src_width, src_height, offset_x, offset_y, dst_width, dst_height, state_x, state_y, state_zoom):
    """
    Computes crop parameters taking in account source and destination dimensions
    : return crop_tx: 
    : return crop_ty:
    : return crop_zoom:
    """
    width_ratio_inv = float(src_width) / float(dst_width)
    height_ratio_inv = float(src_height) / float(dst_height)
    crop_tx = (0.5 - state_x + offset_y) * (state_zoom * width_ratio_inv)
    crop_ty = (0.5 - state_x + offset_y) * (state_zoom * height_ratio_inv)
    crop_zoom = state_zoom * max(width_ratio_inv, height_ratio_inv)
    
    return crop_tx, crop_ty, crop_zoom
```

### Panorama matrix P


```python
camera_model = {
    "aspect_ratio": 3.555556,
    "zoom": 0.33184094,
    "skew": 0.0,
    "pan": -0.10555596,
    "tilt": 109.86491,
    "roll": 0.0,
    "Tx": 0.2886617,
    "Ty": -2.903907,
    "Tz": 68.918928}

crop_params = {
    "src_width": 5760,
    "src_height": 1080,
    "offset_x": 0.0,
    "offset_y": 0.0,
    "dst_width": 3840,
    "dst_height": 1080,
    "state_x": 0.5,
    "state_y": 0.5,
    "state_zoom": 1.0}


# compute crop from params
crop_tx, crop_ty, crop_zoom = get_crop_params(**crop_params)
print("crop: {}, {}, {}".format(crop_tx, crop_ty, crop_zoom))

# compute matrix P
P_proj_Pan = camera_full_projection_matrix_with_crop(**camera_model, crop_tx=crop_tx, crop_ty=crop_ty, crop_zoom=crop_zoom)
print(P_proj_Pan)

```

    crop: 0.0, 0.0, 1.5
    [[  4.96894228e-01   4.71164433e-01  -1.69901810e-01   3.46031487e+01]
     [  2.41603659e-04  -1.31142328e-01  -1.83440982e+00   2.93200755e+01]
     [ -1.73267466e-03   9.40494815e-01  -3.39803620e-01   6.89189280e+01]]


## Master Left matrix P


```python
camera_model = {
    "aspect_ratio": 1.7777778,
    "zoom": 1.2298668,
    "skew": 0.0,
    "pan": -35.048991,
    "tilt": 108.34705,
    "roll": -11.873014,
    "Tx": 38.049879,
    "Ty": -5.1018767,
    "Tz": 56.809674}
P_proj_ML = camera_full_projection_matrix(**camera_model)
```

## Master Center matrix P


```python
# MC
camera_model = {
    "aspect_ratio": 1.7777778,
    "zoom": 1.1596733,
    "skew": 0.0,
    "pan": 0.3709719,
    "tilt": 112.66264,
    "roll": 0.85281113,
    "Tx": 0.32114744,
    "Ty": -6.2029192,
    "Tz": 68.733923}
P_proj_MC = camera_full_projection_matrix(**camera_model)
```

## example on Master videos only


```python
width_master = 1920
height_master = 1080
```


```python
im_master = np.array([[width_master,0,0], [0,height_master,0] ,[0,0,1]])
```


```python
texture = np.array([0.95,0.95])

# Texture on Master Left Camer to pitch
MLtoPitch= texture_to_pitch(texture,P_proj_ML)
print("MLtex_to_pitch: {}".format(MLtoPitch))

# Pitch to texture on Master Center Camera
PitchtoMC = pitch_to_texture(MLtoPitch, P_proj_MC)
print("pitch_to_MCtex: {}".format(PitchtoMC))
print("MCpix: {}".format(texture_to_image(PitchtoMC, 1920, 1080)))

```

    MLtex_to_pitch: [ -9.31987664 -33.7921301 ]
    pitch_to_MCtex: [ 0.22248691  0.86857492]
    MCpix: [427 938]



```python
PImageML = im_master*np.append(texture,1)
print ("Coord on Master Left: ", PImageML[0][0], PImageML[1][1])
```

    Coord on Master Left:  1824.0 1026.0



```python
PImageMC = im_master*np.append(PitchtoMC,1)
print  ("Coord on Master Center: ", PImageMC[0][0], PImageMC[1][1])
```

    Coord on Master Center:  427.174875943 938.060913584



```python

```

## example on Master to Panorama video


```python

```


```python
width_master = 1920
height_master = 1080
im_master = np.array([[width_master,0,0], [0,height_master,0] ,[0,0,1]])
```


```python
width_pan = 3840
height_pan = 1080
im_pan = np.array([[width_pan,0,0], [0,height_pan,0] ,[0,0,1]])
```


```python
texture = np.array([0.95,0.95])

# Texture on Master Left Camer to pitch
MLtoPitch= texture_to_pitch(texture,P_proj_ML)

# pitch to panorama texture
PitchtoPan = pitch_to_texture(MLtoPitch, P_proj_Pan)
```


```python
PImageML = im_master*np.append(texture,1)
print ("Coord on Master Left: ", PImageML[0][0], PImageML[1][1])
```

    Coord on Master Left:  1824.0 1026.0



```python
PImagePan = im_pan*np.append(PitchtoPan,1)
print ("Coord on Master Left: ", PImagePan[0][0], PImageML[1][1])
```

    Coord on Master Left:  1452.18014986 1026.0



```python

```


```python
# texture point in ML
p_ML_tex = np.array([0.95,0.95])
print("p_ML_tex: {}".format(p_ML_tex))
print("p_ML_pix: {}".format(texture_to_image(p_ML_tex, 1920, 1080)))
# Through ML
p_pitch = texture_to_pitch(p_ML_tex, P_proj_ML)
print("p_pitch: {}".format(p_pitch))

# p_pitch to MC
p_MC_tex = pitch_to_texture(p_pitch, P_proj_MC)
print("p_MC_tex: {}".format(p_MC_tex))
print("p_MC_pix: {}".format(texture_to_image(p_MC_tex, 1920, 1080)))

# p_pitch to Panorama
p_Pan_tex = pitch_to_texture(p_pitch, P_proj_Pan)
print("p_Pan_tex: {}".format(p_Pan_tex))
print("p_Pan_pix: {}".format(texture_to_image(p_Pan_tex, 3840, 1080)))

print()
# Panorama to pitch
p_pitch_ = texture_to_pitch(p_Pan_tex, P_proj_Pan)
print("p_pitch_: {}".format(p_pitch_))

print(P_proj_Pan)
```

    p_ML_tex: [ 0.95  0.95]
    p_ML_pix: [1824 1026]
    p_pitch: [ -9.31987664 -33.7921301 ]
    p_MC_tex: [ 0.22248691  0.86857492]
    p_MC_pix: [427 938]
    p_Pan_tex: [ 0.37817191  0.90837128]
    p_Pan_pix: [1452  981]
    
    p_pitch_: [ -9.31987664 -33.7921301 ]
    [[  4.96894228e-01   4.71164433e-01  -1.69901810e-01   3.46031487e+01]
     [  2.41603659e-04  -1.31142328e-01  -1.83440982e+00   2.93200755e+01]
     [ -1.73267466e-03   9.40494815e-01  -3.39803620e-01   6.89189280e+01]]



```python

```

e1e7b8b009190f746739f87a883638a0cccbd0ab
ipynb
Jupyter Notebook
hudl/MatrixP_example.ipynb
marceljanerfont/MatrixP
94fc881e3696f79a1ee6d8ec78535810c26f5b10
MIT
hudl/MatrixP_example.ipynb
marceljanerfont/MatrixP
94fc881e3696f79a1ee6d8ec78535810c26f5b10
MIT
hudl/MatrixP_example.ipynb
marceljanerfont/MatrixP
94fc881e3696f79a1ee6d8ec78535810c26f5b10
MIT
Qwen/Qwen-72B
1. YES
2. YES