

# Basic Python Packages for Science
## The Aeropython’s guide to the Python Galaxy! 


###### Siro Moreno Martín
###### Alejandro Sáez Mollejo

### 0. Introduction

#### Python in the Scientific environment  

##### Principal Python Packages for scientific purposes 

##### Anaconda & conda



http://conda.pydata.org/docs/intro.html

Conda is a package manager application that quickly installs, runs, and updates packages and their dependencies. The conda command is the primary interface for managing installations of various packages. It can query and search the package index and current installation, create new environments, and install and update packages into existing conda environments. 


```python
from IPython.display import HTML
HTML('')
```








##### Main objectives of this workshop 

* Provide you with a first insight into the principal Python tools & libraries used in Science:
    - conda.
    - Jupyter Notebook.
    - NumPy, matplotlib, SciPy
* Provide you with the basic skills to face basic tasks such as:
    - 
* Show other common libraries:
    - Pandas, scikit-learn (some talks & workshops will focus on these packages)
    - SymPy
    - Numba ¿?

### 1. Jupyter Notebook



The Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, machine learning and much more.

It has been widely recognised as a great way to distribute scientific papers, because of the capability to have an integrated format with text and executable code, highly reproducible. Top level investigators around the world are already using it, like the team behind the Gravitational Waves discovery (LIGO), whose analysis was translated to an interactive dowloadable Jupyter notebook. You can see it here: https://github.com/minrk/ligo-binder/blob/master/GW150914_tutorial.ipynb

### 2. Using arrays: NumPy 



#### ndarray object 

| index     | 0     | 1     | 2     | 3     | ...   | n-1   | n  |
| ---------- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| value      | 2.1   | 3.6   | 7.8   | 1.5   | ...   | 5.4   | 6.3 |

* N-dimensional data structure.
* Homogeneously typed.
* Efficient!

A universal function (or ufunc for short) is a function that operates on ndarrays. It is a “vectorized function".


```python
import numpy as np
```


```python
my_list  = list(range(0,100000))
%timeit sum(my_list)
```

    1000 loops, best of 3: 1.54 ms per loop



```python
array = np.arange(0, 100000)
%timeit np.sum(array)
```

    10000 loops, best of 3: 97.8 µs per loop


#### Array creation


```python
one_dim_array = np.array([1, 2, 3, 4])
one_dim_array
```




    array([1, 2, 3, 4])




```python
two_dim_array = np.array([[1, 2, 3],
                                           [4, 5, 6],
                                           [7, 8, 9]])
two_dim_array
```




    array([[1, 2, 3],
           [4, 5, 6],
           [7, 8, 9]])




```python
two_dim_array.size
```




    9




```python
two_dim_array.shape
```




    (3, 3)




```python
two_dim_array.dtype
```




    dtype('int64')




```python
zeros_arr = np.zeros([3, 3])
ones_arr = np.ones([10])
eye_arr = np.eye(5)
```


```python
range_arr = np.arange(15)
range_arr
```




    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14])




```python
range_arr.reshape([3, 5])
```




    array([[ 0,  1,  2,  3,  4],
           [ 5,  6,  7,  8,  9],
           [10, 11, 12, 13, 14]])




```python
np.linspace(0, 10, 21)
```




    array([  0. ,   0.5,   1. ,   1.5,   2. ,   2.5,   3. ,   3.5,   4. ,
             4.5,   5. ,   5.5,   6. ,   6.5,   7. ,   7.5,   8. ,   8.5,
             9. ,   9.5,  10. ])



#### Basic slicing 


```python
one_dim_array[0]
```




    1




```python
two_dim_array[-1, -1]
```




    9



`[start:stop:step]`


```python
my_arr = np.arange(100)
my_arr[0::2]
```




    array([ 0,  2,  4,  6,  8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32,
           34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66,
           68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98])




```python
chess_board = np.zeros([8, 8], dtype=int)

chess_board[0::2, 1::2] = 1
chess_board[1::2, 0::2] = 1

chess_board
```




    array([[0, 1, 0, 1, 0, 1, 0, 1],
           [1, 0, 1, 0, 1, 0, 1, 0],
           [0, 1, 0, 1, 0, 1, 0, 1],
           [1, 0, 1, 0, 1, 0, 1, 0],
           [0, 1, 0, 1, 0, 1, 0, 1],
           [1, 0, 1, 0, 1, 0, 1, 0],
           [0, 1, 0, 1, 0, 1, 0, 1],
           [1, 0, 1, 0, 1, 0, 1, 0]])



### 2. Drawing: Matplotlib




```python
%matplotlib inline
import matplotlib.pyplot as plt
```


```python
plt.matshow(chess_board, cmap=plt.cm.gray)
```

#### Operations & linalg 


```python
# numpy functions
x = np.linspace(1, 10)
y = np.sin(x)
```


```python
plt.plot(x, y)
```


```python
y_2 = (1 + np.log(x)) ** 2
```


```python
# Our first plot
plt.plot(x, y_2, 'r-*')
```


```python
# Creating a 2d array
two_dim_array = np.array([[10, 25, 33],
                                           [40, 25, 16],
                                           [77, 68, 91]])

two_dim_array.T
```




    array([[10, 40, 77],
           [25, 25, 68],
           [33, 16, 91]])




```python
# matrix multiplication
two_dim_array @ two_dim_array
```




    array([[ 3641,  3119,  3733],
           [ 2632,  2713,  3176],
           [10497,  9813, 11910]])




```python
# matrix vector
one_dim_array = np.array([2.5, 3.6, 3.8])

two_dim_array @ one_dim_array
```




    array([ 240.4,  250.8,  783.1])




```python
# inv
np.linalg.inv(two_dim_array)
```




    array([[-0.05372256,  0.00140303,  0.01923512],
           [ 0.10898393,  0.07381761, -0.05250057],
           [-0.03598099, -0.05634759,  0.03394433]])




```python
# eigenvectors & eigenvalues
np.linalg.eig(two_dim_array)
```




    (array([ 133.6946629,  -17.266221 ,    9.5715581]),
     array([[-0.29580975, -0.74274264,  0.0661375 ],
            [-0.24477775,  0.65983255, -0.79576005],
            [-0.92335283,  0.1138173 ,  0.60198985]]))



#### Air quality data 


```python
from IPython.display import HTML
HTML('')
```








##### Loading the data 


```python
# Linux command 
!head ./data/barrio_del_pilar-20160322.csv

# Windows
# !gc log.txt | select -first 10 # head
```

    Estación: Barrio del Pilar;;;;
    Fecha;Hora;CO;NO2;O3
    ;;mg/m³;µg/m³;µg/m³
    22/03/2016;01:00;0.2;14;73
    22/03/2016;02:00;0.2;10;77
    22/03/2016;03:00;0.2;9;75
    22/03/2016;04:00;0.2;3;81
    22/03/2016;05:00;0.2;3;81
    22/03/2016;06:00;0.2;6;79
    22/03/2016;07:00;0.2;24;59



```python
# loading the data
# ./data/barrio_del_pilar-20160322.csv
data1 = np.genfromtxt('./data/barrio_del_pilar-20160322.csv', skip_header=3, delimiter=';', usecols=(2,3,4))
data1
```




    array([[   0.2,   14. ,   73. ],
           [   0.2,   10. ,   77. ],
           [   0.2,    9. ,   75. ],
           [   0.2,    3. ,   81. ],
           [   0.2,    3. ,   81. ],
           [   0.2,    6. ,   79. ],
           [   0.2,   24. ,   59. ],
           [   0.3,   48. ,   37. ],
           [   0.3,   40. ,   43. ],
           [   0.3,   41. ,   44. ],
           [   0.3,   20. ,   68. ],
           [   0.3,   17. ,   74. ],
           [   0.2,   14. ,   84. ],
           [   0.3,   16. ,   88. ],
           [   0.3,   15. ,   94. ],
           [   0.4,   29. ,   81. ],
           [   0.3,   23. ,   82. ],
           [   0.3,   26. ,   81. ],
           [   0.3,   30. ,   75. ],
           [   0.4,   57. ,   39. ],
           [   0.4,   73. ,   17. ],
           [   0.4,   51. ,   42. ],
           [   0.4,   72. ,   16. ],
           [   0.4,   61. ,   28. ],
           [   0.3,   25. ,   62. ],
           [   0.3,   21. ,   64. ],
           [   0.3,   40. ,   39. ],
           [   0.4,   52. ,   19. ],
           [   0.4,   47. ,    8. ],
           [   0.4,   42. ,    8. ],
           [   0.5,   68. ,    8. ],
           [   0.6,   71. ,    9. ],
           [   0.9,   76. ,   10. ],
           [   0.7,   63. ,   29. ],
           [   0.3,   27. ,   66. ],
           [   0.3,   13. ,   88. ],
           [   0.2,   10. ,   92. ],
           [   0.3,   10. ,   98. ],
           [   0.3,   11. ,   99. ],
           [   0.3,   12. ,   99. ],
           [   0.2,   11. ,   98. ],
           [   0.2,    8. ,  101. ],
           [   0.2,   13. ,   92. ],
           [   0.2,   23. ,   79. ],
           [   0.5,   40. ,   56. ],
           [   0.6,   49. ,   43. ],
           [   0.5,   66. ,   25. ],
           [   0.4,   47. ,   44. ],
           [   0.3,   18. ,   76. ],
           [   0.3,   25. ,   64. ],
           [   0.3,   16. ,   77. ],
           [   0.3,   16. ,   59. ],
           [   0.3,   34. ,   31. ],
           [   0.3,   27. ,   33. ],
           [   0.3,   44. ,   17. ],
           [   0.4,   45. ,    9. ],
           [   0.5,   52. ,   22. ],
           [   0.4,   37. ,   53. ],
           [   0.3,   21. ,   73. ],
           [   0.3,   20. ,   76. ],
           [   0.3,   24. ,   76. ],
           [   0.4,   38. ,   71. ],
           [   0.3,   32. ,   78. ],
           [   0.3,   21. ,   89. ],
           [   0.2,   10. ,  105. ],
           [   0.3,   15. ,  102. ],
           [   0.3,   21. ,   93. ],
           [   0.3,   45. ,   63. ],
           [   0.4,   59. ,   47. ],
           [   0.4,   59. ,   44. ],
           [   0.7,   99. ,    9. ],
           [   0.6,   88. ,    9. ],
           [   0.8,   93. ,    9. ],
           [   0.9,   89. ,    9. ],
           [   0.8,   84. ,    8. ],
           [   0.5,   64. ,   10. ],
           [   0.4,   58. ,   11. ],
           [   0.5,   53. ,    9. ],
           [   0.4,   41. ,    8. ],
           [   0.5,   43. ,    9. ],
           [   0.5,   45. ,   13. ],
           [   0.6,   51. ,   25. ],
           [   0.5,   44. ,   40. ],
           [   0.4,   36. ,   59. ],
           [   0.4,   36. ,   68. ],
           [   0.3,   26. ,   84. ],
           [   0.3,   16. ,   98. ],
           [   0.3,   17. ,   97. ],
           [   0.3,   24. ,   89. ],
           [   0.3,   17. ,   99. ],
           [   0.3,   12. ,  100. ],
           [   0.3,   42. ,   61. ],
           [   0.4,   52. ,   44. ],
           [   0.5,   54. ,   39. ],
           [   0.5,   60. ,   28. ],
           [   0.6,   73. ,   13. ],
           [   0.5,   58. ,   23. ],
           [   0.4,   58. ,   16. ],
           [   0.5,   61. ,   10. ],
           [   0.5,   59. ,    9. ],
           [   0.4,   50. ,    9. ],
           [   0.3,   31. ,   10. ],
           [   0.4,   36. ,    9. ],
           [   0.6,   45. ,    9. ],
           [   0.5,   43. ,   18. ],
           [   0.5,   37. ,   24. ],
           [   0.5,   40. ,   38. ],
           [   0.4,   26. ,   59. ],
           [   0.3,   14. ,   67. ],
           [   0.3,   12. ,   64. ],
           [   0.3,   13. ,   62. ],
           [   0.2,   10. ,   63. ],
           [   0.2,    7. ,   58. ],
           [   0.2,    8. ,   53. ],
           [   0.2,   11. ,   51. ],
           [   0.3,   16. ,   47. ],
           [   0.2,   19. ,   42. ],
           [   0.3,   22. ,   38. ],
           [   0.3,   23. ,   36. ],
           [   0.3,   16. ,   43. ],
           [   0.2,    9. ,   49. ],
           [   0.2,    6. ,   48. ],
           [   nan,    nan,    nan],
           [   0.2,    4. ,   64. ],
           [   0.2,    3. ,   89. ],
           [   0.2,    4. ,   90. ],
           [   0.2,    3. ,   92. ],
           [   0.2,    6. ,   89. ],
           [   0.3,   11. ,   83. ],
           [   0.3,    9. ,   87. ],
           [   0.3,    8. ,   84. ],
           [   0.3,   10. ,   82. ],
           [   0.3,   10. ,   80. ],
           [   0.3,   12. ,   80. ],
           [   0.3,   12. ,   81. ],
           [   0.3,    8. ,   84. ],
           [   0.3,   10. ,   85. ],
           [   0.2,   10. ,   85. ],
           [   0.2,   14. ,   82. ],
           [   0.3,   18. ,   72. ],
           [   0.3,   28. ,   60. ],
           [   0.3,   30. ,   55. ],
           [   0.3,   21. ,   61. ],
           [   0.3,   16. ,   63. ],
           [   0.3,   12. ,   65. ],
           [   0.2,    9. ,   67. ],
           [   0.2,    5. ,   70. ],
           [   0.2,    5. ,   69. ],
           [   0.2,    6. ,   65. ],
           [   0.2,    7. ,   63. ],
           [   0.3,   16. ,   55. ],
           [   0.3,   30. ,   45. ],
           [   0.3,   38. ,   39. ],
           [   0.3,   37. ,   41. ],
           [   0.3,   29. ,   53. ],
           [   0.3,   27. ,   53. ],
           [   0.3,   27. ,   49. ],
           [   0.3,   23. ,   54. ],
           [   0.3,   22. ,   57. ],
           [   0.3,   19. ,   61. ],
           [   0.3,   17. ,   63. ],
           [   0.3,   22. ,   59. ],
           [   0.3,   27. ,   53. ],
           [   0.3,   29. ,   50. ],
           [   0.3,   34. ,   44. ],
           [   0.3,   33. ,   45. ],
           [   0.3,   26. ,   50. ],
           [   0.3,   19. ,   56. ],
           [   0.2,   11. ,   63. ],
           [   0.2,    8. ,   63. ],
           [   0.2,    9. ,   58. ],
           [   0.2,    6. ,   63. ],
           [   0.2,    5. ,   66. ],
           [   0.2,    7. ,   62. ],
           [   0.3,   18. ,   53. ],
           [   0.4,   38. ,   37. ],
           [   0.4,   49. ,   28. ],
           [   0.4,   45. ,   35. ],
           [   0.3,   34. ,   47. ],
           [   0.3,   24. ,   62. ],
           [   0.3,   24. ,   68. ],
           [   0.3,   28. ,   68. ],
           [   0.3,   23. ,   78. ],
           [   0.3,   21. ,   82. ],
           [   0.3,   17. ,   87. ],
           [   0.3,   23. ,   80. ],
           [   0.3,   28. ,   75. ],
           [   0.3,   29. ,   71. ],
           [   0.3,   46. ,   50. ],
           [   0.4,   66. ,   27. ],
           [   0.3,   51. ,   38. ],
           [   0.3,   42. ,   46. ]])



##### Dealing with missing values 


```python
np.mean(data1, axis=0)
```




    array([ nan,  nan,  nan])




```python
np.nanmean(data1, axis=0)
```




    array([  0.33717277,  29.79581152,  55.47643979])




```python
# masking invalid data
data1 = np.ma.masked_invalid(data1)
np.mean(data1, axis=0)
```




    masked_array(data = [0.3371727748691094 29.79581151832461 55.47643979057592],
                 mask = [False False False],
           fill_value = 1e+20)




```python
data2 = np.genfromtxt('./data/barrio_del_pilar-20151222.csv', skip_header=3, delimiter=';', usecols=(2,3,4))
data2 = np.ma.masked_invalid(data2)
```

##### Plotting the data 

** Maximum values ** from: http://www.mambiente.munimadrid.es/opencms/export/sites/default/calaire/Anexos/valores_limite_1.pdf

* NO2
    - Media anual: 40 µg/m3
    - **Media horaria: 200 µg/m3 **


```python
plt.plot(data1[:, 1], label='2016')
plt.plot(data2[:, 1], label='2015')

plt.legend()

plt.hlines(200, 0, 200, linestyles='--')
plt.ylim(0, 220)
```


```python
from IPython.display import HTML
HTML('')
```








* CO 
    - **Máxima diaria de las medias móviles octohorarias: 10 mg/m³**


```python
# http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.convolve.html
def moving_average(x, N=8):
    return np.convolve(x, np.ones(N)/N, mode='same')
```


```python
plt.plot(moving_average(data1[:, 0]), label='2016')

plt.plot(moving_average(data2[:, 0]), label='2015')

plt.hlines(10, 0, 250, linestyles='--')
plt.ylim(0, 11)

plt.legend()
```

* O3
    - **Máxima diaria de las medias móviles octohorarias: 120 µg/m3**
    - Umbral de información. 180 µg/m3
    - Media horaria. Umbral de alerta. 240 µg/m3


```python
plt.plot(moving_average(data1[:, 2]), label='2016')
#plt.plot(data1[:, 2])

plt.plot(moving_average(data2[:, 2]), label='2015')
#plt.plot(data2[:, 2])

plt.hlines(180, 0, 250, linestyles='--')
plt.ylim(0, 190)

plt.legend()
```

### 4. Scientific functions: SciPy



```
scipy.linalg: ATLAS LAPACK and BLAS libraries

scipy.stats: distributions, statistical functions...

scipy.integrate: integration of functions and ODEs

scipy.optimization: local and global optimization, fitting, root finding...

scipy.interpolate: interpolation, splines...

scipy.fftpack: Fourier trasnforms

scipy.signal, scipy.special, scipy.io
```

#### Temperature data 

Now, we will use some temperature data from the Spanish Ministry of Agriculture.


```python
HTML('')
```








The file contains data from 2004 to 2015 (included). Each row corresponds to a day of the year, so evey 365 lines contain data from a whole year*

Note1: 29th February has been removed for leap-years.
Note2: Missing values have been replaced with the immediately prior valid data.

These kind of events are better handled with Pandas!


```python
!head data/M01_Center_Finca_temperature_data_2004_2015.csv
```

    # mean; max; min
    3.49;10.87;-2.75
    5.10;10.88;-2.36
    8.77;11.86;3.80
    6.07;14.77;-1.36
    2.48;12.06;-3.68
    1.69;11.47;-3.81
    2.57;9.29;-3.81
    6.28;11.08;0.49
    11.66;16.24;8.76



```python
# Loading the data
temp_data = np.genfromtxt('data/M01_Center_Finca_temperature_data_2004_2015.csv',
                          skip_header=1,
                          delimiter=';')
```


```python
# Importing SciPy stats
import scipy.stats as st
```


```python
# Applying some functions: describe, mode, mean...
st.describe(temp_data)
```




    DescribeResult(nobs=4380, minmax=(array([ -4.21,  -0.04, -11.88]), array([ 30.59,  42.  ,  21.77])), mean=array([ 14.03399543,  21.22098858,   6.86335388]), variance=array([ 59.25648501,  76.46548678,  44.03205308]), skewness=array([ 0.07277802,  0.10781561, -0.0989195 ]), kurtosis=array([-1.09612434, -1.11634177, -0.94371617]))




```python
st.mode(temp_data)
```




    ModeResult(mode=array([[ 21.  ,  24.77,   0.  ]]), count=array([[10, 11, 53]]))




```python
np.mean(temp_data, axis=0)
```




    array([ 14.03399543,  21.22098858,   6.86335388])




```python
np.median(temp_data, axis=0)
```




    array([ 13.52,  20.62,   7.09])



We can also get information about **percentiles**!


```python
st.scoreatpercentile(temp_data, per=25, axis=0)
```




    array([  7.65 ,  13.665,   1.32 ])




```python
st.percentileofscore(temp_data[:,0], score=0)
```




    1.1415525114155249




```python
st.percentileofscore(temp_data[:,1], score=0)
```




    0.022831050228310501




```python
st.percentileofscore(temp_data[:,2], score=0)
```




    19.315068493150687



##### Rearranging the data 




```python
temp_data2 = np.zeros([365, 3, 12])

for year in range(12):
    temp_data2[:, :, year] = temp_data[year*365:(year+1)*365, :]
```


```python
# Calculating mean of mean temp
mean_mean = np.mean(temp_data2[:, 0, :], axis=1)
# max of max
max_max = np.max(temp_data2[:, 1, :], axis=1)
# min of min
min_min = np.min(temp_data2[:, 2, :], axis=1)
```

##### Let's visualize data! 

Using matplotlib styles http://matplotlib.org/users/whats_new.html#styles


```python
%matplotlib inline
plt.style.available
```




    ['seaborn-deep',
     'seaborn-ticks',
     'dark_background',
     'grayscale',
     'seaborn-muted',
     'ggplot',
     'seaborn-talk',
     'seaborn-paper',
     'seaborn-poster',
     'seaborn-bright',
     'seaborn-whitegrid',
     'classic',
     'fivethirtyeight',
     'seaborn-pastel',
     'bmh',
     'seaborn-white',
     'seaborn-darkgrid',
     'seaborn-dark',
     'seaborn-colorblind',
     'seaborn-notebook',
     'seaborn-dark-palette']




```python
plt.style.use('ggplot')
```


```python
days = np.arange(1, 366)

plt.fill_between(days, max_max, min_min, alpha=0.7)
plt.plot(days, mean_mean)
plt.xlim(1, 365)
```

Let's see if 2015 was a normal year...


```python
plt.plot(days, temp_data2[:,0,-1], lw=2)
plt.plot(days, mean_mean)
plt.xlim(1, 365)
```


```python
plt.fill_between(days, max_max, min_min, alpha=0.7)
plt.fill_between(days, temp_data2[:,1,-1], temp_data2[:,2,-1],
                 color='purple', alpha=0.5)
plt.plot(days, temp_data2[:,0,-1], lw=2)
plt.xlim(1, 365)
```

For example, lets represent a function over a 2D domain!

For this we will use the contour function, which requires some special inputs...


```python
#we will use numpy functions in order to work with numpy arrays
def funcion(x,y):
    return np.cos(x) + np.sin(y)
```


```python
# 0D: works!
funcion(3,5)
```




    -1.9489167712635838




```python
# 1D: works!
x = np.linspace(0,5, 100)
plt.plot(x, funcion(x,1))
```

In oder to plot the 2D function, we will need a grid.

For 1D domain, we just needed one 1D array containning the X position and another 1D array containing the value.

Now, we will create a grid, a distribution of points covering a surface. For the 2D domain, we will need:
- One 2D array containing the X coordinate of the points.
- One 2D array containing the Y coordinate of the points.
- One 2D array containing the function value at the points.

The three matrices must have the exact same dimensions, because each cell of them represents a particular point.


```python
#We can create the X and Y matrices by hand, or use a function designed to make ir easy:

#we create two 1D arrays of the desired lengths:
x_1d = np.linspace(0, 5, 5)
y_1d = np.linspace(-2, 4, 7)
#And we use the meshgrid function to create the X and Y matrices!
X, Y = np.meshgrid(x_1d, y_1d)
```


```python
X
```




    array([[ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ],
           [ 0.  ,  1.25,  2.5 ,  3.75,  5.  ]])




```python
Y
```




    array([[-2., -2., -2., -2., -2.],
           [-1., -1., -1., -1., -1.],
           [ 0.,  0.,  0.,  0.,  0.],
           [ 1.,  1.,  1.,  1.,  1.],
           [ 2.,  2.,  2.,  2.,  2.],
           [ 3.,  3.,  3.,  3.,  3.],
           [ 4.,  4.,  4.,  4.,  4.]])



Note that with the meshgrid function we can only create rectangular grids


```python
#Using Numpy arrays, calculating the function value at the points is easy!
Z = funcion(X,Y)
```


```python
#Let's plot it!
plt.contour(X, Y, Z)
plt.colorbar()
```

We can try a little more resolution...


```python
x_1d = np.linspace(0, 5, 100)
y_1d = np.linspace(-2, 4, 100)
X, Y = np.meshgrid(x_1d, y_1d)
Z = funcion(X,Y)
plt.contour(X, Y, Z)
plt.colorbar()
```

The countourf function is simmilar, but it colours also between the lines. In both functions, we can manually adjust the number of lines/zones we want to differentiate on the plot.


```python
plt.contourf(X, Y, Z, np.linspace(-2, 2, 6),cmap=plt.cm.Spectral) #With cmap, a color map is specified
plt.colorbar()
```


```python
plt.contourf(X, Y, Z, np.linspace(-2, 2, 100),cmap=plt.cm.Spectral)
plt.colorbar()
```


```python
#We can even combine them!
plt.contourf(X, Y, Z, np.linspace(-2, 2, 100),cmap=plt.cm.Spectral)
plt.colorbar()
cs = plt.contour(X, Y, Z, np.linspace(-2, 2, 9), colors='k')
plt.clabel(cs)

```

These functions can be enormously useful when you want to visualize something.

And remember!

## Always visualize data!

Let's try it with **Real** data!


```python
time_vector = np.loadtxt('data/ligo_tiempos.txt')
frequency_vector =  np.loadtxt('data/ligo_frecuencias.txt')
intensity_matrix =  np.loadtxt('data/ligo_datos.txt')
```

The time and frequency vectors contain the values at which the instrument was reading, and the intensity matrix, the postprocessed strength measured for each frequency at each time.

We need again to create the 2D arrays of coordinates.


```python
time_2D, freq_2D = np.meshgrid(time_vector, frequency_vector)
```


```python
plt.figure(figsize=(10,6)) #We can manually adjust the sice of the picture
plt.contourf(time_2D, freq_2D,intensity_matrix,np.linspace(0, 0.02313, 200),cmap='bone')
plt.xlabel('time (s)')
plt.ylabel('Frequency (Hz)')
plt.colorbar()
```

Wow! What is that? Let's zoom into it!


```python

plt.figure(figsize=(10,6))
plt.contourf(time_2D, freq_2D,intensity_matrix,np.linspace(0, 0.02313, 200),cmap = plt.cm.Spectral)
plt.colorbar()
plt.contour(time_2D, freq_2D,intensity_matrix,np.linspace(0, 0.02313, 9), colors='k')
plt.xlabel('time (s)')
plt.ylabel('Frequency (Hz)')

plt.axis([9.9, 10.05, 0, 300])
```

# IPython Widgets

The IPython Widgets are interactive tools to use in the notebook. They are fun and very useful to quickly understand how different parameters affect a certain function.

This is based on a section of the PyConEs 14 talk by Kiko Correoso "Hacking the notebook": http://nbviewer.jupyter.org/github/kikocorreoso/PyConES14_talk-Hacking_the_Notebook/blob/master/notebooks/Using%20Interact.ipynb


```python
from ipywidgets import interact
```


```python
#Lets define a extremely simple function:
def ejemplo(x):
    print(x)
```


```python
interact(ejemplo, x =10) #Try changing the value of x to True, 'Hello' or ['hello', 'world']
```

    10





    <function __main__.ejemplo>




```python
#We can control the slider values with more precission:
interact(ejemplo, x = (9,10,0.1))
```

    9.0





    <function __main__.ejemplo>



If you want a dropdown menu that passes non-string values to the Python function, you can pass a dictionary. The keys in the dictionary are used for the names in the dropdown menu UI and the values are the arguments that are passed to the underlying Python function.


```python
interact(ejemplo, x={'one': 10, 'two': 20})
```

    10





    <function __main__.ejemplo>



Let's have some fun! We talked before about frequencys and waves. Have you ever learn about AM and FM modulation? It's the process used to send radio communications!


```python
x = np.linspace(-1, 7, 1000)

fig = plt.figure()

plt.subplot(211)#This allows us to display multiple sub-plots, and where to put them
plt.plot(x, np.sin(x))
plt.grid(False)
plt.title("Audio signal: modulator")

plt.subplot(212)
plt.plot(x, np.sin(50 * x))
plt.grid(False)
plt.title("Radio signal: carrier")
```


```python
#Am modulation simply works like this:
am_wave = np.sin(50 * x) * (0.5 + 0.5 * np.sin(x))
plt.plot(x, am_wave)
```

In order to interact with it, we will need to transform it into a function


```python
def am_mod (f_carr=50, f_mod=1, depth=0.5): #The default values will be the starting points of the sliders
    x = np.linspace(-1, 7, 1000)
    am_wave = np.sin(f_carr * x) * (1- depth/2 + depth/2 * np.sin(f_mod * x))
   
    plt.plot(x, am_wave)
    
```


```python
interact(am_mod,
        f_carr = (1,100,2),
        f_mod = (0.2, 2, 0.1),
        depth = (0, 1, 0.1))
```

#### Other options... 



### 5. Other packages

#### Symbolic calculations with SymPy 



SymPy is a Python package for symbolic math. We will not cover it in depth, but let's take a picure of the basics!


```python
# Importación
from sympy import init_session

init_session(use_latex='matplotlib') #We must start calling this function
```

    IPython console for SymPy 1.0 (Python 3.5.1-64-bit) (ground types: python)
    
    These commands were executed:
    >>> from __future__ import division
    >>> from sympy import *
    >>> x, y, z, t = symbols('x y z t')
    >>> k, m, n = symbols('k m n', integer=True)
    >>> f, g, h = symbols('f g h', cls=Function)
    >>> init_printing()
    
    Documentation can be found at http://docs.sympy.org/1.0/


The basic unit of this package is the symbol. A simbol object has name and graphic representation, which can be different:


```python
coef_traccion = symbols('c_T')
coef_traccion
```


```python
w = symbols('omega')
W = symbols('Omega')
w, W
```

By default, SymPy takes symbols as complex numbers. That can lead to unexpected results in front of certain operations, like logarithms. We can explicitly signal that a symbol is real when we create it. We can also create several symbols at a time.


```python
x, y, z, t = symbols('x y z t', real=True)
x.assumptions0
```




    {'commutative': True,
     'complex': True,
     'hermitian': True,
     'imaginary': False,
     'real': True}



Expressions can be created from symbols:


```python
expr = cos(x)**2 + sin(x)**2
expr
```


```python
simplify(expr)
```


```python
#We can substitute pieces of the expression:
expr.subs(x, y**2)
```


```python
#We can particularize on a certain value:
(sin(x) + 3 * x).subs(x, pi)
```


```python
#We can evaluate the numerical value with a certain precission:
(sin(x) + 3 * x).subs(x, pi).evalf(25)
```

We can manipulate the expression in several ways. For example:


```python
expr1 = (x ** 3 + 3 * y + 2) ** 2
expr1
```


```python
expr1.expand()
```

We can derivate and integrate:


```python
expr = cos(2*x)
expr.diff(x, x, x)
```


```python
expr_xy = y ** 3 * sin(x) ** 2 + x ** 2 * cos(y)
expr_xy
```


```python
diff(expr_xy, x, 2, y, 2)
```


```python
int2 =  1 / sin(x)
integrate(int2)
```


```python
x, a = symbols('x a', real=True)

int3 = 1 / (x**2 + a**2)**2
integrate(int3, x)
```

We also have ecuations and differential ecuations:


```python
a, x, t, C = symbols('a, x, t, C', real=True)
ecuacion = Eq(a * exp(x/t), C)
ecuacion
```


```python
solve(ecuacion ,x)
```


```python
x = symbols('x')
f = Function('y')
ecuacion_dif = Eq(f(x).diff(x,2) + f(x).diff(x) + f(x), cos(x))
ecuacion_dif
```


```python
dsolve(ecuacion_dif, f(x))
```

#### Data Analysis with pandas 



Pandas is a package that focus on data structures and data analysis tools. We will not cover it because the next workshop, by Kiko Correoso, will develop it in depth.

#### Machine Learning with scikit-learn 



Scikit-learn is a very complete Python package focusing on machin learning, and data mining and analysis. We will not cover it in depth because it will be the focus of many more talks at the PyData.

##### A world of possibilities... 



# Thanks for yor attention! 



## Any Questions?


---



```python
# Notebook style
from IPython.core.display import HTML
css_file = './static/style.css'
HTML(open(css_file, "r").read())
```




<link href='http://fonts.googleapis.com/css?family=Source+Sans+Pro|Josefin+Sans:400,700,400italic|Ubuntu+Condensed' rel='stylesheet' type='text/css'>


<style>

#notebook_panel { /* main background */
    background: #f7f7f7;
}

div.cell { /* set cell width */
    width: 900px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 950px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
    margin-top:0.7em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running { 
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    font-family: 'Source Sans Pro', sans-serif;
    background-color: rgb(256,256,256);
    font-size: 110%;
    border-radius: 0px; 
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Josefin Sans', serif;
    line-height: 145%;
    font-size: 125%;
    font-weight: 500;
    width:750px;
    margin-left:auto;
    margin-right:auto;
}


/* Formatting for header cells */
.text_cell_render h1, .text_cell_render h2, .text_cell_render h3,
.text_cell_render h4, .text_cell_render h5 {
    font-family: 'Ubuntu Condensed', sans-serif;
}

.text_cell_render h1 {    /*Use this for Title*/
    font-weight: 500;
    font-size: 38pt;
    line-height: 100%;
    color: #EE9141;
    text-align: center;
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h2 {    /*Use this for Subtitle*/
    margin-top:16px;
    font-size: 32pt;
    font-weight: 500;
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    text-align: center;
    font-style: regular;
    color: #459EBA;
}	

.text_cell_render h3 {   /*Sections*/ 
    font-size: 30pt;
    font-weight: 300;
    text-align: left;
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    font-style: regular;
    color:  #252525;
}

.text_cell_render h4 {    /*Subsections*/
    font-size: 28pt;
    font-weight: 200;
    text-align: left;
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    font-style: regular;
    color:  #377e95;
}

.text_cell_render h5 {    /*Subsubsections*/
    font-size: 20pt;
    font-weight: 300;
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
    color:  #e77615;
}

.text_cell_render h6 {    /*Author*/
    font-family: 'Ubuntu Condensed', sans-serif;
    font-weight: 100;
    font-size: 14pt;
    line-height: 100%;
    color: #252525;
    text-align: right;
    margin-bottom: 1px;
    margin-top: 3px;
}

.CodeMirror{
        font-family: 'Duru Sans', sans-serif;
        font-size: 100%;
}

</style>





7239d9a7d834af9e0ce8c2a1593e48d28c01e630
ipynb
Jupyter Notebook
workshops_materials/20160408_0900_Basic_Python_Packages_for_Science/Basic Python Packages for Science.ipynb
PyDataMadrid2016/Conference-Info
25ed0ec4127a6ee41ba5499098db07bddd24ac84
MIT
2016-04-06T22:04:43.000Z
2021-10-09T14:03:46.000Z
workshops_materials/20160408_0900_Basic_Python_Packages_for_Science/Basic Python Packages for Science.ipynb
PyDataMadrid2016/Conference-Info
25ed0ec4127a6ee41ba5499098db07bddd24ac84
MIT
2016-04-01T17:55:50.000Z
2022-03-24T09:24:18.000Z
workshops_materials/20160408_0900_Basic_Python_Packages_for_Science/Basic Python Packages for Science.ipynb
PyDataMadrid2016/Conference-Info
25ed0ec4127a6ee41ba5499098db07bddd24ac84
MIT
2016-04-01T14:09:05.000Z
2021-10-09T14:03:48.000Z
Qwen/Qwen-72B
1. YES
2. YES