# Music machine learning - Introduction

### Author: Philippe Esling (esling@ircam.fr)

In this course we will cover
1. A [first definition](#definition) on the concept of machine learning
2. An introduction to a simple problem of [linear regression](#regression)
3. An explanation on [model capacity and overfitting](#capacity)
4. An introduction to the [audio datasets](#audio) that we will use

<a id="definition"></a>
## Defining machine learning

In all natural process, there exists complex relations between sets $\mathcal{X} \mapsto \mathcal{Y}$. This can relate some objects with their names, or a cause to a consequence. In most cases, _we do not know the precise relations_ between these sets, all we have is _observations_ such as pairs $(x,y)$, composed of input data $x \in \mathcal{X}$, which have a corresponding expected output $y \in \mathcal{Y}$. The overarching goal of machine learning is to approximate such _unknown processes_ as a function $\mathcal{F}_{\theta}$, which _transforms_ input data $x$ into output data $y$.



Hence, machine learning aims to understand and model the relationship between some (usually complex and high-dimensional) inputs $\mathbf{x}\in\mathcal{X}\subset\mathbb{R}^{\mathcal{X}}$ and outputs $\mathbf{y}\in\mathcal{Y}\subset\mathbb{R}^{\mathcal{Y}}$, given by a set of data examples $\mathcal{D}=\left\{(x_1,y_1),\cdots,(x_N,y_N)\right\}$. This is achieved by defining a parametric model $f_{\mathbf{\theta}}\in\mathcal{F}$ inside a family of functions $\mathcal{F}$, which depends on parameters $\mathbf{\theta} \in \mathbf{\Theta}$ and that could approximate the underlying relationship. The _learning_ aspect refers to the adjustment of the parameters $\mathbf{\theta}$ in order to obtain the best approximation of the given task
$$
\begin{equation}
f_{\mathbf{\theta}}(\mathbf{x}) = \hat{\mathbf{y}}\approx \mathbf{y}.
\end{equation}
$$

To observe this idea in simple setups, we are going to use the `numpy` library and `matplotlib` for plotting. We also set some fixed random generator, to ensure reproducibility


```python
# Base imports
import numpy as np
import matplotlib.pyplot as plt
from helper_plot import prep_plots, finalize_plots, hdr_plot_style
rng = np.random.RandomState(1)
```

<a id="regression"></a>
## Simple learning problem

Imagine that a certain process somewhere follows the form of a quadratic relationship

$$
 y = a x^{2} + bx + c 
$$

In this case, all the **unknown parameters** are that of a polynomial model, therefore we have $\theta = \{a, b, c\}$. However, this is clearly an ideal (clean) case, whereas in natural observations, there might be some noise in our observations
$$
 y = a x^{2} + bx + c +\epsilon \quad \mbox{with} \quad \epsilon \in [-0.1, 0.1]
$$

An example of such noisy observations for different parameters is given below


```python
noise_level = 0.1
a, b, c = 3, 0, 1
# Generating 
x = np.linspace(0, 1, 100)
poly = np.poly1d([a, b, c])
epsilon = np.random.uniform(-noise_level,noise_level,x.shape)
y = poly(x) + epsilon
ax1, = prep_plots(["Simple learning problem"], fig_size=(10,8), fig_num=1)
ax1.scatter(x, y, color='green', s=50, marker='o', edgecolor='w', label="training samples")
finalize_plots([ax1], fig_title="")
#plt.xlim([-1, 2])
```

Now our main problem is that this function can follow different types of parameters


```python
params = [[3, 0, 1], [-2, 1, 0], [0.1, 1, 1]]
ax = prep_plots(["Problem 1", "Problem 2", "Problem 3"], fig_size=(20,6), fig_num=1)
# Generating 
x = np.linspace(0, 1, 100)
for p in range(len(params)):
    poly = np.poly1d(params[p])
    epsilon = np.random.uniform(-noise_level,noise_level,x.shape)
    y = poly(x) + epsilon
    ax[p].scatter(x, y, color='green', s=50, marker='o', edgecolor='k', label="training samples")
finalize_plots(ax, fig_title="Different observations")
```

In real-life settings, this function can also have different levels of noise, as exemplified in the following code.


```python
params = [3, 0, 1]
noise_levels = [0.1, 1.0, 8.0]
ax = prep_plots(["Problem 1", "Problem 2", "Problem 3"], fig_size=(20,6), fig_num=1)
# Generating 
x = np.linspace(0, 1, 100)
for p in range(len(noise_levels)):
    poly = np.poly1d(params)
    epsilon = np.random.uniform(-noise_levels[p],noise_levels[p],x.shape)
    y = poly(x) + epsilon
    ax[p].scatter(x, y, color='green', s=50, marker='o', edgecolor='w', label="training samples")
finalize_plots(ax, fig_title="Different amounts of noise")
```

To summarize, we will have some observations of a function, and we would like to optimize a function that gets as close as possible to the real function that generated this data. Here, we plot the real function and also _subsample_ our number of observations (having only a few points to find the corresponding function)


```python
# Generating the data and subsampling
x_all = np.linspace(0, 1, 100); x_plot = np.linspace(0, 1, 100)
rng.shuffle(x_all); x = np.sort(x_all[:10])
poly = np.poly1d([3,0,1])
# Adding some external
epsilon = np.random.uniform(-0.1,0.1,x.shape)
y = poly(x)+ epsilon
# We keep the standard deviation for later
nnstd = np.std(epsilon); lw = 2
ax1, = prep_plots(["Simple learning problem"], fig_size=(10,8), fig_num=1)
ax1.plot(x_plot, poly(x_plot), color='green', linewidth=4, label='true function')
ax1.scatter(x, y, color='green', s=50, marker='o', edgecolor='w', label="training samples")
finalize_plots([ax1], fig_title="")
```

To get a first grip on what machine learning does, we will rely on the `scikit-learn` library. This contains already coded models and learning procedure, that will allow us to _learn_ the parameters of this unknown function.

Here we already know that we want to use a `PolynomialFeatures` model to perfom `LinearRegression` and that this polynomial should be of degree 2.


```python
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import make_pipeline
# Our data to fit
X = x[:, np.newaxis]
# Degree of our polynomial
degree = 2;
# Create our polynomial model for regression
model = make_pipeline(PolynomialFeatures(degree), LinearRegression())
# Fit the parameters of this model
model.fit(X, y)
```




    Pipeline(steps=[('polynomialfeatures', PolynomialFeatures()),
                    ('linearregression', LinearRegression())])



Now that we have trained the model, we can perform _predictions_ from it, meaning that we can infer the output of the function at values that we did not observe originally.


```python
# Inference points (not observed)
X_plot = x_plot[:, np.newaxis]
# Predict the values
y_plot = model.predict(X_plot)
# Compute the error of our model at observed points
Y_model_err = np.sqrt(np.mean(np.square(y-model.predict(X))))
# Plot the result
ax1, = prep_plots(["Trained model"], fig_size=(10,8), fig_num=1)
ax1.plot(x_plot, y_plot, color='red', linewidth=4, label='trained model')
ax1.scatter(x, y, color='green', s=50, marker='o', edgecolor='w', label="training samples")
finalize_plots([ax1], fig_title="")
print(f'Model error : {Y_model_err}')
```

<a id="capacity"></a>
## Understanding model capacity and selection


In real-life problem, we are aiming to find the parameters of a model, but we do not really know what is the _real_ function underlying this process. So what we can decide to select _any_ function of _any_ **capacity** (complexity of the function). One of the problem with that, is that if we have a too simple function, it will _underfit_ (it is not complex enough for our observations). On the opposite end, if we have a function which is too complex, it might be able to _fit through all training points exactly_ ... even though there is noise in our observations ! This is examplified in the following



We can observe this idea and play with it directly by trying to find a function approximating our previous observations with a polynomial function chosen to have a degree inside \([1,2,8]\).



```python
X_plot = x_plot[:, np.newaxis]
ax1, ax2 = prep_plots(["training error", "model generalization"], fig_size=(15,4), fig_num=1)
ax1.plot([1,8], [nnstd, nnstd], label="noise level")
ax2.scatter(x, y, color='orange', edgecolor='w', s=50, marker='o', label="training samples")
colors = ['blue', 'green', 'red']; labels = ['underfitting', 'good fit', 'overfitting']
axes_sub = prep_plots(labels, fig_size=(15,4), fig_num=2)
# Optimize different degree polynomials
for count, degree in enumerate([1,2,8]):
    model = make_pipeline(PolynomialFeatures(degree), LinearRegression())
    model.fit(X, y)
    Y_model_err = np.sqrt(np.mean(np.square(y-model.predict(X))))
    y_plot = model.predict(X_plot)
    ax1.scatter(degree, Y_model_err, s=50, marker="o", edgecolor='w', label=labels[count])
    ax2.plot(x_plot, y_plot, color=colors[count], linewidth=2,label=labels[count])
    ax2.set_ylim((0,4))
    axes_sub[count].scatter(x, y, color='orange', edgecolor='w', s=50, marker='o', label="Training samples")
    axes_sub[count].plot(x_plot, y_plot, color=colors[count], linewidth=2,label=labels[count])
    axes_sub[count].set_ylim((0,4))
finalize_plots(axes_sub, fig_title="Linear regression")
finalize_plots([ax1,ax2], fig_title="Linear regression")
```


Depending on the _capacity_ of the model, what we can observe is that

- `capacity too low   -> underfitting   : prediction variance >  noise variance`
- `adequate capacity  -> good fit       : prediction variance == noise variance`
- `capacity too high  -> overfitting    : prediction variance <  noise variance`


A similar example can be given for a classification problem in two dimensions as follows




<a id="audio"></a>
## Audio applications

In order to test our algorithms on audio and music data, we will work with several datasets that should be downloaded on your local computer first from this [link ](https://nuage.ircam.fr/index.php/s/FTsaaAMFV1jEwsk)

  |**Type**|*Origin*|
  |-------:|:---------|
  |**Classification**|[*MuscleFish*](http://knight.cis.temple.edu/~vasilis/Courses/CIS750/Papers/muscle_fish.pdf) dataset|
  |**Music-speech**|[*MIREX Recognition*](http://www.music-ir.org/mirex/wiki/2015:Music/Speech_Classification_and_Detection) set|
  |**Source separation**|[*SMC Mirum*](http://smc.inesctec.pt/research/data-2/) dataset|
  |**Speech recognition**|[*CMU Arctic*](http://festvox.org/cmu_arctic/) dataset|

**Unzip the file and place the `data` folder along with the other code folders**
For the first parts of the tutorial, we will mostly rely solely on the classification dataset. In order to facilitate the interactions, we provide the function `import_dataset` that will allow to import all audio datasets along the tutorials.

```Python
def importDataset(class_path, type):
    """
    Helper function to import datasets
    % class_path  : Path to the dataset (string)
    % type       : Type of dataset (string: 'classify', 'plain', 'metadata')
    """ 
    # Returns the data_struct structure with
    data_struct["filenames"]  # Cell containing the list of audio files
    data_struct["classes"]    # Vector of indexes assigning each file to a class
    data_struct["class_names"] # Cell of class names
    return data_struct
```
  
***

**_Exercise_**  

  1. Launch the import procedure  and check the corresponding structure
  2. Code a count function that prints the name and number of examples for each classes 

***


```python
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
from helper_data import import_dataset
# 0.1 - Import the classification dataset
class_path = 'datasets/classification';
data_struct = import_dataset(class_path, 'classification');
```

        - Importing dataset datasets/classification.
    


### Preprocessing

We will rely on a set of spectral transforms that allow to obtain a more descriptive view over the audio information. As most of these are out of the scope of the machine learning course, we redirect you to a [signal processing course](https://ccrma.stanford.edu/~jos/sasp/) proposed by [Julius O. Smith](https://ccrma.stanford.edu/~jos/).  

The following functions to compute various types of transforms are given as part of the basic package, in the `helper_data.py` file  

  |**File**|*Transform*|
  |-------:|:----------|
  |`stft.m`       |[Short-term Fourier transform](https://en.wikipedia.org/wiki/Short-time_Fourier_transform)|
  |`fft2barkmx.m` |[Bark scale](https://en.wikipedia.org/wiki/Bark_scale) transform|
  |`fft2melmx.m`  |[Mel scale](https://en.wikipedia.org/wiki/Mel_scale) transform|
  |`fft2chromamx` |[Chromas vector](https://en.wikipedia.org/wiki/Harmonic_pitch_class_profiles)|
  |`spec2cep.m`   |[Cepstrum](https://en.wikipedia.org/wiki/Cepstrum) transform|
  |`cqt.m`        |[Constant-Q](https://en.wikipedia.org/wiki/Constant_Q_transform) transform|

In order to perform the various computations, we provide the following function, which performs the different transforms on a complete dataset.  

``` Python
def computeTransforms(data_struct)
    """ data_struct   : Dataset structure with filenames """
    return data_struct

# Returns the data_struct structure with
data_struct["spectrum_power"]     # Power spectrum (STFT)
data_struct["spectrum_mel"]       # Spectrum in Mel scale
data_struct["spectrum_chroma"]    # Chroma vectors
data_struct["spectrum_CQT"]       # Constant-Q transform
```


***

**Exercise**  

  1. Launch the transform computation procedure and check the corresponding structure
  2. For each class, select a random element and plot its various transforms on a single plot. You should obtain plots similar to those shown afterwards.
  3. For each transform, try to spot major pros and cons of their representation.
  
***


```python
# 0.2 - Pre-process the audio to obtain spectral transforms 
# (may take around a minute)
from helper_data import compute_transforms
data_struct = compute_transforms(data_struct);
```

        - Performing transforms.
    21665
    1024
    11425
    1024
    12033
    1024
    11553
    1024
    21986
    1024
    11137
    1024
    26658
    1024
    5217
    1024
    7105
    1024
    12609
    1024
    3425
    1024
    6881
    1024
    10977
    1024
    7073
    1024
    10626
    1024
    8865
    1024
    7298
    1024
    7170
    1024
    3362
    1024
    3329
    1024
    5505
    1024
    4577
    1024
    6337
    1024
    7458
    1024
    175196
    1024
    41665
    1024
    134012
    1024
    55170
    1024
    22081
    1024
    127037
    1024
    118497
    1024
    34029
    1024
    27235
    1024
    31831
    1024
    24224
    1024
    29589
    1024
    30343
    1024
    28612
    1024
    24469
    1024
    32502
    1024
    28987
    1024
    29597
    1024
    27589
    1024
    30723
    1024
    35863
    1024
    26375
    1024
    31149
    1024
    31667
    1024
    24719
    1024
    36978
    1024
    27165
    1024
    30077
    1024
    33838
    1024
    35835
    1024
    30605
    1024
    29884
    1024
    29625
    1024
    28010
    1024
    30241
    1024
    27983
    1024
    33241
    1024
    31101
    1024
    28642
    1024
    31579
    1024
    28309
    1024
    18766
    1024
    13006
    1024
    12424
    1024
    18666
    1024
    13390
    1024
    18642
    1024
    27934
    1024
    22242
    1024
    23506
    1024
    22319
    1024
    23870
    1024
    6850
    1024
    7393
    1024
    9281
    1024
    5538
    1024
    3969
    1024
    5089
    1024
    2977
    1024
    4321
    1024
    3265
    1024
    7969
    1024
    6401
    1024
    6689
    1024
    8001
    1024
    6209
    1024
    7553
    1024
    3169
    1024
    10817
    1024
    8545
    1024
    3841
    1024
    6434
    1024
    3681
    1024
    4226
    1024
    4001
    1024
    4161
    1024
    5921
    1024
    4033
    1024
    3777
    1024
    6465
    1024
    8961
    1024
    4226
    1024
    11074
    1024
    6753
    1024
    3905
    1024
    6465
    1024
    5665
    1024
    3479
    1024
    67580
    1024
    5119
    1024
    5054
    1024
    4907
    1024
    4579
    1024
    4450
    1024
    4249
    1024
    62283
    1024
    5360
    1024
    4474
    1024
    59555
    1024
    4169
    1024
    4501
    1024
    4189
    1024
    4397
    1024
    4125
    1024
    5769
    1024
    9795
    1024
    6808
    1024
    44406
    1024
    19651
    1024
    6876
    1024
    7379
    1024
    6836
    1024
    4227
    1024
    38038
    1024
    23102
    1024
    7072
    1024
    20036
    1024
    20645
    1024
    4704
    1024
    8235
    1024
    24275
    1024
    9986
    1024
    7160
    1024
    4479
    1024
    34346
    1024
    17861
    1024
    9952
    1024
    7316
    1024
    5454
    1024
    10422
    1024
    50587
    1024
    7769
    1024
    4778
    1024
    9773
    1024
    8695
    1024
    5722
    1024
    4896
    1024
    12216
    1024
    31226
    1024
    9322
    1024
    10124
    1024
    23648
    1024
    20142
    1024
    11130
    1024
    19786
    1024
    5062
    1024
    33889
    1024
    15168
    1024
    5578
    1024
    14117
    1024
    19848
    1024
    8947
    1024
    10478
    1024
    10711
    1024
    12150
    1024
    13382
    1024
    3509
    1024
    20841
    1024
    8493
    1024
    8924
    1024
    17470
    1024
    17033
    1024
    17601
    1024
    7163
    1024
    15192
    1024
    20074
    1024
    9245
    1024
    7153
    1024
    9206
    1024
    6990
    1024
    5024
    1024
    13619
    1024
    21891
    1024
    22826
    1024
    33104
    1024
    25496
    1024
    7391
    1024
    6571
    1024
    87418
    1024
    11243
    1024
    7591
    1024
    16082
    1024
    6050
    1024
    4087
    1024
    8665
    1024
    45127
    1024
    23458
    1024
    19282
    1024
    10696
    1024
    9463
    1024
    18728
    1024
    10243
    1024
    5914
    1024
    10139
    1024
    12934
    1024
    7048
    1024
    33807
    1024
    22290
    1024
    8778
    1024
    8429
    1024
    10327
    1024
    9529
    1024
    8931
    1024
    5594
    1024
    3597
    1024
    3702
    1024
    4552
    1024
    3341
    1024
    5801
    1024
    5247
    1024
    3415
    1024
    6869
    1024
    5670
    1024
    5055
    1024
    5816
    1024
    6966
    1024
    3650
    1024
    3917
    1024
    13541
    1024
    4033
    1024
    3654
    1024
    3478
    1024
    5338
    1024
    6544
    1024
    6330
    1024
    4584
    1024
    6957
    1024
    5417
    1024
    5233
    1024
    5321
    1024
    5062
    1024
    5895
    1024
    8129
    1024
    6060
    1024
    3381
    1024
    7070
    1024
    6180
    1024
    3234
    1024
    5724
    1024
    5935
    1024
    6921
    1024
    6417
    1024
    6128
    1024
    23700
    1024
    28185
    1024
    32436
    1024
    32549
    1024
    22622
    1024
    36970
    1024
    32428
    1024
    126657
    1024
    218556
    1024
    74130
    1024
    89843
    1024
    64158
    1024
    129976
    1024
    125571
    1024
    70470
    1024
    82587
    1024
    37463
    1024
    32789
    1024
    23010
    1024
    23008
    1024
    20255
    1024
    28535
    1024
    23963
    1024
    18594
    1024
    24431
    1024
    21513
    1024
    24152
    1024
    22150
    1024
    23394
    1024
    23571
    1024
    21621
    1024
    24807
    1024
    21838
    1024
    22351
    1024
    25169
    1024
    24600
    1024
    23478
    1024
    21761
    1024
    60161
    1024
    79072
    1024
    82116
    1024
    76637
    1024
    79644
    1024
    35771
    1024
    51250
    1024
    41803
    1024
    15644
    1024
    39455
    1024
    37735
    1024
    40964
    1024
    40167
    1024
    40840
    1024
    37541
    1024
    39565
    1024
    36555
    1024
    34724
    1024
    40081
    1024
    43556
    1024
    34372
    1024
    40825
    1024
    42990
    1024
    38275
    1024
    39575
    1024
    40690
    1024
    37576
    1024
    34033
    1024
    39518
    1024
    40916
    1024
    31803
    1024
    31653
    1024
    36925
    1024
    33386
    1024
    36955
    1024
    37353
    1024
    31729
    1024
    41851
    1024
    39663
    1024
    36419
    1024
    36103
    1024
    35868
    1024
    40541
    1024
    33843
    1024
    32180
    1024
    38224
    1024
    37906
    1024
    35778
    1024
    36332
    1024
    31677
    1024
    35837
    1024
    39156
    1024
    35764
    1024
    34725
    1024
    33335
    1024
    39813
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=312
      warnings.warn(


    51668
    1024
    46042
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=404
      warnings.warn(


    47670
    1024
    42585
    1024
    38457
    1024
    35143
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=301
      warnings.warn(


    40147
    1024
    30855
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=483
      warnings.warn(
    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=242
      warnings.warn(


    45607
    1024
    33548
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=263
      warnings.warn(


    65551
    1024
    40027
    1024
    30128
    1024
    34709
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=236
      warnings.warn(


    41443
    1024
    21213
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=324
      warnings.warn(
    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=332
      warnings.warn(
    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=166
      warnings.warn(


    21852
    1024
    38471
    1024
    22114
    1024
    34252
    1024
    41734
    1024
    29960
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=469
      warnings.warn(


    16916
    1024
    32976
    1024
    36424
    1024
    19998
    1024
    41317
    1024
    26958
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=422
      warnings.warn(


    39794
    1024
    23099
    1024
    22850
    1024
    34472
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=270
      warnings.warn(


    54436
    1024
    36230
    1024
    50840
    1024
    42036
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=398
      warnings.warn(
    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=329
      warnings.warn(


    32290
    1024
    33000
    1024
    37032
    1024
    48575
    1024


    /Applications/anaconda3/lib/python3.8/site-packages/librosa/core/spectrum.py:222: UserWarning: n_fft=512 is too small for input signal of length=290
      warnings.warn(


    50742
    1024
    47677
    1024
    43119
    1024
    39489
    1024
    40787
    1024
    37713
    1024
    39487
    1024
    33569
    1024
    43640
    1024


### Features

<div markdown = "1">

As you might have noted from the previous exercice, most spectral transforms have a very high dimensionality, and might not be suited to exhibit the relevant structure of different classes. To that end, we provide a set of functions for computing several spectral features in the `helper_data` folder, we redirect interested readers to this [exhaustive article](http://recherche.ircam.fr/anasyn/peeters/ARTICLES/Peeters_2003_cuidadoaudiofeatures.pdf) on spectral features computation.

  |**File**|*Transform*|
  |-------:|:----------|
  |`spectral_centroid`|Spectral centroid|
  |`spectral_bandwidth`|Spectral bandwidth|
  |`spectral_contrast`|Spectral contrast|
  |`spectral_flatness`|Spectral flatness|
  |`spectral_rolloff`|Spectral rolloff|

Once again, we provide a function to perform the computation of different features on a complete set. Note that for each feature, we compute the temporal evolution in a vector along with the mean and standard deviation of each feature. We only detail the resulting data structure for a single feature (`SpectralCentroid`).  

``` Python
def data_struct = computeFeatures(data_struct)
     """ data_struct   : Dataset structure with filenames """
    return data_struct

% Returns the data_struct structure with
data_struct["spectral_centroid"]      # Temporal value of a feature
data_struct["spectral_centroid_mean"] # Mean value of that feature
data_struct["spectral_centroid_std"]  # Standard deviation
```

***

**Exercise**

  1. Launch the feature computation procedure and check the corresponding structure
  2. This time for each class, superimpose the plots of various features on a single plot, along with a boxplot of mean and standard deviations. You should obtain plots similar to those shown afterwards.
  3. What conclusions can you make on the discriminative power of each feature ?
  4. Perform scatter plots of the mean features for all the dataset, while coloring different classes.
  5. What conclusions can you make on the discriminative power of mean features ?

***


```python
#%%
# 0.3 - Compute a set of temporal and spectral features
# (may take around 1-2 minutes)
from helper_data import compute_features
data_struct = compute_features(data_struct);
```


```python
#%% Q-0.3.2 - Plot the various features 

# Just a little helper to make your figures pretty
from helper_plot import hdr_plot_style
hdr_plot_style()
# Use these styles for boxplot
boxprops=dict(linewidth=3, color='white')
whiskerprops=dict(linewidth=3, color='white')
medianprops=dict(linewidth=2.5, color='firebrick')
flierprops = dict(markeredgecolor='white', markerfacecolor='firebrick')

######################
# YOUR CODE GOES HERE
######################

```


```python
#%% Q-0.3.4 - Observe the distribution of classes for different features

# This allows to use 3D rendering in matplotlib
from mpl_toolkits.mplot3d import Axes3D
plt.figure(figsize=(12,8))
# Create a vector of random colors for each class
colorVect = np.zeros((3, len(data_struct["class_names"])));
for c in range(len(data_struct["class_names"])):
    colorVect[:,c] = np.random.rand(3);

######################
# YOUR CODE GOES HERE
######################

```

That's it for this tutorial, now remember that we can use any form of description (features) as a basis for learning algorithms. We will see in the next tutorial what we an do with these features.

ef2c466b06482260ec68b8e1013fadd1d0eb6653
ipynb
Jupyter Notebook
01a_machine_learning.ipynb
aqilahmisuary/atiam_ml
3f175a5964df2a3eee60d350e0c23296a3a4eaa1
MIT
01a_machine_learning.ipynb
aqilahmisuary/atiam_ml
3f175a5964df2a3eee60d350e0c23296a3a4eaa1
MIT
01a_machine_learning.ipynb
aqilahmisuary/atiam_ml
3f175a5964df2a3eee60d350e0c23296a3a4eaa1
MIT
Qwen/Qwen-72B
1. YES
2. YES