I've seen this guide showing a basic NMF of a spectrogram.

My aim is detect guitar pitch from the decomposition. I want to do this by explicitly create a fixed template of pitches and construct the W matrix from these.

I've seen some papers giving an overview of this, but I am not sure how it's practically done.

Do I record every note on the guitar, then do the FFT for each, take its spectrum, and paste it inside a matrix?

I guess my question is basically: how do I create the fixed W matrix, given a set of recordings with each pitch.

This paper says:

The whole note sample k is first processed in a short-time sound representation supposed to be non-negative and approximatively additive (e.g. a shorttime magnitude spectrum). The representations are stacked in a matrix $V^{(k)}$ where each column $v^{(k)}_{j}$ is the sound representation of the j-th time-frame.

So, does that mean we compute the STFT of the recorded sample? Then we use the resulting spectrogram as $V^{(k)}$?

Also, does the duration of the sample matter? If there is absolute silence in the beginning of the sample, will that affect $v^{k}_{j}$?

  • $\begingroup$ These are multiple questions. It seems you have very sensible difficulties in understanding on how to construct the dictionary (or template) matrix $\mathbf{W}$. There actually are some issues with NMF approximation. "Conventional" NMF algorithms approximate the columns of a matrix by superposition of weighted columns of a dictionary matrix ($\mathbf{W}$ here). In your case, that would mean that each note would have to be represented by a single column in $\mathbf{W}$. This is only possible to a certain degree as the tone changes/decays over time. (Compare convolutional NMF algorithms.) $\endgroup$ – applesoup Jun 13 '17 at 9:08
  • $\begingroup$ However, your approach seems worth a try anyway, and I'll try to answer your questions in an answer. $\endgroup$ – applesoup Jun 13 '17 at 9:11

The goal of non-negative matrix factorization (NMF) is to factorize (as the name says) a matrix, denoted by $\mathbf{X}\in\mathbb{R}^{(N\times K)}$ in the reference you mention, into the product of two matrices: $$ \mathbf{W}\mathbf{H}\approx\mathbf{X}, $$ with $\mathbf{W}\in\mathbb{R}^{(N\times M)}$ the template, dictionary, or basis matrix and $\mathbf{H}\in\mathbb{R}^{(M\times K)}$ the activation matrix. The approximation sign ($\approx$) in the equation indicates that in many cases it is not possible to actually find a true factorization but rather an approximation.

Inspecting the above equation reveals that each column of $\mathbf{X}$ is approximated by a linear combination of columns of $\mathbf{W}$, with the coefficients of linear combination in $\mathbf{H}$. (Experimenting with some simple examples, e.g., with $N=3,\,M=3, K=5$ and $\mathbf{X}_{ij}\in\left\{0,\,1\right\}$, will surely help gaining insight into this fundamental behaviour).

In many cases it is desired, given a matrix $\mathbf{X}$, to determine both the dictionary matrix $\mathbf{W}$ and the activation matrix $\mathbf{H}$. Optionally, of course, only one can be estimated if the other is given.

Your goal, if I understand correctly, is to keep $\mathbf{W}$ fixed, estimate $\mathbf{H}$ and use the resulting activations to determine when which note has been played. That should work as a first attempt, I think.

After this coarse overview now for your questions:

  1. You are right: to create the $\mathbf{W}$ matrix put an estimate of each note's power spectral density estimate (e.g., obtained with Welch's method) into a column of $\mathbf{W}$ (I'm not sure what you mean by "do the FFT for each, take its spectrum".).
  2. Yes, the matrix $\mathbf{X}$ in the notation used in this answer ($V^{(k)}$ in the paper that you mention) contains the STDFT magnitude spectrogram of the full input recording.
  3. Silence in the beginning of the recording will obviously have an influence on the first columns of $\mathbf{X}$ (or, respectively, $v_j^{(k)}$ for $j<\text{some number}$). However, what do you guess will happen with $\mathbf{H}$ if the first columns of $\mathbf{X}$ contain zeros or only very small values? Correct: the first columns of $\mathbf{H}$ will also receive zeros or very small values.

I hope this answer helps in clarifying the basic functionality of the NMF algorithm. A rather intuitive description of the NMF idea is also given in this famous paper.

  • $\begingroup$ Thanks! That's very useful information right there. In Dessein 2010 (the paper I linked above), the proposal is to take the STDFT for each note and do a NMF with rank one. Then take the $w$ vector and use it as a template for that note. Will that be better than simply putting the estimate of the note's power spectral density in the matrix? Finally, do you think I should take one column for each note or for every fret/string combination? After all, sound differs significantly depending on the string, even if it's the same note. If I should, then how do I change the structure of W accordingly? $\endgroup$ – pavlos163 Jun 13 '17 at 15:55

Your Answer

