title: Hierarchical classification top down approach to machine learning
body: <p>I have a dataset of sentences that have been annotated with labels from a hierarchy. The hierarchy is a selection of music genres. It is a tree, not a DAG - each node has one parent and one parent only.  Here is an extract as an example:</p>
  
  <pre><code>root = music
       parent = latin
              child = afro-cuban
                      child = salsa
              child = brazilian
                      child = axe
       parent = non-latin
              child = classical
       ...
  </code></pre>
  
  <p>For the sentence <code>Mozart is the best</code> for example, from the collected annotations, the majority agree the class label for this sentence or ground truth is <code>classical</code>. From the hierarchy, we know that <code>classical</code> is also a form of <code>non-latin</code> music, which is a form of <code>music</code>. Whereas <code>i prefer salsa</code> might have been annotated as <code>latin</code>.</p>
  
  <p>In terms of classification, flattening the hierarchy - which I have done - intuitively does not solve the problem, as we're completely ignoring the class hierarchy. It also produces low results whilst using Weka, and a selection of classifiers, as we're faced with a multiclass classification problem.</p>
  
  <p>My problem is, I've read very vague literature and online articles about how hierarchical classification is implemented. I'd like to use Weka and Python. But I just wanted clarification of how to perform hierarchical classification in this situation. So my questions are:</p>
  
  <p>1) what is the best suggestion of going around this? Would implementing a top-down approach be the best option? If I do this, how do I avoid the problem of classifying incorrectly on each level? i.e. it could predict <code>latin</code> on level 1, and <code>classical</code> on level 2. What about a binary classifier? I'm open to suggestions.</p>
  
  <p>2) how does training and testing data come into this?</p>
  
  <p>3) how can one evaluate classification performance? Particularly with a top-down approach, as we'll have evaluations for every separate level.</p>