title: git clean is not removing a submodule added to a branch when switching branches
body: <p>How do I get rid of submodules when switching branches. I do not understand why git clean says it removed the submodule but does not. Is this a bug? Below are cut&amp;paste steps to reproduce.</p>
  
  <pre><code>git --version
  git version 1.7.8.4
  
  git init submod
  cd submod
  echo "This is a submodule" &gt; README.txt
  git add .
  git commit -m "Initial commit"
  cd ..
  git init prog
  cd prog
  echo "This is a program" &gt; README.txt
  git add .
  git commit -a -m "Initial commit"
  git checkout -b topic1
  git submodule add ../submod
  git commit -m "Added submodule"
  
  git checkout master
  #warning: unable to rmdir submod: Directory not empty
  #Switched to branch 'master'
  
  git status
  # On branch master
  # Untracked files:
  #   (use "git add &lt;file&gt;..." to include in what will be committed)
  #
  #       submod/
  #nothing added to commit but untracked files present (use "git add" to track)
  
  git clean -fd
  #Removing submod/
  
  git status
  # On branch master
  # Untracked files:
  #   (use "git add &lt;file&gt;..." to include in what will be committed)
  #
  #       submod/
  #nothing added to commit but untracked files present (use "git add" to track)
  </code></pre>