title: Fastest way to sort very large files preferably with progress
body: <p>I have a 200GB flat file (one word per line) and I want to sort the file, then remove the duplicates and create one clean final TXT file out of it.</p>
  
  <p>I tried <code>sort</code> with <code>--parallel</code> but it ran for 3 days and I got frustrated and killed the process as I didn't see any changes to the chunk of files it created in /tmp. </p>
  
  <p>I need to see the progress somehow and make sure its not stuck and its working. Whats the best way to do so? Are there any Linux tools or open source project dedicated for something like this? </p>