title: Why is this program faster in Python than Objective-C?
body: <p>I got interested in <a href="https://stackoverflow.com/questions/5523058/how-to-optimize-this-python-code-from-thinkpython-exercise-10-10/5523071#comment-6276679">this small example</a> of an algorithm in Python for looping through a large word list. I am writing a few "tools" that will allow my to slice a Objective-C string or array in a similar fashion as Python. </p>
  
  <p>Specifically, <a href="https://stackoverflow.com/questions/5523058/how-to-optimize-this-python-code-from-thinkpython-exercise-10-10/5523071#5523071">this elegant solution</a> caught my attention for executing very quickly and it uses a string slice as a key element of the algorithm. Try and solve this without a slice! </p>
  
  <p>I have reproduced my local version using the <a href="http://icon.shef.ac.uk/Moby/" rel="nofollow noreferrer">Moby word list</a> below. You can use <code>/usr/share/dict/words</code> if you do not feel like downloading Moby. The source is just a large dictionary-like list of unique words. </p>
  
  <pre><code>#!/usr/bin/env python
  
  count=0
  words = set(line.strip() for line in   
             open("/Users/andrew/Downloads/Moby/mwords/354984si.ngl"))
  for w in words:
      even, odd = w[::2], w[1::2]
      if even in words and odd in words:
          count+=1
  
  print count      
  </code></pre>
  
  <p>This script will a) be interpreted by Python; b) read the 4.1 MB, 354,983 word Moby dictionary file; c) strip the lines; d) place the lines into a set, and; e) and find all the combinations where the evens and the odds of a given word are also words. This executes in about 0.73 seconds on a MacBook Pro.</p>
  
  <p>I tried to rewrite the same program in Objective-C. I am a beginner at this language, so go easy please, but please do point out the errors.</p>
  
  <pre><code>#import &lt;Foundation/Foundation.h&gt;
  
  NSString *sliceString(NSString *inString, NSUInteger start, NSUInteger stop, 
          NSUInteger step){
      NSUInteger strLength = [inString length];
  
      if(stop &gt; strLength) {
          stop = strLength;
      }
  
      if(start &gt; strLength) {
          start = strLength;
      }
  
      NSUInteger capacity = (stop-start)/step;
      NSMutableString *rtr=[NSMutableString stringWithCapacity:capacity];    
  
      for(NSUInteger i=start; i &lt; stop; i+=step){
          [rtr appendFormat:@"%c",[inString characterAtIndex:i]];
      }
      return rtr;
  }
  
  NSSet * getDictWords(NSString *path){
  
      NSError *error = nil;
      NSString *words = [[NSString alloc] initWithContentsOfFile:path
                           encoding:NSUTF8StringEncoding error:&amp;error];
      NSCharacterSet *sep=[NSCharacterSet newlineCharacterSet];
      NSPredicate *noEmptyStrings = 
                       [NSPredicate predicateWithFormat:@"SELF != ''"];
  
      if (words == nil) {
          // deal with error ...
      }
      // ...
  
      NSArray *temp=[words componentsSeparatedByCharactersInSet:sep];
      NSArray *lines = 
          [temp filteredArrayUsingPredicate:noEmptyStrings];
  
      NSSet *rtr=[NSSet setWithArray:lines];
  
      NSLog(@"lines: %lul, word set: %lul",[lines count],[rtr count]);
      [words release];
  
      return rtr;    
  }
  
  int main (int argc, const char * argv[])
  {
      NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
  
      int count=0;
  
      NSSet *dict = 
         getDictWords(@"/Users/andrew/Downloads/Moby/mwords/354984si.ngl");
  
      NSLog(@"Start");
  
      for(NSString *element in dict){
          NSString *odd_char=sliceString(element, 1,[element length], 2);
          NSString *even_char=sliceString(element, 0, [element length], 2);
          if([dict member:even_char] &amp;&amp; [dict member:odd_char]){
              count++;
          }
  
      }    
      NSLog(@"count=%i",count);
  
      [pool drain];
      return 0;
  }
  </code></pre>
  
  <p>The Objective-C version produces the same result, (13,341 words), but takes almost 3 seconds to do it. I must be doing something atrociously wrong for a compiled language to be more than 3X slower than a scripted language, but I'll be darned if I can see why. </p>
  
  <p>The basic algorithm is the same: read the lines, strip them, and put them in a set. </p>
  
  <p>My guess of what is slow is the processing of the NSString elements, but I do not know an alternative.</p>
  
  <p><strong>Edit</strong></p>
  
  <p>I edited the Python to be this:</p>
  
  <pre><code>#!/usr/bin/env python
  import codecs
  count=0
  words = set(line.strip() for line in 
       codecs.open("/Users/andrew/Downloads/Moby/mwords/354984si.ngl",
            encoding='utf-8'))
  for w in words:
      if w[::2] in words and w[1::2] in words:
          count+=1
  
  print count 
  </code></pre>
  
  <p>For the utf-8 to be on the same plane as the utf-8 NSString. This slowed the Python down to 1.9 secs. </p>
  
  <p>I also switch the slice test to short-circuit type as <a href="https://stackoverflow.com/questions/5616847/why-is-this-program-faster-in-python-than-objective-c/5617487#5617487">suggested</a> for both the Python and obj-c version. Now they are close to the same speed. I also tried using C arrays rather than NSStrings, and this was much faster, but not as easy. You also loose utf-8 support doing that. </p>
  
  <p>Python is really cool...     </p>
  
  <p><strong>Edit 2</strong></p>
  
  <p>I found a bottleneck that sped things up considerably. Instead of using the <code>[rtr appendFormat:@"%c",[inString characterAtIndex:i]];</code> method to append a character to the return string, I used this:</p>
  
  <pre><code>for(NSUInteger i=start; i &lt; stop; i+=step){
      buf[0]=[inString characterAtIndex:i];
      [rtr appendString:[NSString stringWithCharacters:buf length:1]];
  }
  </code></pre>
  
  <p>Now I can <em>finally</em> claim that the Objective-C version is faster than the Python version -- but not by much.</p>