 <p>Three observations. </p> <ol> <li><p>The best speedups are not coming from optimizations but from <em>good algorithms</em>. So make sure you get that part right first. Often this means just using the right libraries for your specific domain. </p></li> <li><p>Once you get your algorithms right it is time to <em>Measure</em>. Often there is an 80/20 rule at work. 20% of your code will take 80% of the execution time. But in order to locate that part you need a good profiler. <a href="http://software.intel.com/en-us/articles/intel-vtune/" rel="nofollow">Intel VTune</a> can give you sampling profile from every function and nice reports that pinpoint the performance killers. Another free alternative is <a href="http://developer.amd.com/cpu/CodeAnalyst/Pages/default.aspx" rel="nofollow">AMD CodeAnalyst</a> if you have an AMD CPU.</p></li> <li><p>The compiler autovectorization capability is not a silver bullet. Although it will try really hard (especially <a href="http://software.intel.com/en-us/articles/intel-compilers/" rel="nofollow">Intel C++</a>) you will often need to help it by rewriting the algorithms in vector form. You can often get much better results by handcrafting small portions of the bottleneck code to use SIMD instructions. You can do that in C code (see VJo's link above) using intrinsics or use inline assembly. </p></li> </ol> <p>Of course parts 2 and 3 form an iterative process. If you are really serious about this then there are some good books on the subject by Intel folks such as <a href="http://rads.stackoverflow.com/amzn/click/0976483211" rel="nofollow">The Software Optimization Cookbook</a> and the processor reference manuals.</p>