Here are a few general guidelines for finding a good candidate for optimization:

It should be time-critical. This is generally any operation that is perceptibly slow; the user has to wait for the computer to finish doing something before continuing. Optimizing functionality that is already faster than the user can perceive is usually unnecessary.
It must be relevant. Optimizing functionality that is rarely used is usually counter-productive.
It shows up as a hot spot in a time profile. If there is no obvious hot spot in your code or you are spending a lot of time in system libraries, performance is more likely to improve through high-level improvements (architectural changes).