Hello! I have the exact same problem, although I do have some crusting and itching once in a while in the same place behind my right ear. What I do is when I shower in the morning, I take a pair of rough bath gloves, soap them up, and gently scrape away the dead skin.
When they are absorbed, they can be released in sweat through skin. Foods like yogurt, onions, garlic, red meat, cabbage, spices like coriander, and asparagus may result in increased body odor when consumed in large amounts. When you have a diet rich in proteins, you may have vinegar like smell on sweat.
As this quote from wikianswers puts it. you might smell of ammonia due to the presence of bacteria on your skin hydrolyzing the urea in your sweat. Drinking more water or wiping your sweat off more often may solve the quandary.