Brown dropped out of his medical course in 1793. Late in 1794, he enlisted in the Fifeshire Fencibles, and his regiment was posted to Ireland shortly after. In June 1795 he was appointed Surgeon's Mate. His regiment saw very little action, however, he had a good deal of leisure time, almost all of which he spent on botany. He was frustrated by his itinerant lifestyle, which prevented him from building his personal library and specimen collection as he would have liked, and cut him off from the most important herbaria and libraries.