In 1874 Nilson became professor of general and agricultural chemistry at Uppsala.
From then on he could devote more time to research.  He began working on rare earths such as euxenite and gadolinite, using methods of successive fractionation introduced by Nils Johan Berlin. One of his goals was to better understand the relationship of the newly discovered elements to the proposed periodic system.  
In 1879 Nilson separated out the oxide scandia. By doing so he discovered a new element, scandium, which he named for Scandinavia. The discovery was of particular importance because the existence of an element with such properties, "ekaboron", had been predicted by Mendeleev, based on his organization of the periodic system.  
Nilsons spectral analysis indicated a previously unreported pattern of lines. Per Theodor Cleve demonstrated the equivalence of Nilsons scandium and Mendeleev's hypothetical ekaboron.
Nilson and Sven Otto Pettersson were the first to isolate titanium metal in a relatively pure form, extracting a 95% pure sample in 1887. Jöns Jacob Berzelius had extracted titanium metal in 1825, but only in a very impure form. 
In 1910 Matthew Hunter was finally able to produce 99.9% pure titanium metal, through his new Hunter process.
Nilson also studied the gas density of metals which made it possible to determine the valence of various metals.