These are largely just different terms for the same things, but named differently to remind you of some important details. For a rectangle, we usually refer to the length and width; properly, these are the longer side and the shorter side, respectively, but that really makes no difference.
John Adams hated political parties, but he was backed by the Federalists and ran for President as a Federalist and won.   After Adams served, Thomas Jefferson, James Madison James Monroe served two terms each as members of the Democratic Republican party. By this time the Federalist Party all but disappeared.
The last President of the US who was neither Democrat or Republican was Millard Fillmore (1850-1853), who was a member of the Whig party. One could argue that the last (and only) President who wasn't a member of any political party was George Washington.s was also pointed out, the last non Republican or Democrat to become president was Millard Fillmore. You could also argue that John Tyler and Andrew Johnson (as mentioned by Andrew Stein) were not really affiliated with a party while they were in office.