others were simply forgotten , like the army towns of fort griffin and fort mckavett , which garrisoned black troops known as the buffalo soldiers ; both falling into decline after the forts were abandoned .