Johnston was survived by his wife Eliza and six children. His wife and five younger children, including one born after he went to war, chose to live out their days at home in Los Angeles with Eliza's brother, Dr. John Strother Griffin. Johnston's eldest son, Albert Sidney Jr. (born in Texas), had already followed him into the Confederate States Army. In 1863, after taking home leave in Los Angeles, Albert Jr. was on his way out of San Pedro harbor on a ferry. While a steamer was taking on passengers from the ferry, a wave swamped the smaller boat, causing its boilers to explode. Albert Jr. was killed in the accident.