Elders have a higher rate of return visits to the emergency department (ED) than other patients. It is critical to understand the risk factors for return visits to the ED by elders. The aim of this study was to determine the factors associated with return visits to the ED by elders. This study retrospectively reviewed the hospital charts of elders who returned to the ED within 72 h after discharge from ED. The risk factors identified in the Triage Risk Screening Tool were applied in this study. Of the elders discharged from the ED, 8.64% made a return visit to the ED within 72 h. The highest revisit rate occurred within 24 h after discharge. Factors associated with return ED visits within 24 h by elders were difficulty walking and having discharge care needs. The factor associated with ED return visits within 24–48 h was polypharmacy. Difficulty walking, having discharge care needs, and hospitalization within the past 120 days were associated with return visits made within 48–72 h following discharge. Identifying the reasons for return visits to the ED and providing a continuous review of geriatric assessment and discharge planning could reduce unnecessary revisits.