Ellen's itinerant life was over after her husband's death, and she settled permanently in Clifton, where she, James Jr. and Rolinda rented an apartment. The three members of the family concentrated on establishing a portrait practice. Rolinda's career took on a different and more ambitious direction, for she began to paint large portraits and complicated group scenes in oils. James, Jr., who lived a more independent life from his mother and sister, continued to paint portraits. Ellen's diary references to Felix end in 1823, and she made no mention that her children ever married. The family was successful, but both her children predeceased her. Rolinda died of breast cancer in 1838, and James Jr. died of tuberculosis in 1839. Of her loss, Ellen wrote to her friend, Miss Sarjeant: