Background: Health care professionals need a valid tool to assess the physical ability of patients with chronic diseases. We aimed to assess the validity of the results of physical fitness tests estimated by a wrist wearable device in young adults and chronic disease people. Methods: Participants wore a sensor placed on their wrist and performed two physical fitness tests (sit to stand (STS) and time up and go (TUG)). We checked the concordance of sensor-estimated results using Bland–Altman analysis, root-mean-square error, and intraclass coefficient of correlation (ICC). Results: In total, 31 young adults (groups A; median age = 25 ± 5 years) and 14 people with chronic diseases (groups B; median age = 70 ± 15 years) were included. Concordance was high for both STS (ICCA= 0.95, and ICCB= 0.90), and TUG (ICCA= 0.75, ICCB= 0.98). The best estimations were given by the sensor during STS tests in young adults (mean bias = 0.19 ± 2.69;p= 0.12) and chronic disease people (mean bias = −0.14 ± 3.09 s;p= 0.24). The sensor provided the largest estimation errors over 2 s during the TUG test in young adults. Conclusion: This study showed that the results provided by the sensor are consistent with those of the gold standard during STS and TUG in both healthy youth and people with chronic diseases.