section_path: Applications/Medicine
section_name: Medicine
paragraph_text: Medicine
  
  In the field of health care, possible uses and concerns are under scrutiny by professional associations and practitioners. Two early papers indicated that ChatGPT could pass the United States Medical Licensing Examination (USMLE). MedPage Today noted in January 2023 that "researchers have published several papers now touting these AI programs as useful tools in medical education, research, and even clinical decision making."
  
  Published in February 2023, were two separate papers that again evaluated ChatGPT's proficiency in medicine using the USMLE. Findings were published in JMIR Medical Education and PLOS Digital Health. The authors of the PLOS Digital Health paper stated that the results "suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making." In JMIR Medical Education, the authors of the other paper concluded that "ChatGPT performs at a level expected of a third-year medical student on the assessment of the primary competency of medical knowledge." They suggest that it could be used as an "interactive learning environment for students". The AI itself, prompted by the researchers, concluded that "this study suggests that ChatGPT has the potential to be used as a virtual medical tutor, but more research is needed to further assess its performance and usability in this context." The later-released ChatGPT version based on GPT-4 significantly outperformed the version based on GPT-3.5. Researchers at Stanford University and the University of California, Berkeley have found that the performance of GPT-3.5 and GPT-4 on the USMLE declined from March 2023 to June 2023.
  
  A March 2023 paper tested ChatGPT's application in clinical toxicology. The authors found that the AI performed well when answering straightforward clinical case questions that were "unlikely to be missed by any practitioner in the field". They added: "As ChatGPT becomes further developed and specifically adapted for medicine, it could one day be useful in less common clinical cases." They expected to see more clinicians using AI, and did not expect to see AI replacing clinicians.
  
  An April 2023 study in Radiology tested the AI's ability to answer queries about breast cancer screening. The authors found that it answered appropriately "about 88 percent of the time", however, in one case (for example), it gave advice that had become outdated about a year earlier. The comprehensiveness of its answers was also lacking. A study published in JAMA Internal Medicine that same month found that ChatGPT often outperformed answers found at /r/AskDocs, a forum on Reddit where moderators validate the medical credentials of professionals. The study authors suggest that the tool could be integrated with medical systems to help doctors draft responses to patient questions.
  
  In correspondence to The Lancet Infectious Diseases, three antimicrobial experts wrote that "the largest barriers to the implementation of ChatGPT in clinical practice are deficits in situational awareness, inference, and consistency. These shortcomings could endanger patient safety." Physician's Weekly, though also discussing the potential use of ChatGPT in medical contexts (e.g., "as a digital assistant to physicians by performing various administrative functions like gathering patient record information or categorizing patient data by family history, symptoms, lab results, possible allergies, et cetera"), warned that the AI might sometimes provide fabricated or biased information. One radiologist warned: "We've seen in our experience that ChatGPT sometimes makes up fake journal articles or health consortiums to support its claims"; As reported in one Mayo Clinic Proceedings: Digital Health paper, ChatGPT may do this for as much as 69% of its cited medical references. The researchers emphasized that while many of its references were fabricated, those that were appeared "deceptively real". However, according to Dr. Stephen Hughes, ChatGPT is capable of learning to correct its past mistakes. He also noted the AI's prudishness regarding sexual health topics.
  
  Contrary to previous findings, ChatGPT responses to anesthesia -related questions were more accurate, succinct, and descriptive compared to Bard's. Bard exhibited 30.3% error rate in its responses, while ChatGPT had a 0% error rate. At a conference of the American Society of Health-System Pharmacists in December 2023, researchers at Long Island University (LIU) presented a study that researched ChatGPT's responses to 45 frequently asked questions of LIU College of Pharmacy's drug information service during a 16-month period from 2022 to 2023 as compared with researched responses provided by professional pharmacists. For 29 of the 39 questions for which there was sufficient medical literature for a data-driven response, ChatGPT failed to provide a direct answer or provided a wrong or incomplete answer (and in some cases, if acted upon, the answer would endanger the patient's health). The researchers had asked ChatGPT to provide medical research citations for all its answers, but it did so for only eight, and all eight included at least one fake citation.
  
  A January 2024 study conducted by researchers at Cohen Children's Medical Center found that GPT-4 had an accuracy rate of 17% when diagnosing pediatric medical cases. A November 2024 study of 50 physicians on illness diagnosis reported that GPT-4 achieved a 90% accuracy, while physicians scored 74% without AI assistance, and 76% when using the chatbot.