section_path: Applications/Computer science
section_name: Computer science
paragraph_text: Computer science
  
  One study analyzed ChatGPT's responses to 517 questions about software engineering or computer programming posed on Stack Overflow for correctness, consistency, comprehensiveness, and concision. It found that 52% of the responses contained inaccuracies and 77% were verbose. Another study, focused on the performance of GPT-3.5 and GPT-4 between March and June 2024, found that performance on objective tasks like identifying prime numbers and generating executable code was highly variable.
  
  ChatGPT was able in 2023 to provide useful code for solving numerical algorithms in limited cases. In one study, it produced solutions in C, C++, Python, and MATLAB for problems in computational physics. However, there were important shortfalls like violating basic linear algebra principles around solving singular matrices and producing matrices with incompatible sizes.
  
  In December 2022, the question-and-answer website Stack Overflow banned the use of ChatGPT for generating answers to questions, citing the factually ambiguous nature of its responses. In January 2023, the International Conference on Machine Learning banned any undocumented use of ChatGPT or other large language models to generate any text in submitted papers. Samsung banned generative AI company-wide in May 2023 after sensitive material was uploaded to ChatGPT.