eISSN: 2084-9869
ISSN: 1233-9687
Polish Journal of Pathology
Current issue Archive Manuscripts accepted About the journal Supplements Editorial board Abstracting and indexing Subscription Contact Instructions for authors Publication charge Ethical standards and procedures
Editorial System
Submit your Manuscript
SCImago Journal & Country Rank
Search
Editorial Policies
Sarajevo Declaration on Integrity and Visibility of Scholarly Publications

Open access

without publication fees
3/2024
vol. 75
 
Share:
Send email
Share:
abstract:
Original paper

An investigative analysis – ChatGPT’s capability to excel in the Polish speciality exam in pathology

Michał Bielówka
1
,
Jakub Kufel
2
,
Marcin Rojek
1
,
Dominika Kaczyńska
1
,
Łukasz Czogalik
1
,
Adam Mitręga
1
,
Wiktoria Bartnikowska
3
,
Dominika Kondoł
4
,
Kacper Palkij
4
,
Sylwia Mielcarska
5

  1. Students’ Scientific Association of Computer Analysis and Artificial Intelligence at the Department of Radiology and Nuclear Medicine, Medical University of Silesia, Katowice, Poland
  2. Department of Radiology and Nuclear Medicine, Medical University of Silesia, Katowice, Poland
  3. Faculty of Medical Sciences in Katowice, Medical University of Silesia, Katowice, Poland
  4. Dr B. Hager Memorial Multi-Specialty District Hospital, Tarnowskie Góry, Poland
  5. Department of Medical and Molecular Biology, Faculty of Medical Sciences in Zabrze, Medical University of Silesia, Zabrze, Poland
Pol J Pathol 2024; 75 (3): 236-240
DOI: https://doi.org/10.5114/pjp.2024.143091
Online publish date: 2024/09/20
View full text Get citation
 
PlumX metrics:
This study evaluates the effectiveness of the ChatGPT-3.5 language model in providing correct answers to pathomorphology questions as required by the State Speciality Examination (PES). Artificial intelligence (AI) in medicine is generating increasing interest, but its potential needs thorough evaluation. A set of 119 exam questions by type and subtype were used, which were posed to the ChatGPT-3.5 model. Performance was analysed with regard to the success rate in different question categories and subtypes.

ChatGPT-3.5 achieved a performance of 45.38%, which is significantly below the minimum PES pass threshold. The results achieved varied by question type and subtype, with better results in questions requiring “comprehension and critical thinking” than “memory”.

The analysis shows that, although ChatGPT-3.5 can be a useful teaching tool, its performance in providing correct answers to pathomorphology questions is significantly lower than that of human respondents. This conclusion highlights the need to further improve the AI model, taking into account the specificities of the medical field. Artificial intelligence can be helpful, but it cannot fully replace the experience and knowledge of specialists.
keywords:

pathomorphology, artificial intelligence, language model, ChatGPT-35, specialty examination
Termedia
About us Contact
Copyright notice Privacy policy Advertising policy Contact us
Quick links
Journals Books eBooks Events
© 2024 Termedia Sp. z o.o.
Developed by Bentus.