Exploring the performance of ChatGPT-3.5 in addressing dermatological queries: a research investigation into AI capabilities

Introduction:
In the 21^st century’s era of rapid technological advancement, ChatGPT-3.5, an artificial intelligence (AI) language model, is scrutinized for its application in dermatology. Using 119 questions from the National Specialist Examination (PES), we assess ChatGPT-3.5’s performance by comparing it to human skills and addressing ethical implications.

Objective:
Our primary aim is to evaluate ChatGPT-3.5’s proficiency in responding to 119 dermatology questions from the PES. The study emphasizes ethical considerations and compares the model’s knowledge and skills to those of human dermatologists.

Material and methods:
Utilizing the 2023 PES question database, questions were categorized by Bloom’s taxonomy and thematic content. ChatGPT-3.5, version of 3 August 2023, answered 119 questions in five sessions, allowing for a probabilistic evaluation. Statistical analyses, conducted using R Studio, assessed correctness, confidence, and difficulty.

Results:
ChatGPT-3.5 achieved a 49.58% correct response rate, below the 60% passing threshold. No significant differences in difficulty or correlations between difficulty and certainty were observed. Varied performance across question types highlighted strengths and weaknesses. Despite suboptimal results, ChatGPT-3.5’s differential performance offers insights, suggesting future improvements. The study advocates for ongoing research into AI integration in dermatology, envisioning a promising role for AI in assisting dermatologists.

Conclusions:
Ethical considerations are crucial for effective AI introduction, minimizing errors, and enhancing dermatological healthcare quality, fostering optimism for AI’s evolving role in dermatology.