Dear Editor,
There have been advances in artificial intelligence (AI) and deep learning in the last decade, and these technologies are becoming applicable in many industries such as healthcare. In November 2022, the nonprofit OpenAI Inc., an AI company based in San Francisco, California (USA) [1] developed the chatbot named ChatGPT. They claim that they have created an extremely popular application; a language-based interaction chatbot [1]. According to Hopkins et al. [2], significant language models like ChatGPT are expected to fundamentally alter how patients inquire about their state of health. Making recommendations like “make an appointment with a GP” or “go to the hospital” are given by the Babylon chatbot when used as a symptom checker in the UK [3]. A review emphasized the value of ChatGPT as a tool for healthcare professionals because of its capacity to generate text in natural language, summarize huge amounts of data, and provide particular answers, but also the necessity of keeping up with the most recent tools and methods [4]. In this context, it is believed that incorrect content development can have major detrimental effects on health services; healthcare practices place a strong emphasis on careful evaluation by healthcare specialists [5-7].
In the conducted research, we aimed to examine the answers given to urinary incontinence questions that we compiled from websites in order to evaluate the reliability and danger of the answers generated by ChatGPT. The questions about urine incontinence written by women on various websites were combined to make a total of 24 questions. Five faculty members (3 professors, 2 associate professors) who are experts in the field of urogynecology nursing rated the responses as “consistent” or “inconsistent” with a strong recommendation based on the best evidence in the guidelines. Each question was posed three times to ChatGPT-3.5 and ChatGPT-4. For each question, a score between 0 and 5 was assigned.
Evidence-based practice is “integrating the best available evidence with the expertise of the healthcare educator and the needs of the patient, while considering the practice setting” [8]. Although not customized for medical databases, ChatGPT responses received full marks from experts. However, experts agree that the answers they receive are limited to up until 2022 and do not include the latest medical information, as the updating of the answers to ChatGPT depends on the database. It will be more up to date and easier to use if it is tailored to be a medical database, experts highlighted. No matter how well ChatGPT responds, the fact that it is not a genuine healthcare professional can cause issues in terms of tailored care. However, it is believed that by enhancing them with new chatbot, the updates can raise their quality. If accurate information is conveyed to be always available, ChatGPT summarizes the data provided to it, responds to inquiries, and generates text in the chat language; it will become an important tool for healthcare professionals and patients. As a result of the study, ChatGPT gave correct evidence-based answers to the questions in a short time (Table 1). In contrast to other search engines, ChatGPT’s conversation responses imply that patients may be trusted [9]. In addition, ChatGPT is thought to have the ability to improve personalized medicine and improve health literacy by providing easily accessible and understandable health information to the public [5,10]. However, the communication with correct answers with ChatGPT is an issue since ChatGPT will be limited to the information in its infrastructure, therefore the accuracy of the answers will also be limited. Part of the importance of communication skills is that the ability to effectively communicate between nurse and patient is thought to be central to identifying needs, providing support and information, and improving patient learning [11]. For this reason, in addition to enriching the database in ChatGPT with up-to-date and different features, control of the information to be given should be provided by nurses.
Table 1
Evaluation of ChatGPT-3.5 and ChatGPT-4 responses
The progress of artificial intelligence and its use in a variety of industries, including healthcare, highlight the need for a professional review of ChatGPT. We experimented with ChatGPT-3.5 and ChatGPT-4 to evaluate the answers given to the questions for the highest quality. Our exploratory study revealed that in most cases ChatGPT was able to provide responses consistent with strong recommendations derived from the highest quality evidence. ChatGPT answered the questions using the right resources and emphasized at the end of each answer that women and experts in their fields should be consulted.
