Exploring the Accuracy and User Experience of ChatGPT for Actinic Keratosis

To examine ChatGPT's reliability and utility, a new study conducted by researchers from Aalborg University and Bispebjerg Hospital asked the chatbot 38 questions about actinic keratosis. They then asked a panel of 7 dermatologists to review the answers to see if they were accurate, up-to-date, and complete. Finally, the panel was also asked what they thought about using ChatGPT.

The study found that ChatGPT only gave accurate, up-to-date, and complete answers to about 31.6% of the questions. While the chatbot was good at explaining risk factors and how actinic keratosis develops, it struggled with diagnosing and suggesting treatments. The panel of dermatologists especially noticed inaccuracies in the diagnostic workup and clinical management e.g. the latest treatment advice.

Although the chatbot was fast at giving answers, dermatologists thought ChatGPT wasn't accurate or clear enough for use in a clinical setting. This points to AI chatbots still needing improvement, especially for specialized areas like skin cancer. While ChatGPT may become a useful tool for patient education, there is a clear need to make AI models more reliable before they can be used by the broader medical field.

Future advancements in AI modeling, in addition to reliable training data, will be essential to making chatbots better in skin cancer diagnosis and treatment.