Comment on “The new frontier: utilizing ChatGPT to expand craniofacial research”

Article information

Arch Craniofac Surg. 2024;25(4):205-206

Publication date (electronic) : 2024 August 20

doi : https://doi.org/10.7181/acfs.2024.00416

Hinpetch Daungsupawong^,1, Viroj Wiwanitkit²

¹Private Academic Consultant, Phonhong, Lao People’s Democratic Republic

²Department of Research Analytics, Saveetha Dental College and Hospitals, Saveetha Institute of Medical and Technical Sciences Saveetha University, Chennai, India

Correspondence: Hinpetch Daungsupawong Private Academic Consultant, Phonhong, Lao People’s Democratic Republic E-mail: hinpetchdaung@gmail.com

Received 2024 July 15; Revised 2024 July 15; Accepted 2024 August 10.

To the Editor:

We would like to discuss some points from the recently published article, “The new frontier: utilizing ChatGPT to expand craniofacial research” [1]. This study aimed to evaluate ChatGPT’s effectiveness in generating 20 novel concepts for systematic reviews across ten different subspecialties within craniofacial surgery. The findings indicate a total accuracy rate of 57.5%, with general themes achieving a lower accuracy of 39%. However, for specific themes, the accuracy exceeded 76%. These results suggest that ChatGPT is capable of generating precise and detailed research proposals. Nonetheless, challenges remain in expanding the scope of concepts within this field.

This study indicates that ChatGPT’s overall accuracy in generating concepts for systematic reviews falls below expectations. This may be due to the fact that formulating research questions in craniofacial surgery demands a high degree of clarity and complexity. Additionally, algorithms may struggle to grasp the broader context of research issues, leading to concepts that are both more complex and less precise.

A methodological weakness of the study may be its reliance on only four databases for the review: PubMed, CINAHL, Embase, and Cochrane—even though these are reputable medical resources. This exclusion of other databases and additional data sources could diminish the overall comprehensiveness of the review. Incorporating additional databases or search techniques might enable a more complete assessment of the research ideas generated.

This study calls into question the accuracy and reliability of artificial intelligence (AI) algorithms, such as ChatGPT, in generating research ideas in specialized fields like craniofacial surgery. While the algorithm demonstrates considerable accuracy in generating specific concepts, its failure to produce broader concepts underscores the necessity for improvements in and deeper understanding of context-specific research topics. Future investigations might explore ways to enhance the algorithm’s ability to generate more accurate and appropriate general research concepts.

Overall, this study offers new insights into the use of AI for generating research ideas in the field of craniofacial surgery. Specifically, ChatGPT has demonstrated its capability to generate ideas for systematic reviews. However, there is still potential for improvement in formulating more general and context-relevant research questions. Future efforts could focus on enhancing our understanding of algorithms in specialized fields, integrating additional databases or resources for literature reviews, and exploring methods to increase the accuracy and reliability of AI-generated research ideas.