Skip to main content

Advertisement

SpringerLink
Log in

ChatGPT and most frequent urological diseases: analysing the quality of information and potential risks for patients

  • Original Article
  • Published:
World Journal of Urology Aims and scope Submit manuscript

A Letter to the Editor to this article was published on 26 February 2024

A Letter to the Editor to this article was published on 26 September 2023

A Letter to the Editor to this article was published on 26 September 2023

Abstract

Purpose

Artificial intelligence (AI) is a set of systems or combinations of algorithms, which mimic human intelligence. ChatGPT is software with artificial intelligence which was recently developed by OpenAI. One of its potential uses could be to consult the information about pathologies and treatments. Our objective was to assess the quality of the information provided by AI like ChatGPT and establish if it is a secure source of information for patients.

Methods

Questions about bladder cancer, prostate cancer, renal cancer, benign prostatic hypertrophy (BPH), and urinary stones were queried through ChatGPT 4.0. Two urologists analysed the responses provided by ChatGPT using DISCERN questionary and a brief instrument for evaluating the quality of informed consent documents.

Results

The overall information provided in all pathologies was well-balanced. In each pathology was explained its anatomical location, affected population and a description of the symptoms. It concluded with the established risk factors and possible treatment. All treatment answers had a moderate quality score with DISCERN (3 of 5 points). The answers about surgical options contain the recovery time, type of anaesthesia, and potential complications. After analysing all the responses related to each disease, all pathologies except BPH achieved a DISCERN score of 4.

Conclusions

ChatGPT information should be used with caution since the chatbot does not disclose the sources of information and may contain bias even with simple questions related to the basics of urologic diseases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data sharing

The questions and answers of ChatGPT can be consulted in Open Science Framework repository (https://doi.org/10.17605/OSF.IO/8UNQV). The correspondence author will provide data from valid questionarries upon reasoned request.

References

  1. OpenAI Help Center ChatGPT general FAQ. https://help.openai.com/en/articles/6783457-chatgpt-general-faq. Accessed 9 Mar 2023

  2. Fraser H, Coiera E, Wong D (2018) Safety of patient-facing digital symptom checkers. Lancet 392(10161):2263–2264. https://doi.org/10.1016/S0140-6736(18)32819-8

    Article  PubMed  Google Scholar 

  3. Loeb S, Sengupta S, Butaney M et al (2019) Dissemination of misinformative and biased information about prostate cancer on YouTube. Eur Urol 75(4):564–567. https://doi.org/10.1016/j.eururo.2018.10.056

    Article  PubMed  Google Scholar 

  4. Loeb S, Reines K, Abu-Salha Y et al (2021) Quality of bladder cancer information on YouTube. Eur Urol 79(1):56–59. https://doi.org/10.1016/j.eururo.2020.09.014

    Article  PubMed  Google Scholar 

  5. Betschart P, Pratsinis M, Müllhaupt G et al (2020) Information on surgical treatment of benign prostatic hyperplasia on YouTube is highly biased and misleading. BJU Int 125(4):595–601. https://doi.org/10.1111/bju.14971

    Article  PubMed  Google Scholar 

  6. Biswas SS (2023) Role of ChatGPT in Public Health. Ann Biomed Eng. https://doi.org/10.1007/s10439-023-03172-7. (published online ahead of print, 2023 Mar 15)

    Article  PubMed  Google Scholar 

  7. Baclic O, Tunis M, Young K, Doan C, Swerdfeger H, Schonfeld J (2020) Challenges and opportunities for public health made possible by advances in natural language processing. Can Commun Dis Rep 46(6):161–168. https://doi.org/10.14745/ccdr.v46i06a02

    Article  PubMed  PubMed Central  Google Scholar 

  8. Millenson ML, Baldwin JL, Zipperer L, Singh H (2018) Beyond Dr Google: the evidence on consumer-facing digital tools for diagnosis. Diagnosis (Berl) 5(3):95–105. https://doi.org/10.1515/dx-2018-0009

    Article  PubMed  Google Scholar 

  9. Szczesniewski JJ, Tellez Fouz C, Ramos Alba A, Garcia Tello A, Diaz Goizueta FJ, Llanes Gonzalez L (2023) Answers of ChatGPT to questions about urologic diseases. OSF (Database). https://doi.org/10.17605/OSF.IO/8UNQV

  10. Charnock D, Shepperd S, Needham G, Gann R (1999) DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 53(2):105–111. https://doi.org/10.1136/jech.53.2.105

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Spatz ES, Suter LG, George E et al (2020) An instrument for assessing the quality of informed consent documents for elective procedures: development and testing. BMJ Open 10(5):e033297. https://doi.org/10.1136/bmjopen-2019-033297

    Article  PubMed  PubMed Central  Google Scholar 

  12. Rivas JG, Socarrás MR, Blanco LT (2016) Social media in urology: opportunities, applications, appropriate use and new horizons. Cent Eur J Urol 69(3):293–298. https://doi.org/10.5173/ceju.2016.848

    Article  Google Scholar 

  13. Haug CJ, Drazen JM (2023) Artificial Intelligence and machine learning in clinical medicine. N Engl J Med 388(13):1201–1208. https://doi.org/10.1056/NEJMra2302038

    Article  CAS  PubMed  Google Scholar 

  14. Lee P, Bubeck S, Petro J (2023) Benefits, limits, and risks of GPT-4 as an AI Chatbot for medicine. N Engl J Med 388(13):1233–1239. https://doi.org/10.1056/NEJMsr2214184

    Article  PubMed  Google Scholar 

  15. Sng GGR, Tung JYM, Lim DYZ, Bee YM (2023) Potential and pitfalls of ChatGPT and natural-language artificial intelligence models for diabetes education. Diabetes Care. https://doi.org/10.2337/dc23-0197. (published online ahead of print, 2023 Mar 15)

    Article  PubMed  Google Scholar 

  16. Fode M, Jensen CFS, Østergren PB (2021) How should the medical community respond to the low quality of medical information on social media? Eur Urol 79(1):60–61. https://doi.org/10.1016/j.eururo.2020.09.050

    Article  PubMed  Google Scholar 

  17. Singhal K, Azizi S, Tu T et al (2022) Large language models encode clinical knowledge. http://arxiv.org/abs/2212.13138. Accessed 30 Mar 2023

  18. Whiles BB, Bird VG, Canales BK, DiBianco JM, Terry RS (2023) Caution! AI bot has entered the patient chat: ChatGPT has limitations in providing accurate urologic healthcare advice. Urology. https://doi.org/10.1016/j.urology.2023.07.010

    Article  PubMed  Google Scholar 

Download references

Funding

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Author information

Authors and Affiliations

  1. Department of Urology, Getafe University Hospital, Carretera Madrid-Toledo km 12,500 Getafe, 28905, Madrid, Spain

    Juliusz Jan Szczesniewski, Carlos Tellez Fouz, Francisco Javier Diaz Goizueta, Ana García Tello & Luis Llanes González

  2. Faculty of Medicine, University of Salamanca, C. Alfonso X el Sabio, s/n, 37007, Salamanca, Spain

    Juliusz Jan Szczesniewski

  3. DXC Technology, C. de José Echegaray, Las Rozas, 28232, Madrid, Spain

    Alejandra Ramos Alba

  4. Rey Juan Carlos University, P. de los Artilleros 38, 28032, Madrid, Spain

    Alejandra Ramos Alba

  5. Francisco de Vitoria University, Carretera Pozuelo a Majadahonda, Km 1.800, 28223, Madrid, Spain

    Luis Llanes González

Authors
  1. Juliusz Jan Szczesniewski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Tellez Fouz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alejandra Ramos Alba
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francisco Javier Diaz Goizueta
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ana García Tello
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Luis Llanes González
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JJS: Project development, data collection and analysis, manuscript writing. CTF: data collection. ARA: data analysis, manuscript writing. AGT: Manuscript writing. FJDG: Manuscript writing. LLG: Project development, manuscript editing, final approval.

Corresponding author

Correspondence to Juliusz Jan Szczesniewski.

Ethics declarations

Conflict of interest

The authors declare no conflict of interests.

Ethics approval and informed consent

No human patients were involved in the study. No need Ethics Committee was required.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 16 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Szczesniewski, J.J., Tellez Fouz, C., Ramos Alba, A. et al. ChatGPT and most frequent urological diseases: analysing the quality of information and potential risks for patients. World J Urol 41, 3149–3153 (2023). https://doi.org/10.1007/s00345-023-04563-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00345-023-04563-0

Keywords

Search

Navigation