Rinderknecht, Emily and von Winning, Dominik and Kravchuk, Anton and Schaefer, Christof and Schnabel, Marco Julius and Siepmann, Stephan and Mayr, Roman and Grassinger, Jochen and Gossler, Christopher and Pohl, Fabian and Siska, Peter J. and Zeman, Florian and Breyer, Johannes and Schmelzer, Anna and Gilfrich, Christian and Brookman-May, Sabine D. and Burger, Maximilian and Haas, Maximilian and May, Matthias (2024) Modification and Validation of the System Causability Scale Using AI-Based Therapeutic Recommendations for Urological Cancer Patients: A Basis for the Development of a Prospective Comparative Study. CURRENT ONCOLOGY, 31 (11). pp. 7061-7073. ISSN 1198-0052, 1718-7729
Full text not available from this repository. (Request a copy)Abstract
The integration of artificial intelligence, particularly Large Language Models (LLMs), has the potential to significantly enhance therapeutic decision-making in clinical oncology. Initial studies across various disciplines have demonstrated that LLM-based treatment recommendations can rival those of multidisciplinary tumor boards (MTBs); however, such data are currently lacking for urological cancers. This preparatory study establishes a robust methodological foundation for the forthcoming CONCORDIA trial, including the validation of the System Causability Scale (SCS) and its modified version (mSCS), as well as the selection of LLMs for urological cancer treatment recommendations based on recommendations from ChatGPT-4 and an MTB for 40 urological cancer scenarios. Both scales demonstrated strong validity, reliability (all aggregated Cohen's K > 0.74), and internal consistency (all Cronbach's Alpha > 0.9), with the mSCS showing superior reliability, internal consistency, and clinical applicability (p < 0.01). Two Delphi processes were used to define the LLMs to be tested in the CONCORDIA study (ChatGPT-4 and Claude 3.5 Sonnet) and to establish the acceptable non-inferiority margin for LLM recommendations compared to MTB recommendations. The forthcoming ethics-approved and registered CONCORDIA non-inferiority trial will require 110 urological cancer scenarios, with an mSCS difference threshold of 0.15, a Bonferroni corrected alpha of 0.025, and a beta of 0.1. Blinded mSCS assessments of MTB recommendations will then be compared to those of the LLMs. In summary, this work establishes the necessary prerequisites prior to initiating the CONCORDIA study and validates a modified score with high applicability and reliability for this and future trials.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | AGREEMENT; COEFFICIENT; QUALITY; artificial intelligence integration; large language models; multidisciplinary tumor boards; non-inferiority CONCORDIA Trial; SCS; urological cancer treatment; validation study; clinical decision support; artificial neural network |
| Subjects: | 600 Technology > 610 Medical sciences Medicine |
| Divisions: | Medicine > Lehrstuhl für Innere Medizin III (Hämatologie und Internistische Onkologie) Medicine > Lehrstuhl für Strahlentherapie Medicine > Lehrstuhl für Urologie Medicine > Zentren des Universitätsklinikums Regensburg > Zentrum für Klinische Studien |
| Depositing User: | Dr. Gernot Deinzer |
| Date Deposited: | 26 Jan 2026 07:19 |
| Last Modified: | 26 Jan 2026 07:19 |
| URI: | https://pred.uni-regensburg.de/id/eprint/64119 |
Actions (login required)
 |
View Item |
