Regulatory Needs for Radiation Protection Devices based upon Artiﬁcial Intelligence - State task or leave unregulated?

Artiﬁcial intelligence (AI) is increasingly employed in radiation protection, encompassing both medical devices and software. These technologies are integrated with AI throughout their manufacturing and application processes. This article underscores the imperative for comprehensive regulation in the utilization of AI. Decisions regarding AI application should not solely rest with manufacturers, medical professionals, or patients. Instead, an overarching "neutral" authority must be engaged to regulate, review, and enforce adherence to established protocols. The authors contend that relying on "self-regulation" within the free market, absent clear guidelines, proves to be inadequately e ﬀ ective and leads to patient's radiation protection safety issues.


Introduction
The differentiation between Artificial Intelligence (AI), machine learning (ML), and deep learning (DL) is initially irrelevant for radiation protection in radiology (8).It is important to emphasize that any application of such methods in radiation protection inevitably leads to a some-what ambiguous responsibility in the event of errors.Critical aspects in the future application of this technology will be eg potential for mass manipulation, but also substantial productivity increases (2).
As an example, consider the AI algorithm used in computed tomography (CT) scanners, which calculates radiation exposure based on the provided protocol data and patientspecific information, and also releases it during the examination.Radiology, as a service provider within interconnected modern medicine, must not be overlooked when evaluating patient outcomes and patient's radiation exposures.The capacity for improvement can be achieved, in our belief, through enhanced facilities and increased productivity.One tool that offers significant opportunities is information technology (IT) based on artificial intelligence (AI).To date, AI research in radiology has primarily focused on image analysis (2).The marketing of healthcare devices enabled for use with artificial intelligence (AI) or machine learning (ML) is regulated in the US by the US Food and Drug Administration (FDA), which is responsible for approving and regulating medical devices.Presently, there are no consistent guidelines established by the FDA to regulate AI-or ML-enabled medical devices, and the disparities between FDAapproved indications for use and device marketing necessitate clarification (3).

FDA Approval
Guidelines and manuals governing adherence and regulation of items approved for use are readily accessible to the public for all devices under FDA jurisdiction, with the notable exception of most AIor ML-enabled software and software considered as medical devices.Instead, the Good Machine Learning Practice for Medical Device Development Guiding Principles fills this gap, delineating 10 guiding principles.While widely embraced by AI and ML developers, these principles are not presented as rigid mandates (Tab. 1) (3,12).

Challenges of AI in Radiology
The rationale underlying the design of AI must also be taken into account, as certain systems can be programmed to behave unethically.For instance, Uber's algorithmic tool 'Greyball' was developed to predict which ride-hailers might be undercover law enforcement officers, enabling the company to 1 Corresponding author: stefanie.garni@insel.ch-received: 20.01.2024 -published: 16.02.2024detect and evade local regulations.Similarly, Volkswagen's algorithm enabled vehicles to pass emissions tests by reducing their nitrogen oxide emissions during testing.Likewise, designers in the private sector tasked with creating AI algorithms for clinical applications may face similar ethical dilemmas, potentially programming AI systems to steer users towards clinical actions that would boost profits for their purchasers (such as by recommending medications, tests, or medical devices in which they have a financial interest or by modifying referral patterns), without necessarily ensuring better patient care (9).Another concern for policymakers is timing.Presently, companies recognize the potential machine / deep learning and are consistently amassing new forms of data for analysis and exploitation.In a rapidly evolving and unpredictable environment such as the technology sector and AI, regulations must be promptly enacted to remain pertinent (9,10).

Examples of AI-based radiation protection devices still to be regulated in Switzerland
"Simple" Questions All components of a medical device in radiation protection should be labeled or assessed regarding the use of AI.This applies to both hardware and software.As an example, consider the user manual of any X-ray device.
In Switzerland, with its four official languages, user manuals are required to be available at least in French, German, and Italian.If the manufacturer of a certain X-ray device chooses to have it's user manual translated, for instance, by ChatGPT®, the following simple questions arise, among others: -Should it be labeled as a "AI-driven translation"?-Who bears responsibility when dubious translations result in user errors and incorrect radiation protection for patients?-Are there quality control measures implemented by the manufacturer?What are the protocols?-If quality control protocols exist, who oversees their implementation?Humans or AI?
These questions above mentioned among many others are the so called "easy ones" to answer.Of course the correct answers depend on the insight and the striking will of the politics and the earnest pursuit towards implementation by the officially responsible authorities which would have substantially more of hard administrative workload than ever before.

Key Inquiries (not simple)
The complexity and criticality of these questions escalate significantly when AI-based software is responsible for determining the dosage of radiation administered to individual patients during X-ray or CT examinations.This intricate matter poses considerable challenges due to liability concerns, responsibilities, and other technical inquiries.AI-based software fundamentally resembles a human brain.As per its intrinsic nature and the original definition AI is capable of autonomously deriving novel conclusions based on input, environmental interpretation, and continuous learning.This process enables the AI to execute direct actions independently, comparable to a child's brain.Only the AI operates and learns at lightning speed, devoid of any moral constraints.
Hence, in the context of radiation protection measures concerning autonomous AI-based X-ray devices, we pose the following key inquiries: The manufacturer?
The operator?
The radiologist?
The patient?The X-ray device with AI?
-Are control measures in place to prevent the AI software from overriding accurate information regarding the actual dose of radiation administered?
-How can we develop step-by-step protocols to prevent AI from circumventing control measures?

Ethical Considerations (FAT) paradigm of AI ethics: -Fairness -Accountability -Transparency
Many ethical concerns related to AI in healthcare can be grouped into three main categories: fairness, accountability, and transparency.This has led some researchers to discuss the fairness, accountability, and transparency (FAT) paradigm of AI ethics.While it may not be feasible or desirable to eliminate all biases, it is essential to acknowledge them and consider their implications for both machine and human decision-making processes.Biases can arise from the utilization of datasets that either overrepresent, underrepresent, or entirely miss certain relevant characteristics for the task at hand.Additionally, there is a risk of "automation bias", where humans begin to overly rely on machine outputs without exercising their own critical judgment and scrutiny.Furthermore, there is the potential for spurious correlations that, if acted upon, could result in harm to patients (4,5).
Presently, there is limited experience employing AI for patient care across its various demanding and diverse settings.Extensive research is still needed to comprehend the optimal utilization of AI in clinical practice and the essential operational characteristics it should possess.The approach to addressing these issues will be influenced not only by technical factors but also by the ethical considerations within the community.Individuals engaged in any phase of an AI product's life cycle must possess a thorough understanding of it.It is our responsibility to comprehend the risks associated with the products we utilize, to communicate any potential pitfalls to patients and stakeholders as necessary, and to continuously monitor AI products to mitigate harm.Moreover, we are obligated to ensure not only that the overall use of the product is beneficial but also that the distri-bution of benefits among potential stakeholders is fair and equitable (5).

Recommendations
AI in healthcare is best conceptualized as a sociotechnological practice, where humans and machines do not vie against each other, but rather where machines augment the abilities of humans.Innovations in AI within radiology are likely to be propelled by academic healthcare organizations collaborating with industry.Guidelines are essential to govern these partnerships in an ethical manner.
Bias stands out as one of the foremost ethical concerns associated with AI in radiology.While not all biases are detrimental (such as purposefully oversampling critical characteristics or tailoring algorithms for specific contexts), all applications of AI should undergo scrutiny for potential impacts stemming from various forms of bias.Guidelines for the ethical utilization of imaging AI must be established, with radiologists collaborating with ethicists at the forefront of this initiative.
For the responsible integration of AI into clinical practice, robust validation strategies are imperative.One approach is to align AI use cases, as defined by clinical end-users (including labeled data), with challenges designed to objectively evaluate and compare algorithm performance (7,10).
To train resilient AI algorithms, diverse datasets are essential.Standards are required to facilitate the efficient utilization and sharing of these datasets through clinical trials.
Data sharing holds the potential to advance the development of clinically relevant AI tools, provided that barriers to data sharing are addressed and appropriate incentives are established (3).Radiologists will need varying levels of proficiency in AI depending on their roles, but it is crucial for all radiologists to acquire a fundamental understanding of AI, encompassing both its capabilities and limitations.
Developing the educational resources necessary for an AI curriculum will necessitate collaborative efforts among multiple stakeholders, including national and international societies, regulatory officials and academic radiology departments.

Discussion:
In the pertinent scientific literature, three critical areas of intervention in AI can be discerned: ethical concerns, the international regulatory framework, and bottlenecks in regulatory development.Specifically, bottlenecks have been pinpointed, particularly regarding workload, indicating that regulating it without a scientific foundation may pose greater risks than having no regulation at all (6).The majority of governmental regulatory officers, radio-logists, specialized radiation protection experts like medical physicists, technicians and above all radiation exposured patients have not been adequately informed about AI and its exact tasks in radiation protection.However, they are evidently eager to participate in further courses to enhance their understanding and technical proficiency in this field (1).The overwhelming consensus is optimistic, with many believing that AI will positively impact their future practice.Their primary expectations revolve around enhancing the quality of care for patients and optimizing the time spent with them (11,13).Regulations, standards, and codes of conduct must be established and consistently revised.Central to these codes of conduct is an ongoing focus on transparency, safeguarding patient interests, and robust oversight of data versions and utilization (5).Continuous postimplementation monitoring is essential to detect unintended consequences and potential declines in quality, with established protocols for identifying causes and implementing corrective measures (7).
Important and uncomfortable questions need to be answered.Therefore, it is incumbent upon policymakers and official regulatory bodies to ensure the continuous reliability of medical radiation protection for the general public, especially in the face of regulatory vacuums created by new technologies such as AI.Entrusting this task to the so-called free market is not considered conducive to achieving this goal.

Fig. 1
Fig.1 Image of blind justice -metaphor for inactivity in radiation protection concerning AI?