29 - A Pilot Initiative to Implement Artificial Intelligence (AI) Specific Requirements in Ethical Research at Four VA Medical Centers

The National Artificial Intelligence Institute (NAII) manages AI research, implementation, policy and collaboration at the VA. In response to Executive Order 13960, the Institute has designed an AI-IRB Module that supplements existing IRB processes to incorporate trustworthy AI standards into human subjects research regulations at the VA.



Program

Description: The module and process were developed by four Centers - VA medical centers in different regions. The supplemental components can be added to the review process for VA local IRB or the Research and Development Committee (for externally reviewed studies). The supplemental AI-IRB module components include:

1. AI IRB Module application and integrated reviewer checklist (adapted from Ito 2021; Makridis 2023)

2. Informed consent document language that prompts the investigator to thoroughly explain AI use and risk in an accessible manner for patients.

3. AI specific Model Card templates, which present basic information in a clear and concise manner to subjects to further support transparency and responsible use of AI. The investigator is prompted to include specific details about their study. Model cards are adapted for the AI community from the General Services Administration - Technology Transformation Services to convey model performance and other characteristics.

Program

Assessment: The AI IRB initiative was rolled out in two phases. Phase 1 was implemented in Oct 2022, at one center. In Phase 1, the limited module identified a study, as submitted, that did not meet the Trustworthy AI requirements. Phase 2 was implemented at the additional Centers in Jan 2023. The sites are collecting metrics on the use of the AI-IRB Module and the effectiveness of the initiative. The data collected is currently being used to validate the responses to the VA AI Data Call (April 2023).

Limitations: The current limitations in implementation of this new initiative include the limited number of new AI studies at these centers, submitted studies are minimal risk and do not utilize all components of the module, lack of an electronic version of this module application for integration into the electronic research management software used by the VA.



Discussion: This AI-IRB Module and Initiative is one way the VA is translating AI Trustworthy principles to meet EO13960 requirements to protect our veterans and enhance their care by providing consistency and transparency in reporting and review of AI research. Further plans include addition of a limited number of other sites; analysis of metrics on usability for investigators, IRB staff and reviewers; analysis of the transparency and understandability of AI studies for the subject; and electronic formatting of the AI-IRB Module. Ultimately this module will be available for all sites performing AI human subjects research at the VA.