56 - Risks in AI and Computer Science Human Subjects Research

Background: Following SACHRP’s 2022 advice to OHRP about how the Common Rule applies to research involving artificial intelligence (AI), questions remain about how IRBs should identify and mitigate the associated risks. AI developers, computer engineers, and IRBs need guidance for determining when AI projects are subject to the human subjects regulations.

Methods: Using conceptual analysis and analogies from biomedical and behavioral research, we categorize AI projects and propose guiding principles for determining whether an AI project is research (i.e., systematic investigation designed to create generalizable knowledge) and, if so, whether it involves human subjects (i.e., living individuals about whom an investigator obtains information through intervention or interaction, or obtains or generates identifiable private information).

Conclusion: We identified three categories of AI activities that need to be considered: Research about AI, research using AI, and research to develop AI. Research about AI, such as consumer perceptions about AI, use patterns of AI, or evaluating ways they can be biased, would be considered human subjects research when investigators are attempting to create generalizable knowledge and collecting information from individuals through intervention or interaction, or accessing existing, private and identifiable information about individuals. Analogous to research about any other product or service, research about AI fits well into the existing human subjects protection framework. Research using an AI tool to collect data or to implement an intervention, would be considered human subjects research if investigators intend to create generalizable knowledge and collect information from individuals or access existing, private and identifiable information about individuals. Analogous to research using any other intervention, research using AI fits well into the existing framework. Research to develop AI is the most complex activity to categorize. Not all AI development is designed to develop generalizable knowledge and not all AI development involves human subjects as currently defined by the Common Rule. Computer engineering practice activities, such as building an AI tool, are not considered research, as they do not collect information for the development of generalizable knowledge. Rather they can be viewed as analogous to consumer product development. The resulting products may present risk of group harm, which are not under IRB jurisdiction. Group harms ought to be addressed, but are more of a consumer- or user-protection task.

Limitations: Exceptions to this approach include AIs that are developed for use in health care, in which case FDA human subjects regulations would apply, similar to the application of regulations to software as a medical device. AI and AI research is moving swiftly and we are unable to imagine all possible implications of AI on the protection of human subjects. Our analysis is based on the current state and may be revised as we learn more.

Discussion: AI research presents challenges for IRBs and program staff. Our outline of the various types of research involving AI will help IRBs and staff classify and consider the applicability of the Common Rule to various AI and computer engineering research activities.