
Overview
To address this, GAMMA is collaborating with the child behavioral health experts at the University of Maryland Baltimore for an UMCP-UMB MPower COVID-19 project. Artificial Intelligence (AI) strategies assess engagement as an emotional state. We propose a new semi-supervised AI learning paradigm using Dual Attention Networks (DANs), which will jointly leverage visual and vocal/text attention mechanisms from our caregivers to capture the fine-grained interplay between visual cues and language. DANs attend to specific regions in images and words in text/voice through multiple steps and gather essential information from both modalities to measure affective states, i.e. engagement. This multimodal fused analysis will be used to derive an automated engagement score. From a health care perspective, an automated engagement score can be used in future studies to give providers real-time, quantitative feedback during or immediately post-session. Relevant to artificial intelligence science, this research can be used to model pandemic-related contextual factors (e.g. social isolation), and will influence the discordance between trained observer ratings and self-report on an emotional state.
We are also working with rural child behavioral health providers (social workers, counselors, nurses, psychologists) and caregivers enrolled in treatment at the Community Behavioral Health or Maple Shade Youth and Families services clinics in Maryland Mid-Shore and Eastern Shore counties (10 clinics, 6 rural counties). These clinics are overseen by the Medical Director, Dr. Sushma Jani.
Social isolation may impact caregiver engagement behaviors
Innovation
This proposal involves an innovative collaboration between AI and Child Behavioral Health Experts at University of Maryland. From an AI perspective, the application of automated engagement strategies for child behavioral health care is novel and the use of a multimodal recognition algorithm can advance quality of AI engagement research in real-world settings. From the Behavioral Health perspective, this research will help develop an approach for real-time provider feedback on caregiver engagement. This automated technology will be used to enhance provider engagement training and improve quality of care. The social contextual factors examined in this research (isolation, social role and functioning) will also allow us to better adapt clinical services to pandemic conditions.
Project Members
UMCP Computer Science: Aniket Bera (PI), Pooja Guhan, Dinesh Manocha
UMB Medical School: Gloria Reeves (PI), Susan dosReis, Mathangi Gopalakrishnan, Kristin Bussell, Kathryn Mcdonald, Kay Connors, Katrina Escuro
Some Related Publications
| Project | Conference/Journal | Year |
|---|---|---|
| ABC-Net: Semi-Supervised Multimodal GAN-based Engagement Detection using an Affective, Behavioral and Cognitive Model |