"Not only could the app be used to learn more about childhood autism, it could possibly reveal signs of post-traumatic stress disorder (PTSD) and mild traumatic brain injury (TBI) in warfighters - conditions that often have subtle symptoms and are difficult to diagnose," said Predrag Neskovic, from US Office of Naval Research (ONR)'s Mathematical Data Science programme.
The app, called "Autism and Beyond," was developed by researchers and software developers at Duke University and the Duke Medical Centre in US. "Autism & Beyond" has children complete a series of questionnaires and watch short videos designed to make them smile, laugh and be surprised. Parents or caregivers use a smartphone's user-facing "selfie" camera to record children's facial movements for evaluation by doctors, researchers and software.
The app's core technical component is a complex mathematical algorithm that automatically maps key landmarks on children's faces and assesses emotional responses based on movements of facial muscles. "We analyse the video to track position and movement of the head and face, including the lips, eyes and nose - all of which indicate emotions," said Guillermo Sapiro, a professor at Duke University, who developed the algorithm.
"For example, while watching stimuli like a funny video, does the child smile, look towards the caregiver or ask the caregiver to view the video as well? We study all of that.