Using AI-Powered Digital Humans for More Efficient Care
In this blog
Introduction
This February, WWT attended the ViVE conference in Los Angeles, which brought together over 7,000 healthcare professionals for a week of sharing innovations and the art of the possible in healthcare technology. WWT's focus on AI was on display at our booth in the form of a fully interactive digital human, tackling a specific healthcare use case around vaccine schedules.
What is a digital human?
A digital human is typically a 3D avatar displayed on a large screen, with full voice-to-voice interactivity and powered by a large language model behind the scenes. Digital humans occupy a unique space in the AI world because they blend emerging technologies for realistic, human-like 3D avatars with evolving AI voice models. The combination represents an interaction often portrayed in science fiction, but one that has begun to take hold in specific scenarios where a more natural human interaction has intrinsic benefits.
How we created it
Our version of the digital human for ViVE was named "Cassie," a 3D avatar built in Epic's Unreal Engine using state-of-the-art MetaHuman technology. Built into the engine, we utilized Epic's Fab marketplace to procure a highly realistic MetaHuman model, which we then customized for our use case. MetaHumans also have a feature called Live Link that allows lip and facial animations to be created on the fly from a voice source.
As mentioned previously, the other primary component of a digital human is the large language model. We used the cloud-based Google Gemini Live API to power the voice-to-voice interactions. Users can speak to the model, and the API receives the audio data, then returns the model's audio response. By playing the audio response locally on our booth's machine, the MetaHuman Live Link feature instantly animated our avatar so the words appeared to be spoken in real time. The result is a highly scalable and portable digital human that can adapt to nearly any voice-to-voice use case.
Vaccine schedule use case
The use case we chose for the demonstration was focused on vaccine schedules for children. If a patient comes to a doctor's office and has not been following the recommended vaccine schedule, they typically need to follow a "catch up" schedule, which involves receiving specific vaccines based on their age and vaccine history. It may take a doctor or an assistant 15-20 minutes to aggregate this information across a variety of data sources and complex visual charts. However, "Cassie" was given all this data (fed into the model's context window) at the very start of the interaction. This allowed her to become a model with access to the custom data required for our use case. We achieved this through retrieval-augmented generation rather than custom training, meaning that applying new data or updating existing data could happen nearly instantly.
Versatility of the solution
While this use case focused on improving care efficiency at a doctor's office, the versatility of this tech stack is readily apparent. With the ability to easily update 3D assets, modify cloud API calls, and add custom data to the model's context window, this digital human setup can quickly adapt to use cases across healthcare and other verticals. Increasingly, customers are looking for solutions that are highly scalable and adaptable as the speed of innovation in the AI field continues at its unprecedented pace. If a more realistic 3D model is released, it can become the new face of the solution with minimal effort. If a higher fidelity cloud voice model goes live, it's an easy switch. This architectural flexibility gives this digital human a unique advantage, allowing it to continue evolving at the same pace as the underlying technologies.
Conclusion
The future of AI remains a broad and diverse landscape. Digital humans and human-like AIs will continue to improve. With thoughtful architecture and innovative approaches to solving business use cases, we are seeing our customers succeed in deploying AI solutions like these, and we are incredibly excited to be a part of this stage of technological ingenuity.