AR ICU

This project is in progress and developed through the BioInformatics department at UC San Diego under Dr. Shamim Nemati

I am the sole individual responsible for Design, Development, and User Testing of the entire system - the only portion I am not responsible for is the DeepAISE Machine Learning algorithm predicting sepsis onset.

Only sketches and ideation is presented on this page as the project is currently being developed.

Using Augmented Reality to display and navigate patient data in Intensive Care Units (ICUs)

Initial Concept

The initial idea was to surround the patient bed with data - increasing physician-patient contact and using space to create new relationships between data. A network-like visualization strategy was proposed which considered how organs connected together in various ways - avenues for the spread of infection.

Showing networked organs allowing for an overview of the patient's status

Clicking nodes on the patient brings up detailed break downs for that particular organ

Using space as an axis for time to show how patient status changes with time

Clicking a node now shows detailed break downs of how measures have changed over time

Data Comparison

The predictive analytics tool (I did not develop this) allows for extraction of gradients for individual data inputs and can be used to measure a particular measurements sensitivity in influencing the overall predicted "sepsis score." In doing so, comparison's between patients can be made - providing physicians with a patient's "path" through their illness, providing the opportunity to show what medications were successful in a particular case, and which ones weren't.

Patient Progress

Each point in the cloud represents a single patient at a given time. The purple line shows the current patient and their progress. (The 3 dimensional representation is the result of the compression of 40 dimensional data)

Nearest Path

Comparing the current patient to other patients can provide examples of previous patients who have experienced similar paths in the ICU, allowing for individual moments to be compared.

Comparison

Comparing patients breaks down the various labs, medications, and scores they have received - providing physicians with side-by-side information.

Scans

X Rays and Ultrasounds are taken often in ICUs, and can be placed side-by-side for comparison and changes over time

CT Scans

3 dimensional imagining techniques have become more and more popular, AR can provide means for viewing these scans

Prototyping

Using PrototipAR to create quick prototypes and video walk-throughs.

Proposed Design

Using AR to prototype my design proposal, being situated and immersed in a mock ICU environment allowed for new understanding of the problem space

Co-Design with Physician

Brought in a physician to participate in a co-design process where we walked through different tools he was familiar with and what kinds of information he believed might be useful.

Visiting ICU

I made a number of visits to 2 of UC San Diego's ICUs, following around different groups of physicians, oberving their work flow, noting pain points and needs, and discussing the struggles they face and where AR might aid them.

The following are sketches made while in the ICU

Google Sites

Report abuse