PrototipAR (formerly PintAR)
PrototipAR is an Augmented Reality prototyping tool which has featured at a CHI interactivity and a DIS demo. It combines web connected tablet and Microsoft HoloLens to quickly produce sketched content and interactions for both AR and non-AR applications.
NodeJs, Unity3D, C#, Fabric.js, HTML, CSS, JavaScript
I developed and helped design this system with guidance from Danilo Gasques from the Human-Centered and Ubiquitous Computing Lab at UC San Diego
CHI 2019
What You Sketch Is What You Get: Quick and Easy Augmented Reality Prototyping with PintAR
Danilo Gasques, Janet G. Johnson, Tommy Sharkey, Nadir Weibel.May 2019 CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems
"Augmented Reality(AR) tools are currently primarily targeted at programmers, making designing for AR challenging and time-consuming. We developed an interactive prototype, PintAR, that enables the authoring and rapid-prototyping of situated experiences by allowing designers to bring their ideas to life using a digital pen for sketching and a head-mounted display for visualizing and interacting with virtual content. In this paper, we explore the versatility such a tool could provide through case studies of a researcher, an artist, a ballerina, and a clinician. "
DIS 2019
PintAR: Sketching Spatial Experiences in Augmented Reality
Danilo Gasques, Janet G. Johnson, Tommy Sharkey, Nadir Weibel. June 2019 DIS '19 Companion: Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion
"In this demo, we present PintAR: an interactive prototyping tool that explores Augmented Reality for the design of interactive spatial experiences. Our system aims to remedy the lack of tools for rapid-prototyping situated experiences in AR without programming or 3D modeling. PintAR combines a digital paper and pen interface with a head-mounted display to allow users to sketch and interact with digital content in their environment. Users can take PintAR anywhere, leveraging objects and information available in the real world to bring context to their prototypes. "
Hard to Reach Experts
Often user studies can be difficult if they involve experts like surgeons - they are busy and highly specialized. PrototipAR targets these populations specifically to allow for co-design with experts. It does this through its simple sketching mechanism. As users become comfortable, they learn more and more of the system, but the basis is as simple as:
Sketch
Press a button
Place content
Repeat
Tablet
Top View
Shows an aerial map of content placed around your space. Users in the space show as icons moving around. Individual sketches can be selected and given behaviors, deleted, set to scenes, and even moved. As sketches in the virtual environment change, they update in the Top View
Scenes
An overview of the scenes in a prototype, showing movements between scenes. If a sketch is given a behavior which causes it to change scene, that change is represented by arrows in the Scene page. Scenes can be selected to open that particular scene for editing
Canvas
Simple drawing interface which allows user to create low fidelity content
Library
A list of all of the sketches created
Loading Page
Lists saved prototypes - selecting a prototype allows for deletion or loading
Phetching
Allows for quick computer vision from the tablet camera to create a sketch from a photo
Sketching
Sketching was chosen to encourage low fidelity and rapid iteration, and it's a skill everyone has
Behaviors
A simple interface exists which provides a simple sentence structure for ascribing interactions to placed sketches. These behaviors consist of Actions and Triggers
Actions: changes in sketch or application state (movement, changing frames, hiding/showing, etc)
Triggers: events which cause an Action to occur (proximity, gaze, airtap, collision, time, frame change, etc)
A behavior is any combination of Actions and Triggers (multiple Actions can happen as a result of a single trigger, vice versa - n to n actions to triggers)
Any number of behaviors can be given to a sketch
Actions can target other sketches, and sketches can be set to trigger each other
Behaviors can be expanded through code - simple class inheretence with implementation of a single function. Additionally behaviors for IoT integration already exist.
Scenes
Possibly the most valuable feature of PrototipAR are its low cost "Scenes". These are application states of a prototype which can be used to show logical breaks in an application or can be used to mock behaviors. Being suck a quick state machine, it is easy to use scenes for a number of practices.
Wizard of Oz
An extremely valuable tool in PrototipAR is its ability to allow for complex behaviors to be easily mocked through use of the shared space, scenes, the 'Wizard' action, and a filterable button array in the top view which triggers actions.
Groups
Any number of sketches can be made into a group. Combinations of groups and sketches can also be made into groups. Groups function identically to sketches - they can be given behaviors, moved, set active in scenes, etc. in a non-destructive form - they don't override behaviors and scenes set for their nested sketches or groups, allowing for complex behavior to stack.
Sharing
PrototipAR allows for any number of devices to connect simultaneously - combining designers and experts for co-design or wizards and users for user studies. Spaces are calibrated using simple user agreement on physical objects. Then, movements, triggers, frames, and actions sync between devices. User 1 can see User 2 trigger a sketch.
