Photon Headset - Augmented reality for vision assistance

2023-2024 / during gap year before college

Summary

I was curious about retinal projection. It seemed very sci-fi: no screens in front of the eye at all, with the image drawn directly onto the retina instead. I tried a few AR headsets at CES 2024 (that year it was the 'big thing' in consumer electronics) and was pretty disappointed by the bulkiness of the displays, and wanted to see if I could come up with something better. This page is documentation of the various research projects I did in order to try to simulate, replicate, and iterate on that fiction: an AR headset for vision assistance (navigation, obstacle avoidance, depth perception for visually impaired users), built around 3D-printed optics, eye tracking, and a steering mirror.

In the end it did not work. I was naive and overly optimistic about the precision of the admittedly limited arsenal of manufacturing equipment I was using (a couple of few-hundred-dollar printers) when realistically I needed an industry-grade lithography machine to manufacture the optical components. But hey, you only find this stuff out by trying (and failing).

Turntable animation of the Photon SLA v2 headset

How It Works

Diagram of a reconstruction beam passing a photographic plate to form a virtual image — [1]The principle behind projected imagery: a reconstruction beam hits a plate and the viewer receives wavefronts that appear to come from a virtual image.

The concept_[1]: instead of a display panel, a beam is steered and shaped so the wavefronts arriving at the eye appear to come from objects in the world. Done right, that makes visuals sharper and easier to follow for low-vision users. The design had a steering mirror scanning the image across the eye while eye tracking followed the user's gaze, adjusting the projection in real time.

The hard part of retinal projection is the eye-box: the small region where the projected image is actually visible. The design attacked it two ways. Small actuators would adjust the lens position in real time as the eye moves, and the optical system was shaped around a wide field of view so the image stays usable across it. Most of the project was spent finding out how far these ideas could be pushed with 3D-printed optics; the honest answer turned out to be "not far enough," but the attempts are documented below.

Optics Simulation

Before printing anything, the optical path was simulated end to end. Flat 2D ray tracing_[2] worked out the steering angles and mirror placement; a full 3D ray tracer then modeled the mirror array itself, following every element's reflection as the rays converge through the pupil position_[3] and land on a model of the retina_[4].

Ray trace showing the full path from array through the pupil onto a hemisphere retina model — [4]The full path: array, pupil, and the reconstructed image landing on a hemispherical retina model.

The Mirror Array

Render of the hexagonal mirror array module — [5]The array module: dozens of hexagonal facets, each angled to aim its reflection at the eye.

The core optical component is a 3D-printed hexagonal mirror array_[5]: a honeycomb of small facets, each set at its own angle so that every element reflects the projected image toward the same point_[6]. Printing the geometry instead of grinding glass is what keeps the headset affordable, and the honeycomb went through many CAD iterations_[7] to get the facet angles, spacing, and printability right.

Side view of the angled hexagonal facet geometry — [6]The facet geometry from the side: every hexagonal pin rises to a different angle.

CAD view of a mirror array iteration — [7]One of the array iterations in CAD.

The printed-optics direction was inspired by some 3D-printed optics projects by Andy Kong: for the sake of transparency, plus this one and this one.

I was curious about the feasibility of the optical system I had designed, so I emailed Dr. Mark Schnitzer from Stanford (I later ended up working in his lab!) and he basically told me that anything I could fabricate myself with a resin printer or otherwise inexpensive machinery would produce far too much aberration and would distort the image reflected onto the retina beyond recognition.

Development Process

Head mesh in CAD with the optical linkage positioned at the eye — [8]Fitting the optical linkage against a head model to hold alignment with the eye.

The project went through over 20 iterations across six months, with each prototype testing and improving specific parts of the design: 3D-printed optical systems, fit and assembly against a head model_[8], and the laser projection PCBs.

The retinal projection required precise alignment of optics, which drove the custom optical system design and its wide field of view. The small eye-box was the other persistent problem, and the actuated lens positioning was the approach for it. Alignment is also where the project finally hit its wall: hobby printers hold tolerances measured in tenths of a millimeter, and the optics needed better than that by orders of magnitude.

Conclusion

The headset was shared with AR/VR developers and presented at meetups for feedback, and the platform is open source for anyone who wants to build on the parts that do work: the frames, the eye-tracking approach, the simulations, and the mirror-array geometry.

I'll pick this project up again once I learn more about graphics and rendering, and ideally once I have access to manufacturing that can actually hit the tolerances the optics demand.

Gallery