She is 84. She fell in her hallway at night and broke her hip. She will go home with a walker.
No one has ever seen the home she is going back to.
Home fall-hazard assessment from phone photos, reviewed by a professional before discharge.
The problem
Who falls, and where
were advised by a doctor on how to avoid them.
Prevention works
That expert is the occupational therapist.
Cochrane systematic review 2023: 22 studies, 8,463 people, high-certainty evidence.
The bottleneck
Occupational therapists. Each square = 1,000.
1 therapist for every ~5,000 people over 65.
The missing rule
If an occupational therapy assessment is not available, review potential home hazards with the patient and family.
The CDC recommends it at every discharge. Step Wise makes it possible, with an expert eye.
With his phone, room by room: overlapping photos from several viewpoints. The nurse adds Maria's profile: walker, hip replacement, gets up at night.
0 photos · the whole living area
Jev answers atomic questions on the areas of the Westmead Home Safety Assessment, the tool named by the World Falls Guidelines. A box counts only when presence is confirmed; what the photo cannot show stays "to check in person".
Living room and sofa: real photos of the home used in the demo. Bathroom: Point3D Commercial Imaging Ltd. / Unsplash. Boxes are illustrative.
VGGT is a transformer that looks at all the photos at once. In a single pass it works out where each one was taken from, how far away every pixel is, and where it sits in space.
Real prototype output: 39 phone photos, camera poses estimated by VGGT, 757,792 Gaussians. Shown here: about 150,000 points. Scale not yet calibrated.
Gemma reads the evidence and Maria's profile, ranks the hazards by urgency and explains why. Every sentence points to a photo and a source.
Gemma 4Google DeepMind · open modelMaria uses a walker: the wheels catch on the edges. And it lies right in front of the stove, where she walks every day.
Evidence: kitchen photo · Source: CDC STEADIWith her hip replacement she has to step over the edge with nothing to hold on to.
Evidence: bathroom photo · Source: CDC STEADIThe discharge nurse sees each hazard with its photo, its place in the room and the reason. Confirm, edit, exclude or leave it for the home visit. Minutes, not a visit.
Nothing reaches the family without her sign-off.
Real prototype screen. Both hazards in this room were flagged by live TypeSafe Jev runs recorded with the source.
Maria's daughter gets the approved plan on her phone: actions in order of priority, in plain language, each with its photo. Before Maria comes home, not after the next fall.
The prototype
Silent screen recording of the working prototype on a real home.
Privacy
Photos of a home and clinical data cannot go to just any cloud. Step Wise is designed to run on a single server at the care facility: the 3D reconstruction and the model that reasons about the patient share the same machine.
Upright orientation, metadata stripping, quality checks.
FastAPI · Pillow · NumPyPoses, depth and 3D points.
PyTorchChecks, priorities, plan.
vLLMTodayResearch prototype. Gemma runs on a hosted endpoint and the reconstruction on a local GPU; running inside the hospital is the goal.
On Friday, Maria goes home.
This time, someone looked at the hallway.
The team
They built the prototype end to end: 3D reconstruction, AI pipeline, review app and family report.
A clinical advisor (occupational therapist or geriatrician) and a care facility to run a first pilot with.
Sources