Bilawal Sidhu(@bilawalsidhu):Semantically annotating 3D gaussian splats on the fly using gemini 3.1 + sparkjs 1. Load any 3D scene and hit scan 2. Get 2D detections from VLM 3. Cluster outputs & project into 3D world space 4. Save as a persistent 3D semantic layer Inspired by @alexanderchen's experiments with gemini visual intelligence. Just had to try to lift it from 2D to 3D!

Bilawal Sidhu

@bilawalsidhu

Spatial intelligence. World models. Visual effects. Creator w/ 2.1M+ audience. Tech Curator @ TED. A16z Scout. Ex-Google PM (AR/VR & 3D Maps)

Joined December 2010

6K Following 107.6K Followers

Bilawal Sidhu@bilawalsidhu

2026.05.15 02:15

Semantically annotating 3D gaussian splats on the fly using gemini 3.1 + sparkjs 1. Load any 3D scene and hit scan 2. Get 2D detections from VLM 3. Cluster outputs & project into 3D world space 4. Save as a persistent 3D semantic layer Inspired by @alexanderchen's experiments with gemini visual intelligence. Just had to try to lift it from 2D to 3D!