Meta AI makes its debut on Quest 3 headsets

One of the features the company is teasing on the Quest 3 is Meta AI with Vision. Although wearers of the Ray Ban Meta smart glasses have had this functionality since April, this marks the first time an app will have access to the passthrough cameras of the mainstream Quest 3 headset. Meta has not mentioned anything about releasing a passthrough API for third-party developers, but it does seem likely, especially as more developers clamour for such access. 

It is possible that in later upgrades, users will be able to ask questions about objects in VR environments in addition to physical objects, since Meta is saying its visual AI functionality is only available in Passthrough “for a start”. However, the capabilities of Meta AI with Vision are presently limited to analysis of objects in real-world surroundings captured via the front-facing passthrough cameras. Outside of visual input, other possibilities exist to use Meta AI in mixed reality via voice commands. This effectively replaces the existing voice commands on Quest headsets as those will soon be deprecated.

Meta AI with Vision first launched on Ray-Ban smart glasses as a beta feature limited to the US and Canada. Similarly, the experimental AI features on Quest headsets will also be restricted from countries in the EU. Meta – like Apple and other tech giants based in the US – is having a hard time with data privacy regulators in the EU, and as a result has decided not to release future multimodal AI models in the region despite Llama being open-source. The company plans to launch a text-only version of its Llama 3 model to serve its users in the EU.