Hi!!! I am trying to develop an app that lets me take measures of rooms using the passthrough api. There is an app that already does this, and you should have a look at it as it does quite a good job. The only thing that lacks is better looking assets and textures, but the idea is really cool.
I think that if vrsketch includes passthrough api features it would be ground breaking as no one else does it yet (apart from this guy, but his approach is more on the VR games side rather than for real architecture use).
In my case, I managed to grab the measures of my apartment, and with the help of the developer I got the 3d file (FBX), and then modified it in sketchup. After that, I took the model into unity and developed a mixed reality app that allows me walking in my own house and see the modifications we have planned for our living room and our kitchen. Have a look at my video demo-casting my approach:
Quest 2 Passthrough API
I finally managed to get the passthrough working in the Quest 2 version. I don’t know if it would work in the PC version with Oculus Link, but maybe. I can make a development version of VR Sketch which displays the passthrough in the background, for the people reading this forum topic. Would you be interested in the standalone Quest, or in the PC version with Oculus Link?
(Note: as stated previously, I remain convinced that the passthrough quality is by far not enough for a proper experience. But if it doesn’t look difficult to make a checkbox in VR Sketch to use it instead of the skybox, then why not.)
Hello Arigo! For me it makes sense to have it as a stand alone application runing on the Quest… that would allow an architect going to a clients property and sketch the modifications the customer wants. Please have a look at the app I mentioned in my post (Custom home mapper) and give it a go. It costs only 8$. Check this video to know better what it is capable of. If you could bring that funcionality to SketchVR it would be GOLD
Check this tutorial video on how it works: https://www.youtube.com/watch?v=Ww8jUW_UfHY
IMO now that airlink is a think, standalone is somewhat less interesting.
Anyone who is into VR enough to use this app to model in VR will also probably have a VR capable PC would be my assumption.
Standalone Quest would be useful. Model in table top mode (warehouse) would be great
Here is a first version of VR Sketch for Quest which is able to display the passthrough:
There are two modes: the standard grayscale image with the Sketchup model fully opaque on top of it; and a variant where an edge-enhancement algorithm is run and you get black lines, which are displayed even through the Sketchup model. To enable these two modes, go to the settings dialog, “Background…” button on the first page. The two versions are basically two new built-in “background images”.
Reminder: installation instructions at https://vrsketch.eu/docs-getting-started-quest.html section “Installation without using the App Lab”. Also, once you are set up as a “developer”, you can disable the guardian in the Quest (settings, in the Developer page). This works well particularly if you use the “Passthrough #2” mode because you should see the room’s outlines even through your own model in this mode.
Hello Arigo! I have been testing the app and it works great! Amazing job! There is something I want to do but I don´t know if it is possible right now with the app or is something that doesn´t exist… If you check the video of the custom home mapper app, you can set the boundaries of a house with the help of the joystick. Example, imagine you have a big furniture in a room and you can´t move it, if you are able to move the drawing line with the joystick you can approximately draw that wall.
Let me know what you think.
You are trying to draw the walls, but can’t reach a particular point because there is an obstacle - did I understand you correctly?
At the moment you can find various workarounds using the general flexibility of SketchUp and VRSketch. For example, if there is a table, you can still draw the shape of the walls but higher, and fix it later. Or you can drag the whole model with the grip button, sketch the missing wall, and drag the model back as closely as possible. (What might help here is positioning the model by putting both controllers at known physical points, like in the video.)
But I also see the point of being able to reach places where the controller cannot physically go, either behind an obstacle or to draw the correct height of a wall too high to reach, say. I think a special option could be added: the tool usually takes action just in front of the controller, but we could imagine configuring it to be farther away, much like drawing with a pencil held at the end of a stick instead of directly in your hand.
Hi Arigo,
Thanks so much for looking into this. For our use case in an educational context, I would prefer the PC version with Oculus Link. We have had issues getting the Quest 2 connected and working on enterprise Wi-Fi (which schools use), but have managed to get it working more reliably by pushing the SketchUp model from a PC/laptop directly to a wired headset via Oculus Link.
I will test out the the APK at school when I’ve got some time and report back soon.
I will look into it. Right now Oculus seems to consider passthrough with Oculus Link as a “developer-only” feature. I think you should be good if you already registered as developer to install an older or non-official version of VRSketch on the Quest (without the Link).
I think it will be very difficult to support the Passthrough API “officially” on the Oculus Link version.
May I suggest an alternative: at one point I had troubles with my wireless network, but managed to work around it in the following way. On Windows 10, “Settings”, “Mobile wireless access point”, “Activate” it (top button). This enables “Share my Internet connection via… Wi-Fi”. You can do that whether your Windows machine is itself connected to Internet via Wi-Fi, or a network cable, or even nothing at all, I believe. Note the network name and password here (or change it by clicking “Modify”). Then configure your Quest to connect to this new Wi-Fi network. As long as the Quest remains in range, then it will get a very-good-quality Wi-Fi network connection directly to your PC. The VRSketch extension in SketchUp should be able to find this Quest on the local network. (Make sure it does: only click the big “GO” button after it says “local wifi connection”, not “encrypted connection via the cloud”.)
Hi Arigo,
I finally found some time to experiment with the Passthrough API during the school holidays.
I documented my test demo with a video:
Thanks again for implementing passthrough! For the record, I prefer the passthrough 2 mode over passthrough 1 but appreciate you including the option for both styles.
Btw, I got around the enterprise Wi-Fi issue by hotspotting both my laptop and Quest 2 headset to my mobile phone as a temporary measure.
I forgot to test your alternative method - will give that a go next time.
Hi Arigo,
The mobile hotspot from my laptop worked well! Thanks again for this tip.
Cool, I did something similar myself. Now that Quest Pro is out imagine the possibilities!
Imagine this with full color passthrough. With fake objects overlaid onto the real world
Thanks Dat!
Quest Pro: we are testing it, and the passthrough works out of the box already in existing versions of VR Sketch. You get a colorful and less-grainy passthrough that is definitely an improvement over the Quest 2. Not a perfect image, but good enough to convince our senses that it is the real world, maybe seen through some kind of slightly uneven glass which induces some minor deformations when you look around. There are no noticeable delays nor other stuff that could create sickness.
The next version will just move the VR Sketch setting to a less hidden place.
Hi. I’m loving VR Sketch and the Passthrough.
Is the option of drawing the height of a wall that we can’t reach already available? As in, be able to draw shames at a distance, maybe with the laser pointer, or similar to how the Quest Sytem room setup works.
Thank you.
Hi!
Drawing in VR Sketch works a little bit differently than drawing in Sketchup. In Sketchup, your are moving the mouse over a screen. A screen position can mean any 3D point that is behind that screen’s pixel. So Sketchup contains logic to know which 3D point you mean, i.e. at which “depth” inside the screen you are really pointing. For example, it’s difficult to draw in Sketchup a line that ends at a free-standing 3D position (you have to make use of alignments).
In VR Sketch, we took the opposite approach. Almost always, you specify a real 3D position, and that’s where the controller is. There are only a few exceptions, like laser-selection and painting of faces, where you select an existing face by pointing a laser.
When you draw a room, the way you are supposed to make the wall is to scale and move the model until the target points become easily reachable. That’s the way VR Sketch is designed. Drawing using the passthrough as the basis invalidates that approach…
It’s difficult at this point to merge a completely different approach into VR Sketch, but maybe we could tweak existing approaches. For example, right now you can “lock” one or two axes when you draw a line or a rectangle; this allows you to start drawing a wall (a rectangle) and lock it to be vertical. But you still need to physically move the controller as high as the top of the wall. What I’m thinking now—which would be more useful in your case—is that the locked rectangle should go not up to the same height as the controller, but instead up to a point defined by laser. It might be useful in general to make this change if it allows some positioning-at-distance to be done. I’ll try it out.
Armin Rigo
Thank you for your answer.
Something like your suggesting would be incredibly useful, especially for interior designers who could go to someone’s house, put the Quest Pro on, and quickly point the laser at the walls and draw the room layout in seconds. Exactly how the Quest Pro “Room Setup” works when you draw your walls and palying aread.
Thank you.
I’m coming back to this now that VR Sketch 18 has been released, sorry for the delay.
It’s unclear to me how what you have in mind could integrate with the rest of VR Sketch. You basically want something to draw walls on the floor like in the “Room Setup” (maybe as wavy lines? or by placing and connecting points on the floor, without needing to physically bend down?); and then something else to specify how high these walls should go until they reach the ceiling (with a uniform-height ceiling? or something more complex?).
All this is in theory possible to implement, and it would be a quick way to start making a model in your situation. But this looks very custom-made; it wouldn’t have any other more general purpose. If we were talking about Sketchup instead of VR Sketch, it would be the kind of thing you would find in a custom extension made by a 3rd party, and not in the core of Sketchup itself.
Needs more thinking from our side… Maybe at some point we should provide a way to write such 3rd party extensions for VR Sketch?