Lock 3D model in AR

#1

First of all, congratulations on providing such an efficient and user-friendly tool. Among all the applications I’ve tried with the QUEST-3 for AR, it is undoubtedly the best made.

My inquiry is whether it would be possible to introduce a more precise way to orient the digital model in the real environment, and then lock that position (so that the model cannot be accidentally moved or rotated), as well as lock the scale (1:1).

This is necessary in the mixed reality environment to accurately check that the (real) constructed elements are in the defined position in the 3D model.

A logical way to achieve this could be:

1st: Lock the SCALE of the model at 1:1

2nd: Move the model until a control point § is positioned over the mark that has been previously defined in the real environment (by a surveyor). Then, lock the TRANSLATION of the model.

3rd: Rotate the model (around the Z-axis and around point “P”) until its orientation matches the real environment (for example, using another point). Then, lock the ROTATION of the model.

proceso
proceso1746×1306 319 KB

With this, it would be possible to compare the digital 3D model with the real environment without fear of it moving, rotating, or scaling.

As I don’t like to ask for something without giving anything in return, I’m sharing the link to a library of 3D models (created by me with SketchUp) that is completely free to use. For those related to the building industry, you might enjoy viewing these models with VR glasses:

Biblioteca Detalles Constructivos 3D

Thank you very much for everything.

2 Likes
#2

Hi! A quick reply: yes, this looks possible to implement. It would need to use the feature that the Quest calls the “spatial anchors”. I haven’t looked at it yet, but I will and tell you if it seems reasonable.

#3

Ah, a simpler solution (Quest 3 only): maybe we can use the depth sensor. It produces an approximation of the real room surface (walls, floors, ceilings, tables and chairs, etc.) as a rough list of triangles. The trick is that this set of triangles is fixed, as long as you don’t re-run the depth sensor explicitly to update it. So it should be easy to fix the virtual model relative to that list of triangles. (You can play with it in the latest VR Sketch, setting “Passthrough: room depth”.)

#4

I’ve got an experimental version ready, if you want to try it out. It just forces the model to be at a fixed position relative to the real world if you pick “Passthrough: room depth”. There is no way to configure where, but you can make use of the usual Move and Rotate tools to change the Sketchup model until it matches.

If you make a new room-depth scan, then the model will likely be off and you’ll need to use Move and Rotate again. I’m interested in knowing if the approach actually seems to work in practice, or if you need to re-Move and Rotate the model all the time.

Do you need the standalone Quest version, or the PC VR version over Oculus Link?

If it works, I’ll add an option to set or unset this special “fixed” mode, and make it work also if you don’t pick “Passthrough: room depth” (in that case, it should still ask for the room depth information but not use it for display).

1 Like
#5

Thank you very much!.. I’m impressed by the speed of your response. I would love to try the solution you propose. I need the standalone Quest version, although since I’m new to these topics, I’m not sure how to install it. As soon as I run some tests, I’ll let you know the results.

#6

Low lighting in indoor construction sites may pose a problem. This could potentially affect the accuracy of surface scanning

#7

Surface scanning on Quest 3 is not done by reconstructing the image (I think it’s by lidar) which means it doesn’t need ambient light.

Here’s the APK: v20.1.ALPHA1.

You need to install it as described here (sorry, the process is a bit lengthy): https://vrsketch.eu/docs-getting-started-quest.html, section “Installation without using the App Lab”.

#8

Thank you very much! I’ll let you know how it works as soon as I can try it.

#9

Finally, I’ve been able to complete the installation following all the steps in the instructions (previously I had to uninstall the official version). I’ve tested it, and the ‘passthrough: room depth’ mode has stopped working and behaves the same as the ‘Passthrough: regular mode.’ The rest is working fine, so I don’t know what to do.

#10

Unsure. Maybe VR Sketch no longer receives the shape of the room? Make sure you agree to the dialog box that asks if you want to let VR Sketch use “spatialisation”. Also make sure the shape of the room is actually defined. You may need to go into the main Oculus menu, in Settings, Physical space, Space setup, Setup.

#11

I’ve conducted several tests (in different environments), but I haven’t been able to get the ‘passthrough: room depth’ mode to work, and consequently, the model doesn’t position itself relative to the real space.
I have reinstalled the official version of VR Sketch, and with it, the ‘passthrough: room depth’ mode works perfectly. However, I’m afraid that the solution to automatically place the 3D model in the real environment is going to be of little accuracy. Perhaps it would be more precise (and simpler) to do it through two points (anchors).

#12

Still thinking about it and re-reading your original post. What you are asking about is a way to “lock” the model’s position and rotation, right? Sorry, I went ahead and thought about additional functionality, but let’s start instead with just the “locking” part. I may be wrong, but I think that it’s relatively easy right now to move the model until it is at the correct place with the correct rotation, using the grip button on both controllers. You don’t need to lock the translation and the rotation independently. Is that correct? The only thing missing would be a “stop moving the model now” option in some menu.

The rest would be additional non-essential functionalities:

  • the locked position and rotation could survive after we take off the headset and put it back on—normally, this resets the position and rotation of the “camera rig” in VR Sketch, which changes the position and angle at which we see the VR world. Maybe it should be done just with trying harder to NOT change the position and angle in this case, using “spatial anchors” or the “room depth” if it works. (I’m not entirely sure why the change of position and angle occurs. It might be because when we put the headset back on, it needs to figure out instantly a mapping between VR and real coordinates, but it takes a few seconds until it recognizes the real world and figures out where it is and where the boundary is; the Oculus team may have decided it would be better to not make the world jump at that point, so that’s why you get the position and angle of the VR world changing randomly if you take off and back on the headset.)

  • the locked position and rotation could be saved and restored together with the model itself, so that when you open the model again in VR, it is still locked. That is possible to do by saving the spatial anchors inside the model.

  • possibly: if you have multiple VR headsets in multi-editing mode, and they all share the same physical room, then there should be a way to lock the model at the same place for all headsets, instead of setting them up one by one—but I think that’s harder to achieve right now…

#13

Indeed, it is about locking the translation and rotation of the model once it is in place, but it is also important to lock the scale from the beginning to a factor of 1:1, since when handling the controls to rotate the model, it is very easy to inadvertently change the scale.
I suppose the simplest process is as follows:

  1. lock the scale
  2. move one control point of the object to its final position
  3. Move a second control point to its final position (actually, rotate the model around the first point)
  4. Lock the position and scale of the model to avoid inadvertently moving it.
#14

Actually, I think it’s better to position the model first and then rotate it through a point, as in the real environment, it’s easy to have such points marked by the surveyor with a target. If the model is positioned simultaneously with its rotation, I believe it will be very difficult to ensure accuracy.

#15

There is in Settings -> Options a checkbox that says something like “grip button rotates OR scales”. But yes, it’s not perfect. And also, if you have set up the Sketchup model so that z=0 should be exactly on the floor in VR, then you can’t get that. You can teleport to a z=0 face, thus ensuring it is exactly on the floor in VR, but then you need to move or rotate the model to align it precisely, and with the grip button it always changes the vertical position a little bit.

Just like you suggest, the best solution might be a menu option that enters a mode in which scale and vertical position are locked, but you can still drag and rotate the model horizontally; you click on some point and that point is fixed and any more dragging is a pure rotation; you click anywhere again and the model is completely fixed. It might need some text displayed prominently to really know what is going on, though.

#16

Indeed, that’s exactly what I was suggesting. If it were truly possible to do it just as you describe, I believe it would be a very valuable tool for supervision and reassessment in a construction site. Thank you very much for your effort.

#17

the ability to anchor a model on a site and have it locked for each headset in the same session would be huge! it would allow people to see the actual view from modeled windows as well as the house on the lot.
model_1

#18

@Greg_Hale: I agree in principle, but that requires sharing spatial anchors via the Oculus cloud storage. This is something that comes with strong requirements that VR Sketch can’t completely fulfill. For example, in theory it might work together with Oculus Link for tethered Quest usage, but in practice for VR Sketch that is just impossible (e.g. VR Sketch needs to be downloaded from the Oculus PC app store, which would be a complete change of the practice of installing it from the Trimble extension warehouse or directly from our site). Even if we only target standalone Quests, I fear we might conclude that the amount of work needed for us is far too much to justify the small benefit for a few users, sorry! You can still calibrate each headset manually.

#19

@Jaime_VR62, here is the standalone Quest version. It adds a new entry in the context menu of the teleport tool: APK v20.1.ALPHA2

Installation instructions (the same as before): https://vrsketch.eu/docs-getting-started-quest.html , section “Installation without using the App Lab”.

In particular, I’m not sure if that’s enough for your purposes or if the headset resets the position too often for it to be useful…

#20

Thank you very much!!.. As soon as I run tests, I’ll tell you how it works.