RealityCapture tutorial: Scan objects from all sides using Masks.

[@H0w3r]

THANK YOU! This tutorial saved my ass. For those of us who are just shooting with phones cameras and don't have fancy green or black backgrounds to use.Super insightful and helpful. Awesome stuff guys, keep it up !

[@postandgraphics]

legendary. This is my exact workflow I use in [competing product] and now I can do it many many times faster. Love you guys!

[@RazsterTW]

Oh my! This is going to be so useful for our project. Thank you so much!

[@darcy4143]

Excellent tutorial clear and concise in its delivery, thank you for making it easy to understand the process.

[@EugeneLychany]

Great tutorial man, thank you so much for this. I personally use shooting in the void method but this could come in handy as well.

[@3dprintfoundry221]

Please can you tell us what "shooting in the void method" is? I have used green screen before, but it involves a lot of pre-processing. Thanks.

[@EugeneLychany]

@@3dprintfoundry221 Try looking up "Scanning in the Void - A short video on good turntable setups " by Erik Christensen on KZbin

[@jvanko89]

Thank you, I am glad you liked the video. Ofcourse shooting in the void method is also great and I use it too. I actually combine it a little with masks as well sometimes if the object requires it.

[@johanboekhoven]

I'd like to see a tutorial on managing colors through the whole process. Maybe talk a bit about the white balancing, ACES compliant workflow for bonus points etc.

[@jvanko89]

Thank you Johan for the request. We are getting multiple requests about texturing so it is on the list. RC supports texturing from 16 bit Tiffs and RC doesn't touch the colors unless you use the color correction in RC. You can use layers for this kind of workflow. Use 8 bit JPGs for alignment, meshing and Tiffs for the texturing layer. I personally am familiar with ACES but this topic is something that would need to be consulted with someone from the industry.

[@edrift3d]

This answered so many questions. Thanks!

[@texasjakit]

I'd like to see more tutorials on how to get a clean mesh when dealing with trees and complex terrain from drone scanning properties. Great tutorial tho. Very helpful.

[@CapturingReality]

Thank you for the ideas. But this is more possible in some 3rd party soft, in RC you can do just basic cleaning.

[@pheelers1831]

Great video. Can you do more videos on the tools and their settings like you did with unwrap tool here (16:30) but in-depth with values and and how it effects the model?

[@Outdoorshuntingshooting]

Awesome, I have ignored this feature for way too long. Time to up my model game.

[@Abrimaal]

Can I draw manually a mask on an image to include by force this part for meshing and texturing? To prioritize one image part over all the other inputs?

[@mahen9394]

Excellent well explained. Thank you for sharing. Cheers

[@claudesigma3784]

Very helpful.

[@alexandrsavciuc8564]

Thank you for this amazing tutorial!

[@j6077xxd]

Awesome thanks!! I would love to see a tutorial of modelling a shoe.

[@CapturingReality]

Like using mask for model creation? The principles are the same, you can have shoe instead of house.

[@Truong1557]

Thanks! learn many things from your video

[@jetSonic]

Ďakujem pekne Jakub.

[@davevenmans7693]

Hi, when I export masks of the same component for a second time (I changed the preview mesh), the masks will override the previous masks in Windows explorer, but won't update in RC and the 2D viewport. Is this a bug of am I missing something?

[@RotorWorks]

Thanks. Great tutorial.

[@grahamtunnadine4427]

Thanks: I cannot find "Large triangle removal threshold" in the current version (1.3) of Reality Capture. Is it still in use. I usually just run WORKFLOW>Start when I am ready which gives me several models at different resolutions. But if "Large triangle removal" will give me better quality then I would like that. OOPS - I have found it. Thanks again

[@HusseinElAmine-k7p]

Great tutorial! thanks can you do a tutorial about small metal object, for example a scalpel

[@CapturingReality]

Hi, the principles will be basically the same. You can also apply these principles in the process: kzbin.info/www/bejne/emjMimRqa8iIaLs

[@danieljackson182]

good, it works! :)

[@vishalanime]

I am using markers for object scaling. Can the markers be masked out?

[@AetherScout]

If capturing a building, is it better to take photos following the the straight lines of the walls of do a loop? Thanks

[@CapturingReality]

It depends on the wall, but lines following the building's shape would be better.

[@AetherScout]

@@CapturingReality brilliant thanks

[@alsettw]

Thank you for this @Jakub Vanko! Working on table top objects right now, so it's really helpful. I was wondering - it doesn't seem that you're using a turntable, rather that you are moving the camera around the object. Why did you choose this approach here? Also - could you share some advice regarding turntable workflow - common workflows for background, masking, etc.?

[@jvanko89]

You are welcome Tomasz. I just tried to simulate a scenario for a total beginner without a turntable or other special equipment. I also have a turntable studio setup. To put it simply, everything is black or white. The background and the turntable has the same color. It is the shooting in the void method, it was mentioned in one of the other comments. In that case RC will detect only the object and will ignore the white or black background. No masks needed. Sometimes I lower the detector sensitivity to eliminate features on the background or turntable. But that is a rare occasion.

[@Iam_9801]

@@jvanko89 So, in other words, if i make photos with white background and white turntable, i don't need any masks? Or it will be worked only with black background?

[@jvanko89]

@@Iam_9801 It will work but I prefer black. White reflects light while black absorbs light and eliminates reflections from the background and turntable.

[@Iam_9801]

@@jvanko89 i got it, thank you for answer

[@Juleru]

I use a black turntable on a table: I bought small, wooden cubes and glued them together to create 4 different Tetris-like pieces that I then attached to the turntable with double-sided tape (one in the front, back, left and right) - the triangle pattern of the turntable probably helped too. My object is pretty symmetrical but if yours isn't, then there's probably no need for that. The background in my images is simply the table that of course always looks the same, I didn't use a sheet or anything like that and simply masked the object as shown in the video (great result!). The only thing that you have to be careful about is the lighting, you don't want any shadows on the object itself or different colors between image sets. To help with that I got a color checker, then converted the images to tif, using the camera profile that was created with the color checker.

[@fx3casey]

Hi there. Utilizing a turntable, I photograph a pinecone in 5 different positions: Orientation for the various series are UP, ON SIDE, ON SIDE rotated 90 degrees, ON SIDE rotated 180 degrees, ON SIDE rotated 270 degrees. This generates a series of 5 sets of 60 images, each 6 degrees from the next as rotated on the turntable. I generate the masks for each set with the table element removed. RC doesn't align them properly when brought into a new project. All the camera loops remain parallel to the grid plane. It seems RC does not recognize the rotation imparted in the general position one set to the next. Thoughts?

[@CapturingReality]

Did you remove just table or a whole background?

[@fx3casey]

@@CapturingReality the entire background. Table and background are black drape. Masks were generated as you presented, and saved in the source folders with appropriate, automatic naming convention applied.

[@CapturingReality]

@@fx3casey It is hard to say what could be wrong without seeing it. There could help using CPs. Can you contact our support with this issue?

[@chrisgoodine88]

Why do you need to include a portion of the table in the preview mesh and then use the lasso tool to filter the table selection? What would happen if you removed the table completely from your initial reconstruction region and created a preview mesh that already had the table cut out? Would the exported masks still remove the table if you skipped the filter selection step?

[@jvanko89]

Hi Chris, that is also another valid workflow. No need to filter something out if it was not reconstructed in the first place.

[@autodriv3r405]

How can we scan "soft" Objects like Garbage Bags, Bags, Towels and so on...from all sides? Hanging them in the air is not possible because they will look like they´re "hanged". Flipping them upside down will make the topside look different and not recognizable. So, is it any possible to scan Soft Objects from all sides? Thank you!

[@CapturingReality]

This is quite hard for photogrammetry, when the object is different between shots. For this I would consider use some kind of varnish, which should create a tough object, which won't deform by moving.

[@relaxvideo3d]

Cool. It would be even better if we wouldn't have to export the masks all to a new file and import back again, instead RC could calculate with the unwanted area automatically. Then import the next loop, select table, delete and calculate the final geometry in only one project. Possible in future?

[@CapturingReality]

Thanks for you idea. Will create a feature request for this.

[@theoadamsiv7844]

Can you guys do a video on converting a 3D scan into a asset for gaming/film? (Retopology... UV unwrapping... Texture/Normal Baking etc..) I have gotten some great looking scans but I've been banging my head against the wall for days trying to get them workable in Blender. Its pretty tricky!

[@danieljackson182]

Getting your model into Blender is pretty simple and you can do all steps in RC (Except Retopo). Simplify your mesh to let say 500 K, then reproject your texture and normal map. Export- and import your model to Blender, and assign your textures.

[@jvanko89]

We develop RealityCapture so we primarily focus on tutorials about RealityCapture but you never know so stay tuned. If you are scanning a static object that will not be animated the workflow Rafal mentioned is good enough. Filter out the excessive geometry, simplify the mesh to the desired poly count and use the reprojection tool to bake the color, normal or even displacement maps. RC also supports LOD export where you can define the strategy how the LODs will be generated.

[@Norma-so7mu]

Hello! So I finished all the masks, create a new project to import all my folders that contain the masks, but then I get a scene with multiple components and I don’t have a mesh merged, they are still showing separely! How did you do to make realitycapture merge everything in one mesh only? Thank you very much for your help

[@CapturingReality]

Hi, how is your object captured? Is it like a from a globe? What overlap did you used? It all depends on the way, how your object is captured. Did you try to use Merge components tool? You can also use control point to merge separate components.

[@jamesvella]

Is it possible to create a mask for all images with 1 mask of the background for turntable situations? For example I take the first photo of the area without the object, then the following images all have the object moving on the turntable while the camera is stationary. Then process all images using the first photo so it knows what should be removed automatically. Currently I get a panorama of points which is a mess, walking around the object does a better job but is not ideal for lighting, camera refocus etc. Hope this makes sense.

[@jvanko89]

Hello James, I understand what you mean but this workflow with one image of the background without the object is not implemented in RealityCapture. I recommend using a featureless background like a black cloth. In that case RealityCapture will not detect features in the background but only on the object itself.

[@jamesvella]

@@jvanko89 Thanks for letting me know Jakub! I will try again with the 'shooting in the void' technique.

[@fxsrider]

I followed along step by step 3 times now and I still get the components laying together in the different poses the photos were taken in. Side, upright, upside down etc. Another issue is I am being dinged for my credits at the Depth and Mask export. Well before I can see if I even wanted the final result. That seems like a midpoint in the process. Anyone know why they are asking for payment before it's even done?

[@CapturingReality]

Hi, what kind of object were you capturing? How many images did you take? If there is no enough of overlap this could happen. In that case you can also use control points to merge those components. Depth and Map are also part of exported products so therefore you need to license your inputs to create this kind of export.

[@fxsrider]

​@@CapturingReality A True Value truck, think coin bank size. I have since built an automatic turntable run by an Arduino. In a very brightly lit light box that is all white. I'm going to keep trying at 64 shots per revolution next. I have a polarizing filter to cut some of the glare as well. I will let you know when I get that mask thing to turn the models. Control points are going to get a shot on the images I already have but I will still slow the turn down to get more overlap. Thanks for the reply!

[@simonSTSSorcerer]

So why is the -Large triangle removal threshold- set to only 10 within your standard settings? It seems this is quite unnecessary beside for the case that...... ?

[@jvanko89]

RCs algorithm creates water tight meshes. The areas which are not visible in photos are filled out with large triangles. With the default setting the large triangles will be mapped only to 1 texel and not occupy too much UV space.

[@Tritoon710]

Can you generate the mask using Photoshop or other software and export black & white image and relink it inside Reality Capture?

[@jvanko89]

Yes that is possible. I briefly mentioned it in the tutorial. Just make sure you use the proper naming convention for the masks. This is also explained in more detail in the tutorial.

[@samehbijo]

Thanks alot , is it possible to do a human bust scan tutorial ?

[@CapturingReality]

Do you mean for masks? It is basically the same as other sculpture or model

[@samehbijo]

@@CapturingReality No I mean the whole process for taking someone's head's scan , almost all the tutorials are talking about rigged object , would be interesting to see how it would work on a human bust . Appreciate it

[@CapturingReality]

@@samehbijo For head scan or full body scan are using camera rigs (something like in this video kzbin.info/www/bejne/p2GwhZirpcuLg6c). You can also use your phone, but the results will be worse (kzbin.info/www/bejne/i3fJe3ecmLyeoKs)

[@JoseChambrot]

Is there a way to do this using a green screen then key out the masks?

[@jvanko89]

Yes that is another valid workflow. Just import your photos as an image sequence to some video editing software that can do keying. Export the masks with proper naming convention to RC and it will work.

[@phischtv4497]

What if my low-quality run to generate the masks already fails? ;)

[@CapturingReality]

What do you mean? Did it crash?

[@phischtv4497]

@@CapturingReality No, but just not getting any proper results

[@CapturingReality]

@@phischtv4497 Can you contact our support with this issue?

[@robinhook8977]

This is really useful. However the object that I have scanned is not geometric/ square. It is a shoe which I have scanned from the top, and separately for the bottom. There are only two sets of images that i am trying to merge into one: 1) the top of the shoes, 2) the sole of the shoe. I have followed your steps to the tee, but when import the two sets of images with their matching masks, they do-not align into one. The result is still two separate components that wont merge. I have also tried to connect components manually by using control points, but the same problem persists. Anyone have any idea what I am doing wrong?

[@jvanko89]

It is ok that it is not a square. You can still put it on the sole, after that flip it 90 degrees to the right and or left. If you still have the shoe just take more photos of it. Focus on areas that will be used for merging the top and the bottom. Two loops is simply not enough. Take photos in a way that the camera positions create a dome around the whole scanned object. Also in RC Alignment settings set Image overlap to low.

[@robinhook8977]

@@jvanko89 Thanks for your advice, but it still isn't working :( Here is my workflow: I added loads more detail than previously. Shoe upright: three orbits at eye level, 45 degrees and above. Then i went into plenty close range detail, paying specific attention to the are where the sole and rest of the shoe would need to merge. I repeated the same process with the shoe up-side-down. Orbits at various angles as well as plenty detail of the sole and area where I want it to merge. I changed the overlap setting in RC to low as you have suggested. I exported masks for all images from top and bottom separately, and then brought them all back together in RC to align. When i alight I am having the same problem sadly :( What am I still doing wrong?

[@robinhook8977]

Perhaps you wouldn't mind getting on a zoom call with me so I can share screen and you help me figure this out?

[@Eclectic_Chicken]

Hi, I have generated masks (in 1.4), but, evven though they've been loaded and are associated with the correct pictures, the mask is not applied... Any idea ?

[@CapturingReality]

Hi, what were your steps there after masks creation? Have you created a new project? Have you imported the images also with the masks together? How many images is in your projects after masks import (is it the same as images number or double)? When you check the image in 2D view and use TAB, do you see the masked area there? Do you have set How to use masking layer in the Selected inputs panel correctly?

[@Eclectic_Chicken]

@@CapturingReality I have reimported the project. for each pic, there's a path to the mask but the mask do not show in 2d view :/ The how to use mask is set for both aligment and meshing (default) in the picture viewer list, I only see the origal pictures but when using the tools tab, I can see my masks images. 142 images for pictures, 142 for masks. The masks applies in the 2d image view when I select the option. But the output alignement or mesh has no mask applied to it.

[@CapturingReality]

@@Eclectic_Chicken Try to copy masks to the original image folder and then import that folder to the new project. Then the masks should be applied (you should see only 142 inputs after that, not 284 in 1Ds view).

[@Eclectic_Chicken]

@@CapturingReality masks images are in the right folders, number in 1d view was ok (maybe my explanations weren't ;D ). But I thought the mask would apply to the set it was made from and that's what was not working. I tried with another multiple set like in the tutorial and it's now working. So If i understand it well, masking is only used in multiple sets scenarii (ie top / bottom...) and will not be used a a mask to hide parts of a mesh in a single set. Thanks for the help !

[@CapturingReality]

@@Eclectic_Chicken It can be used in any type of project. You just need to create a new component from masked images, it is not working for the project from the masks were created off.

The version seems to have change, the menu are different

[@CapturingReality]

Hi! Yes, the UI has changed, but the principles are still the same.

[@FelipeOliveira34910]

In my case, i'm getting two components when i align the images.

[@CapturingReality]

You can contact our support, if you have problems with the application.

[@jorgeh.1223]

Great explained, thank you so much! Is needed to pay to every mask export? (In your example you have 7 compositions that means 7 payments)

[@CapturingReality]

Hi, you can buy PPI credits and use these credits to license your inputs. Then you can use them in different projects. But when you have 7 different objects, you need to license all of the wanted inputs in PPI.

[@jorgeh.1223]

@@CapturingReality Thank you for the answer! I understand, I already have credits (and the price looks me great) but how I´m learning I wanted to be sure that the result gonna be satisfactory to me. Last question here, there are other way to put together, e.g. the sole part of a sneaker with all the rest? May be merging 2 components and using control points?

[@CapturingReality]

@@jorgeh.1223 I will answer you through support channel

[@CapturingReality]

@@jorgeh.1223 Yes, you can use control points for this.