Space Carving - Computerphile

I love these videos of Mike Pound, so interesting!

[@Acid113377]

Running ubuntu and then a "WPF in C#" book in the background. Seems like a jack of all trades. Good stuff!

[@wanderingrandomer]

I love the use of technical terms. "A sort of triangulary cylinder" Otherwise known as a prism.

[@onemanenclave]

triangular prism*

[@Bradley_UA]

Triangularly cylindrical prism.

[@SkyFoxTale]

2:20 "What's this?" Topologist: a torus

[@unaliveeveryonenow]

Physicist: a point mass Materials engineer: a group of crystal lattices Biologist: a population of skin bacteria on a rock Mechanical engineer: a part that is very difficult to manufacture Graphic designer: an ugly font Photographer: bokeh with artifacts Programmer: an object

[@DavidWillanski]

Surrealist: a horse galloping on a tomato

[@Croxmata]

Chemist: A ceramic, probably silicate, with a mostly organic substance applied to the surface.

[@AntiComposite]

Lawyer: An object that appears to be consistent with a description of a mug.

[@KasranFox]

The office's single IT guy: My fuel for the day.

[@phildxyz]

I could listen to Dr Mike all day...

[@Mezurashii5]

That took a long time to explain a reeaaly simple concept. There's plenty interesting stuff on the 3d modeling topic to talk about though, now is a great time to jump onto the theme too.

[@adelarscheidt]

A good way to illustrate that the hulls can't be understood by the camera would be to imagine the object made from a vantablack material. You'd only see the silhouette, you'd understand the convex hull, yet wouldn't be able to tell wether there's really a hull in there. (Apart from your intuition about the object, as they mentioned haha)

[@adelarscheidt]

+Adelar Scheidt *whether

[@Mengmoshu]

That is a really cool way to help wrap one's head around the problem. Thank you.

[@Anvilshock]

*Edit button

[@smileyball]

This reminds me of the CVPR paper "3D ShapeNets: A Deep Representation for Volumetric Shapes" which models the uncertainty of the 3D geometry of an object and tries to find the next-best view to minimize the uncertainty.

[@Huizelogica]

+1 for pronouncing Wageningen correctly

[@goeiecool9999]

Love this guy! Clear and to the point.

[@AnimilesYT]

You said 'Wageningen' with the proper 'g' sound. You're awesome! :D

[@ThePamimo]

It's amazing how much progress we've made on this in the last 3 years. Look up meshroom or photogrammetry if you haven't seen it yet. I don't know if it works the same way as this but man, new software is becoming so powerfull its incredible

[@ITR]

You forgot to write beneath the Rubik's cube >.>

[@ArnoldsKtm]

No

[@Diggnuts]

+MMMIK13 Bit of a Parker Square...

[@PhilBoswell]

+MMMIK13 maybe they meant :-P

[@BrickOfDarkness]

+MMMIK13 maybe it is a loosely defined derivative of XML and hexagon is defined to not need a closing tag.

[@666Tomato666]

+BrickOfDarkness in that case it's missing the SGML doctype

[@Permutatorem]

So basically, a silhouette tells you where the object definitely isn't and where it might be, but not where it definitely is.

[@Vospi]

About halfway though a video, when it was stated that "how would you know that something's hollow", then it started pulsating in my mind and it just can't leave it. Light! Additional light source should be a huge help in that, because you can base a lot of predictions of your shape based on shadows, right? Capture an image without that extra light, then let your additional light source to rotate around an object or gradually get closer to it, capture a shadow and get a definite evidence that your mug isn't stretching a mile behind -- even without rotating it! Even just turning some static light source on and off for every picture should help -- if we know its exact position, for instance. Can't you use that in this whole scheme?

[@cmm90871]

another alternative is to use narrow spectrum lights each emitting from a 60 degree offset and shadow/shader for each color to determine vectors to cleave, (interestingly possibly use the color mixing gradients for depth and opposing surface shape, especially useful if you compare the objects texture color profile to a full spectrum lighted )

[@IngviGautsson]

I once implemented space carving using POV-ray. Because it's very easy to use it to code extrusion of 2D paths and intersections of 3D objects. The drawback it that you don't get a point mesh that can easily be imported into modeling software.

[@mrnarason]

3:29 Basically the rule for all youtube videos.

[@mrembeh1848]

+Victor P. thought the same :D

[@richardpike8748]

read this comment at the exact moment he said it lol

[@Triantalex]

false.

[@sjaaksafari709]

I see dr. Mike Pound in the thumbnail. I click.

[@thesomething8467]

Voxel ambient occlusion would make the 3d models a lot more pleasant to look at, it's easy to implement and it can be zero overhead for the GPU.

[@DannyBurkeBanjo]

its great when you watch these videos and they help with your revision!

[@AutisticThinker]

If the camera had a light source directly behind it illuminating everything it see's (and always aligned with the camera's PoV), so shadows are dependent on the camera position; you could calculate depth from shadows.

[@5imian]

This channel is cool as hell.

[@TheBigBigBlues]

It's like a 3D CT scan. You measure how well X-rays pass through an object from a number of angles in a 2D to get a 'slice' image. Expand this to 3D.

[@joshinils]

+TheBigBigBlues only that either all or no ray gets blocked, there's no value in between

[@Mr30friends]

+TheJoshinils proof ?

[@ninjafruitchilled]

Similar idea, but CT is way more powerful because you have way more information related to depth.

[@TheBigBigBlues]

+TheJoshinils Yeah true.

[@Mr30friends]

***** I thought we were talking about x rays ,in which case i am pretty sure this " there's no value in between" doesnt apply

[@pppppaaaaaccccchhh]

You should make a time-lapse of him just drawing lines all day

[@DFX2KX]

there used to be some software, way back in the day, that let you take a toy or something small, stick it in front of the lens of your webcam, and manually mark out the edges in each frame, then it would run through it's algorytyms and generate a painted 3D mesh of the thing. This might well explain how that software worked.

[@sebbes333]

@Computerphile and the person in the movie. What if you move the camera sideways, eventually the center of the camera lens will line up with the edge of the Rubrics Cube and then you can see that it actually is a straight edge.

[@KnakuanaRka]

Sion Yeah, that requires you to get lucky with how you place your camera.

[@Ivo--]

Shout out from The Hague, a mere 15 minutes from the Westland, where at night the sky is bright with light from the greenhouses.

[@TheHardRage13]

Very great video, proud of your work:)

[@valivasiliu2200]

ubuntu and sublime on the background, you have my thumbs up, from a fellow brother in arms

[@tbpotn]

There has to be some elegant way to detect a 90 degree corner and set the camera up such that one of it's peripheral lines lines up perfectly with one of the sides of such a corner..

[@Anvilshock]

That's the point of this method: You can't know that very well (if at all) from the outset, you can only optimise so far. You'd need pretty much to know the shape beforehand to set the cameras up. And when you know the shape already, why would you then still bother going through the space carving?

[@DataCab1e]

This technique was used to produce the crude "holograms" in Steven Spielberg's film "Minority Report."

[@mihirsalot1346]

why not use a distance sensor for measuring distance and hence hollowness??

[@manmanman5371]

The way he pronounces Wageningen! Fantastic. Wacheningen

[@gabrielbeedles1172]

That tomato seedlings footage looked awesome! Would love to see more of that! Is there any additional KZbin footage available of the high speed use of space carving?

[@World_of_OSes]

Why was Ubuntu in the background?

[@TheImaxify]

The same as what's called "backprojection" which is widely used in medical imaging specially for a semi-transparent objects.

[@Kruglord]

That's really cool, I haven't heard of space carving before, but it makes a lot of sense. I wonder if there's a sort of hybrid between space carving and photogrammetry that'll sort of a bunch of problems associated with each.

[@Dieze]

shouldn't you go able to get rid of the bulges by moving the cube to the side so that the edge would be in the center?

[@littlebigphil]

+Dieze TA Good point.

[@martinhill7304]

+littlebigphil Depends how much harder doing everything else becomes after translating the camera location; should work though

[@daringblitz2316]

Can you release a 10-hour video loop of you pulling a sheet off a stack, drawing lines on it, rinse and repeat? Purpose? just to see who would watch you draw lines for (almost) all day? XP

[@DC-yl3yl]

What if you also used translational instead of only rotational motion to get different camera angles? Wouldn't that carve away the sides of the cube more accurately?

[@eideticex]

+David Clemens That's what I was wondering when he described space carving. How much translating and rotating would improve the results as compared to the rotational approach. That and "why not use an orthographic lens" but he answered that one, never realized just how much they cost.

[@DavoidJohnson]

How far backwards do we have to go to have cameras that cannot detect distance (autofocus)? You simply need multiple focus points to measure the hollowness of the mug. A bit of extra software will communicate this to your computer.

[@MikMoensted]

Couldn't you use a single light source with a known position in space, to cast a shadow, and post process the resultant pixel intensities to approximate hollow features?

[@Fake_Blood]

This reminded me of the puzzle game Picross 3D.

[@leedaniel2002]

I completed that game a few months ago. I've never had a happier moment that that

[@leedaniel2002]

+Watch The World Burn *than that

[@Fake_Blood]

Hah! Me too, had it for 5 years or sow! There's a sequel on 3DS but Japan only :/

[@lobosw3]

How is this diferent to what is done with Computed tomography reconstruction?

[@zxuiji]

Why are you not factoring in light into the equation? light effects the shades of colour and thus gives depth

[@GordonAitchJay]

13:09 "And finally, our rubik's cube, which is almost, in some sense the worst of our reconstructions. Because it's cube-ish, right? But it's not particularly a cube. I mean, that's cube-kind-of." haha!

[@nbase2652]

@COMPUTERPHILE: Regarding the hollow shape thing... Why don't they make use of an external light source shining on the scene at a different angle than the camera? The shadow that's being cast could be used to compute some depth information, couldn't it?

[@StefanReich]

But it wouldn't help you with the inside of the mug

[@mrvlhs]

It's pretty easy to know if something is hollow or not... cameras have to focus to take a picture and if they focus in different places then you have your answer

[@calfischer1149]

You can't always do that, but that's an interesting idea

[@pnw_dev7934]

What about using stereography to develop depth information? Just a thought...

[@4.0.4]

I wonder, aren't Kinect cameras very cheap right now? Does anyone know how hard it is to use several?

[@TemporalOnline]

What if you always assumed that all shapes are hollow, and removed parts of that shape based on pattern recognition (but you would need to have a database of shapes) and see what that blob would look like, if I removed "this" part (whatever that is) would it be closer to some other part in my DB via heuristics?

[@8bit_pineapple]

1:21 Is that Oskar's Treasure chest Rubix cube on the shelf I see?

[@atmunn1]

12:40 youtube compression why

[@RedEyedJedi]

Would it not be possible to have 3 cameras. One aimed at the centre and the other two aimed at the edges instead of buying an orthographic lens?

[@zacharybarbanell1064]

Why are some of the green cubes Bigger?

[@JavierSalcedoC]

4:00 the key is to avoid the computer to think it´s a #parkercube

[@Triantalex]

false.

[@RonJohn63]

0:05 Ubuntu. And is that Python in the right hand window?

[@Rykemasters]

My takeaway from this video is that, with low-resolution space-carving that can't take depth into account, the jungle plant is functionally undistinguishable from Sideshow Bob's head.

[@LowtechLLC]

thanks for the info on optical 3d systems. (I bought a lytro after watching that video.) can you discuss how fringe projection 3d systems work? I dont get the phase wrapping and unwrapping that is explained in the white papers. thanks again for the great series.

[@TheBlenderer]

Netherlands represent, woo! Wageningen, haha, solid effort on trying to pronounce that :)

[@Konan9000]

Can someone tell me why they don't use triangles? rather squares?

[@recklessroges]

+James H I think its because pixels are square, so when they wanted to push them into 3d they just turned into cubes.

[@daisy3067]

I'm thinking,could you using sound to help in fine tuning the hollow shape ,the frequency change

[@MiniArts159]

In a single video, Mike has singlehandidly upset every Geometer in a 500km radius of nottingham

[@bartz0rt928]

Can't you get some depth information from the way the colours change as you rotate the object?

[@mr7clay]

+Bart Stikkers Yes, but presumably that's not space carving. Carving is safer considering the object *could* be painted deceptively.

[@callummunro7380]

I will never trust mugs to be mugs again

[@nickhowatson4745]

a mug with an ugly mug who likes to mug mugs.

[@finfan7]

Congratulations, you have begun your training to be a Certified Fair Witness.

[@JinKee]

Morgan Yu is that you?

[@martin128]

Romb?

[@CloudWalkBeta]

Interesting stuff, Moving on from what i can understand, what this computer is missing is colour and lighting, as humans we can see how flat a surface is because of how the colour + lighting changes across objects surface to return depth values or a sense of roundness, if a computer had the same concepts it could make better results right?

[@KasranFox]

Have they made a video on octrees yet?

[@juliusfucik4011]

Use monocular SLAM and you get a point cloud that you can mesh easily.

[@Cutest-Bunny998]

Is that a rhombus?

[@nightshadefns]

7:56 A voice edit?

[@clockWorks10]

"Voxels are quite popular these days due to a certain piece of software called Minecraft"

[@mf-rozi]

That got me 😂

[@Teraku1503]

I'm waiting for the follow up video :)

[@Desirdef]

Can we get the voxel data that you generate?

[@cookiesliyr2]

good luck on that project mate!

[@dgamma1]

I am quite lost on this subject. Which is the playlist (or first video) one can watch to understand computer vision better?

[@mateovozila]

cant you just move the camera a bit to either side so it is looking right along the edge and carving it as it should be?

[@RealCadde]

+Mateo Vozila Imagine you are moving the camera (or the subject) 360 degrees for the full angles. Then you also do pitch for another 360 degrees. Then, you move the camera left/right 30 centimeters in .25 centimeter increments for a total of 120 steps. And another 120 steps up/down. Multiply them all together... 360*360*120*120 = 1,866,240,000 or 1.8 billion images taken. Now do you understand the problem? Given infinite time, one could move the camera around forever and get a better representation of the object. But they don't have infinite time now do they? And that is before you consider that 360 degrees is not that great of an resolution nor is 120 by 120. You would still end up with a coarse looking object. Just ever so slightly more detailed. If you really want to scan something you need depth measurements and generalizations / averaging.

[@mateovozila]

+Cadde but you would only need the six pictures that go along the sides and each pic. eliminates all the empty space from that side of the cube.

[@RealCadde]

Mateo Vozila Cube yes. But this system is meant to work on any shape, remember? If you design your camera movement around the particular shape then you might as well model the shape in a 3D editor and fill it with voxels.

[@cosmicsans67]

so are depth camera's just like, 3d cameras? Or do they have some sort of sensor like infrared or something?

[@KuraIthys]

+Soda POP 67 The two I know of (because they are cheap and easy to find information on) are the kinect and intel realsense cameras. Details vary, but the first generation kinect is an ordinary 2d camera, an infrared camera, and a infrared projector. The projector projects a specific pattern of infrared light, and the infrared camera picks up the pattern on the environment. The way the pattern is distorted gives the depth, and then everything else about a picture comes from the standard visible light camera. There's other ways of doing it, different kinds of patterns, and so on, but at the moment, infrared cameras seem to be involved in most of the 'cheap' depth camera setups...

[@cosmicsans67]

KuraIthys ah ok, thanks ^-^

[@lukeW7344]

photogrammetry programmes like agisoft photscan and reality capture seem to be way ahead of what they are doing, or is space carving just one of the methods used during photogrammetry?

[@PangolinMontanari]

Surely it would be possible to detect depth with stereoscopic images...? In fact you wouldn't even need to do that, a bit of laser or sonic trickery would do it.

[@Ceelvain]

How about using the texture / reflecting properties (BRDF and stuff) to infer the shape more precisely? IIRC, the Université de Poitiers (France), did this for quite some time now. At least that's what they told their students. :)

[@shaderbytes]

+Ceelvain photogrammetry does this and is widely available. Under the right conditions the results are spectacular. I doubt it is as fast though so in light of the end of this video where he mentions space carving working rapidly in a factory setting it would not be practical. check out sketchfab website there are literally tens of thousands of uploaded photogrammetry results which you can view in your browser.

[@Ceelvain]

+ian pretorius Absolutely awesome.

[@rumraket38]

It's hexagonal, the rubix cube, when viewed from a side where you see three faces.

[@AssassinGrudge]

9:15 am sorry but here you are mixing visual hull with photo hull....and the algorithm you are descring is called shape from silhouette which is based on binary segmented images (background/forground)..space carving is different algorithm

[@iagocasabiellgonzalez7807]

Great vid! Thanks

[@raymondlee1024]

Oct-tree video! We want the oct-tree video! Today!

[@hoffybeefe]

reinventing the wheel? you guys heard of structured light scanning? :P (tongue in cheek) very interesting how different things can be used for different purposes based on their benefits and flaws! cool vid!

[@ASOTFAN16]

Yay, love for the Netherlands XD

[@luffyorama]

I'm currently doing somewhat similar project, but using microCT-scan. And also try to use Meshlab for meshing purpose (3D Slicer is confusing). lol

[@JacksMacintosh]

Why not hack a Kinect to get that depth sensing?

[@joshd3773]

You still have to rotate it to carve. The kinect would know the mug is in the foreground but it wouldn't know how far back the mug extends for the same reason a camera couldn't.

[@Draugo]

The point was that Kinect, which is quite accessible for custom use cases, is a cheap source of RGBD camera information so you don't need a 20K budged for it.

[@alexweeda2826]

half of your video about doesn't have a caption

[@jamesessex7374]

Why has this channel not done a video or even talked about memristors yet? And the computer technological significance of this equipment. I am sure many of your viewers have not even heard about memristors yet. ;-)

[@Angloth]

+James Essex You just made me smarter, thx

[@JP05CPSN]

probably already do... couldn't you change the lighting and use shadows to measure depth and shape?

[@Freedom-js4th]

There were his fingers behind the mug

[@retepaskab]

You could use a moving lamp to detect holes.

[@lawrencedoliveiro9104]

0:04 At a guess I’d say that’s Python code on the right half of the screen.

[@JimCullen]

"It's not hollow, because how would we know?" Can you not use an rgbd camera to do it? I found it odd that you started talking about how it's impossible to tell that a mug is hollow immediately after discussing rgbd, but never mentioned why/why not use that to make space carving more accurate.

[@garygordus4465]

Hi, where are you located?

[@Twitchi]

Want more on orthographic camera!!

[@zoranhacker]

Yes! Same here, all I (think I) know is it just zooms a lot, which matches with Dr Mike saying it gives a small image and also the smaller the image, the smaller the angles and more parallel the "outer lines" of the camera, giving you an orthographic view! Sorry for going off lol you might already know this

[@finlayl2505]

If your not using RGBD then wouldn’t a mesh be better rather than voxels

[@EdSchroedinger]

pthe rotated cube shape might actually be a rhomboid... or a parallelepiped...