Box by Bot & Dolly | Behind the Scenes

[@ForestRainMedia]

I really wish they focused more on the technical aspects of what went into the final product... We're looking, at the very least, at two projects-- one for the panels, one for the floor. More likely, there's multiple projectors getting blended for the moving panels... There's also lighting cues and ways that the lighting in the room and on the subject mirror what is happening in the projections-- what kind of lighting was used? Did they use standard DMX control in conjunction with their automated motor control software? How's that all work together? Who designed the lighting aspects of it? So many technical aspects that could've been explored, but they focused more on the artistic "feeling" of the piece...

[@hanseldsilva2393]

BTSs are intended for non-technial audience but I agree, missed too many crucial elements

[@beneveritt2720]

Only assuming, but perhaps a lot of those processes were proprietary

[@GMoney-B]

I agree, I love all that technical stuff; but as a graphic designer and aspiring motion graphics artist, without the art, this would not exist, nor be able to exist. The feeling is in actuality, everything embodied into one, and therefore in my opinion the most important. I still love technical stuff though don't get me wrong. Before graphic design and motion graphics interest; I wanted to be a mechanical engineer going into automotive automation :), hence why I love this project probably.

[@juananania449]

I totally agree with you. I clicked to see this video to find that technical rider. It would be great to find a sheet with that info who can help us to learn and see what we need to do an installation like this. Great video though.

[@ABrokenTV]

They only used real-life projection for the text and things like that, not the canvas themselves. Here's how they made the video: The robots are holding a plain white canvas and moving them around in sync with the animation. As for rendering it out, they motion-tracked where the camera is. In post, they used information from where the canvas and camera were to render a video where the canvas acted as some sort of window with things moving in terms of where the camera is. Everything is flawless, except for at kzbin.info/www/bejne/oomZe5avl8x3fNE where the render failed to follow the canvas and is trying to catch up.

[@OwnerOfTheCosmos]

> Ideally, we would have everyone come and experience it in the room. [cut] The reason we decided to make a film was just to get it out there and that the widest audience see Box. I assumed that the 3D images were only possible due to the observer PointOfView being aligned with the projection. It seems like the only possible way, since the canvases are flat surfaces. The above direct quote from the "Behind the Scene" video, however, suggests that other POV would have conveyed the same kind of experience. So, if what the quote suggests were correct (And I believe it isn't, making the statement in the video *very* questionable. Don't get me wrong, it's awesome what you guys did.), I would have expected the "Behind the Scenes" video to deal with the question of 3D versus POV a little.

[@shornoMALONEY]

+God, creator of the universe Yeah I was hoping they would shed some more light on that. I think there's a couple of ways it could be acheived, with certain sacrifices. The ideal way would be to show it to one person at a time, who was wearing a motion tracker. The animation software could then change the virtual camera angle in realtime depending on the viewers position. This would be way more impressive than the video because the voluntary movements of your head would be reflected in the 3d shapes, enhancing the illusion. But I'm thinking that lag could be an issue, and only one person at a time. Maybe you could add a second person using 3d style shuttered/polarised glasses, at a cost of processing power and resources. You could also animate it so it worked for a more static viewing angle, then have the audience in a thin line perpendicular to the stage, at the same angle. However you would lose some of the illusion, and it wouldn't be as effective.

[@MarinoCanal]

the perceived depth is created by mapping on to a moving projection surface, the 3D shape is changing as the screen moves, it's not the viewer that changes the depth perception, you are still looking at a 2D surface as a viewer.

[@OwnerOfTheCosmos]

@Marino Canal: I'm not sure if you've understood what I mean, or if I understand what you mean. To clarify: If I and you would both be standing in that room where the video's action takes place, except I would stand a few meters to the left of you, would be both see correct 3D on the surface? If not, then please re-read my comment and/or please clarify yours.

[@xl000]

the effect is at its best when the spectator would stand in the exact location for which the projection is designed. However, if you redesign the whole thing to be viewed from a larger distance, this would have two effects: 1/ the 3d effect would be less dramatic, because the further you get, the closest it gets to being just a 3d animation projected on a 2d surface, which everyone is familiar with since the start of the cinema 2/ if you increase the viewing distance, you can also allow several people to be close to the point of view. The person at the perfect spot will have the best experience, and the further you get from the point of view, the shittier it gets. Our brain is pretty good at correcting small perspective issues; for example when you look at your PC monitor at an angle, you still understand that this is a rectangle under perspective projection without problem; So I think they could take this into account to create a animation that is viewable in a larger cone and still looks ok.

[@AngelMartinez-by1mx]

why am I only seeing this now five years later?? This is incredible!

[@ReverendTed]

I think this is simultaneously less impressive and more impressive than they make it out to be. The wording of some answers suggests this could have been performed before a live audience, when that isn't the case. The projection has to be precisely calibrated to the position of the viewer\camera to maintain the integrity of the illusion. Taking the end result by itself, this exact product could have been achieved more simply "in post", since the live performance is simply a means to an end. However, taken as a demonstration of the technology, it's even more impressive than presented, since the projection "video" has to be (relatively) precisely calibrated to the position of the camera robot. So we've got at least three robots and the output of a (stationary?) projector all synchronized. And yes, with all that ranting out of the way, the technical aspect of projection onto moving surfaces is still the highlight.

[@SamGriffinGuitar]

The path of the camera was tracked AFTER the physical installation was in place, not as it was being constructed. Sure, if you were in the room and viewed it at an extreme angle, it would distort the panels enough to destroy the illusion, but you wouldn't need your eyes to lock into the exact position of the camera to have an experience comparable to what we see in the final video. This is indeed as impressive as it's made out to be ; )

[@Bhatt_Hole]

@@SamGriffinGuitar That's what I thought when reading his comment...WTF is he talking about? You have a relatively wide (ish) area where a live audience could be. Why does that guy assume a projected image is only viewable from an extremely narrow angle directly in front? Maybe he's suggesting it couldn't be viewed by an audience of several thousand?

[@rowinvanderschee8043]

@@Bhatt_Hole What he is saying is that the perspectives of the objects (the rendering of cubes and spheres) don't hold up if you view from a slightly different angle. You immediately start to see that you are looking at a 2d canvas with a projection

[@notnilc2107]

@@rowinvanderschee8043 I'm pretty sure he's just trying to sound highbrow and sophisticated. If the creators really wanted to show it in front of a live audience, they could've just up scaled the whole thing and changed it so that the target POV wasn't moving (which would've costed quite a bit of money and is probably why they didn't). From what I'm reading he's downplaying the presentation due to its budget limitations (it can't be shown to a live audience) and praising it for its demonstration of technological precision, when, personally, I think it should be the other way around.

[@Zetalight16]

@@rowinvanderschee8043 It doesn't even need to be a different viewing angle. If you were watching it in person, the simple fact that you have two functioning eyes (assuming you do) would tip you off to the fact that it's a flat surface. It looks 3D in video because it's exactly as 3D as everything else in video is--which is to say, all the signals we use to determine depth other than binocular disparity are telling us it's 3D, but the camera is a single lens so binocular disparity can't come into play.

[@CHGOfficial512]

I honestly thought it was some high res led panels mounted to the robots instead of projection mapping, it's amazing, how you managed to pull it off in perfect sync

[@peterallely5417]

4:50 you got your wish, dude. I love the process that ILM have been using with LED volumes and Unreal Engine - so, so cool. Amazing what they simply capture in camera, on set now. It’s like it’s come full-circle.

[@B3LOVXD]

agreed. such a creative 'groundbreaking' art form!

[@MissSamGreen]

Can't believe I've just stumbled on this video 5 years later. But this is still, even today, amazing! I was so curious after watching the original video about how the depth was put into the projections. Well I found out! How impressive! I haven't seen anything like it!

[@StudioAnnLe]

Looks like a front projection. Stunning video. The BTS lends a little more to how they did it, but i'm still fascinated by how the projectors and images moved. Were the projectors also on robots as well?

[@stasisfoil7540]

So minimal being shot in monochrome, any colour would have made it less pure, Box is probably the coolest thing i have seen on youtube. Excellent work, just excellent.

[@Rulerofwax24]

I was also really curious how the depth worked before this video (because I thought the camera was handheld). The short answer to your question is no, the illusion of depth works at only the precise location of the camera at any time. Because the camera was also tracked in 3D space for Maya, the projections were rendered based on that changing camera location.

[@TimJSwan]

So this behind the scenes video was published Oct 1 2013, but at 6:05 the VICE copyright is 2015.

[@Z3912]

How long did the project take?

[@RanaBarker_rb]

Superb. I now believe in magic.. the magic of film and design can be brought together. Simply wonderful.

[@holographic_red]

This would be a great experience if a person could sit in place of the camera, like a solo ride. Cool project, very inspiring

[@lukedominicodonnell2347]

GMUNK you are no graphic designer...You are an Artist!

[@aaronbazil]

I still come back to this masterpiece

[@kathykakadelis4169]

This should be in a Museum. Magic.

[@keeyip319]

dont understand why people can dislike this amazing piece of work?

[@arasharfa]

thats a lot of work for something that looked like it was all CGI...

[@hanseldsilva2393]

watch the BTS, they said "live-action" CGI not pure CGI

[@mihaly_creative]

This a real life experience idiot, it's not made for KZbin... As he said, the idea of making a film of it came later just to achieve the widest audience they can.

[@dandan84]

@@mihaly_creative Well, it was all made "live" but would only "work" from the point of view of the camera, which itself is motion-controlled along with the canvas robots. So, in a way it was ONLY made for youtube!

[@AstralExtend]

I think the depth effect is just to prove the technology's precision. Being able to project on moving surfaces only regardless of said effect is still a massive innovation and will be valuable to theater production.

[@brettmclemore8175]

Yes. Mapping is essentially that. Masking out where you want the video shown. As far as the camera work goes, I gathered from this video that they had a guy make camera moves and then took that info and entered it into Maya. That info was passed on to the camera arm with the camera so it would make the same moves and they just animated the video accordingly for those perspectives. I'm glad someone else understands how impressive the technology is and not focusing on just the end product.

[@georgegrat]

Beautiful project but what was the final budget? I mean, one project like this is really very expensive especially the robotics.

[@AlRoderick]

I think it was just one fixed DLP projector. It's far enough from the screens that the small change in distance from the source doesn't blur the images. If you notice, the entire area in which the panels could move was much than a theater screen. The actual rendering process for the CGI takes account of what kind of distortion needs to be applied based on where the projector is relative to the screen and the viewer.

[@DJArcan3]

Question - Is the position of the viewer important? ie will for example the 3d box only look like a 3d box if you are viewing it from a specific angle?

[@tracymcsheery1223]

Happy we helped make this happen at PhaseSpace

[@CalvinK]

your assumption is exactly it. They had to map out the movement first, then made their graphic follows those two white canvas. Thus the movements had to be measured and duplicated exactly (by robots no less) so the projection won't miss the canvases. There was always ambient lighting so you don't see the "black" light border as it got washed / diffused.

[@OenopionsCrush]

Just as cool as the presentation! So, the projection is coming via rear projection through the robots holding the panels?

[@brettmclemore8175]

The illusion that they want for the film doesn't work live, but the actual projection mapping onto moving surfaces does work. And with projection mapping, you have one timeline with one screen. With screens you are now working with multiple timelines and multiple screens and becomes a lot harder and more time consuming to do it.

[@adam.g]

How can the canvases appear more black than the background at some points when they are white. Surely light from other parts of the room would reflect onto them??

[@jasonparker1454]

would have loved to see a more "nerdy" in depth behind the scenes. But weill done in any case! Even nearly 10 years later its spectacular!

[@zarkaznews]

WHO KNOWS THE NAME OF THE SONG?

[@davidwatts4517]

Composer is Sounds Red

[@JorickBronius]

This project totally blew my mind! Such insight! This, this is why art should always be supported!

[@jaquellchandlerphoto]

Who's watching this in 2020? These guys are pioneers in Robotic art!

[@eddieandersson5570]

This is so cool. So in the final product you use both CG generated robots and the real ones? I might come across as really thick here , but If don't ask questions, we never learn anything right? I don't get why you needed CG ones at all? I get the CG canvases and the final outline of the robots at the very end of the film had to be done with CG, but the other parts?

[@Emanemoston]

I saw "Box" some years ago, I always thought it was large TV screens with movies playing on them.

[@jshaolin0]

when I saw Box it blew me away... great job guys!

[@robb600]

The X-ray view of the robots is amazing

[@brettmclemore8175]

Projection mapping is used like crazy in a lot of live concerts. Just watch the AMAs and you'll see a lot there. However, you don't see it on moving surfaces especially this much movement. I believe its the technology they want to move into a bigger audience and in real life. As cool as the video actually is, its just what they came up with to test and show this technology.

[@greggv8]

The projections are like those 3D sidewalk and street chalk drawings. There's one exact POV where it all looks correct. From anywhere else it's all distorted. At best for a live show this technique would work for a few people sitting real close together.

[@pjp100]

Everything is robotically controlled and pre-choreographed. Would be cool to walk around it and see how wired it looks. Great stuff!

[@DavidLeeKersey]

Kind of disappointed that the BtS didn't answer the big question I had. Where were the projectors and were they fixed or did they motion track as well.

[@Killjoy0329]

The work is absolutely stunning

[@xl000]

I always feel bad for the people standing in the middle of it, looking at some broken perspective, and pretending it looks amazing. Example at: 2:04 He's looking at some deformed 3d extruded shapes that don't really make sense from his point of view, and he looks amazed for the camera only........

[@waynehedd]

Were most of the visuals created in Cinema 4D?

[@Alxbeginner]

Maya. Another 3D program ;)

[@jasonkerr4549]

Maya and Cinema 4D

[@pentzzzify]

Did I get it right ? You used Maya for the graphic stuff and simulations and also to control the robots ?

[@aaronbazil]

This could revolutionize filmmaking, and more importantly THEATER!

[@derekswales8606]

What a amazing project never seen anything that is as good it is a credit to you and everybody involved keep up the amazing work

[@ChrisMartz]

i think most people at first glance have no idea what is going on here so thank you for this behind the scenes :)

[@mortezalotfi8394]

Where are the projectors? Are there two projectors? I want to do this at home.

[@macronencer]

For those asking, the first song in this video is Fly Away by Mokhov. Not sure about the other music.

[@shaunbukhuth4410]

Bravo!!! out of words. just perfect!

[@geret13]

Finally! Was waiting for this a long time :D

[@ChopLabalagun]

what software are you using for the mapping?

[@DrakiniteOfficial]

He said it several times, they used Maya for the 3D stuff.

[@ChopLabalagun]

Drakinite the 3D's and the spatial mapping are different stuff.

[@MarinoCanal]

TouchDesigner

[@vrk1131]

We want to see how you guys projected the light with precision on the board

[@user-sl309jd90]

4:19 so this is not seen in real but only captured through camera?

[@tykill]

Thanks for the reply. So the mapping is like a mask right? A filter that partially shows video. And I can't wrap my head around how they did the hollow cube thing while moving the camera. That means they should have edited the video to only be seen from a specific moving point of view which is the camera. And the robot arms and beamer mapping must be super precise in synch which it is. It's so impressive!

[@kead_davidson]

I want the soundtrack PLEASE!!!

[@GigibyteTV]

+Kead Davidson The soundtrack is by the synthband SOVIET and can be found here: sovietbot.bandcamp.com/album/monomyth. I have no idea why they aren't credited. Hope this helps!

[@kead_davidson]

Gigibyte Thank you for sharing this with me.

[@RafaelHEscobar]

+Gigibyte Without looking I found the answer to my question about that song. Thank you so much!

[@tykill]

So there is are like two beamers that project the 3D art onto the canvases? And those beamers should follow the point of vision from the camera filming it right? In order to get perspective correct. Can anyone clarify?

[@Silentnight1979]

What brand projectors were used?

[@calmetosis]

Anyone know the music, or the composer/artist? Thank you!

[@wmarler]

SoundsRed did the Box main music, it might be him/her/them. :)

[@andyprotech]

+Irã Helder soundcloud.com/soundsred/box-exposition?in=soundsred/sets/box-monomyth

[@TheO5Council]

Every time I see this video of tear comes to my eye for some reason I cried the first time I saw this

[@tylersmyler]

Its great work. But it seems like a lot of work to do for something that can be done mostly in post. And is Maya still standard? (don't ask me I'm just an AD)

[@RUFilming]

Simply cool, unique, cool and awesome!

[@baohen]

that's one of the best film project I've ever seen1

[@EdwardNavu]

This masterpiece is asking for a sequel, obviously. Don't forget the other effects of magic. :D

[@Chrisdgallego]

what about the music in background? thanks

[@aswin5792]

But where was the projector?? Oh someone please tell me

[@roboconker]

You can see the projector at 3:09 on the wall on top of them.

[@aswin5792]

@@roboconker thanks man

[@iceavsales8390]

Great work guys. You are really pushing the tools.

[@darrenplace3697]

They're using Maxon Cinema 4D to create the actual animation. This is keyframed to move around the stage. Then they're sending the keyframe data to Autodesk Maya, and the robots are interpreting how to move the canvas(s) to the exact point. From the looks of the lens on the projectors they're projecting 4K covering the entire wall. they then playback the rendered animation as the robots move. Finally, as said in the video, they move the camera around with a third robot and that's how it's done.

[@reflexmotioncontrol]

Great work, and the integration with graphics is slick. And, it's nice to see we have been vindicated..we have been using conventional robotics for motion control for many years now.

[@tedstrauss999]

Does anyone know how the canvases (boards) were prepared? What type of paint was put on them?

[@splattru]

I think it's just usual white PVC board, like those that you can buy in Hobby Lobby.

[@tylrhoot]

great job amasing what tecnoligy can do now a days that is just amazing love it love to see you guys do another one how did you get your idea for this

[@petemk73]

And from this project, they were signed on to be the major FX house for Cuaron's Gravity.

[@darrenplace3697]

I think he means that if you were actually to walk around the room and watch the video, the depth perception would not line up correctly. You would only see a flat image. The animation was rendered with the tracked camera, so it would look convincing.

[@KevinCease]

how could this have been projected from one stationary projector when, at times the canvases were rotated at deep angles away from the viewer?

[@just1video]

AMAZING!!! What song? I really like the background music

[@mnmlst1]

Me too. If you find it, please remember to share.

[@maacor]

Its called tracking projection. It projects only on specific canvas using infrared points and follows the moving object. This was done by Johnny Lee on youtube with a wiimote. Check out his page on "Foldable Displays"

[@DigitalKarlK]

So intersting... everyone needs friends this smart/fun/creative

[@Shiftactivemedia]

This project has really caught my attention. I have a project for a cycling client that I would like to create something similar but am interested to know more about the programming time and costs. Is there a contact @Box I can talk to please?

[@tykill]

Thanks a lot for clearing things up, or at least nice to hear your thoughts. One last thing. They said they wanted to translate the experience from seeing this project in real life to film, so they can reach a bigger audience. But I how would you be able to do this in standing in the room and walking around? Perspective wouldn't be correct.

[@angelina8158]

wow.. amazing.. it makes think about action, space, performance, visual. I want to try too, Inspired.

[@brettmclemore8175]

Should just be one projector, but the video it projects is mapped out to only have video in places where the canvases are. A lot of live concerts use this to project onto multiple screens and let them interact with each other. But they don't move.

[@JackBeckettasoto]

Could one replace the canvas with large retina displays???

[@BluePi1313]

So, was it motion tracking or was it an actual projector on the canvas? :O

[@MirkoSick]

you guys are amazing. great job!

[@TeddyLeppard]

So, to summarize... the point of view camera recording this is simultaneously projecting the images on the canvases?

[@EYELEVELAwakening]

brilliant....thank you for sharing

[@miguelonha]

Amazing work guys, I am truly impressed. Keep it up!

[@sonic604]

I think they have two more robots from the "camera" side with two projectors. That way, someone can program it to follow the canvases with the projectors perfectly. That's my guess.

[@divedeepinside-JK]

you boys are out of the Matrix , congrats , WOOW done well

[@FunkySh1tter]

so glad I subscribed to this channel.... So much amazing, inspiring stuff shown here…

[@nicolasaguilar5940]

This is art

[@ValentTurkovic]

They said that they worked for months on this, so what was the budget and who was paying this? I guess that there were at least 5 people working on this, and they probably don't work for pennies :)

[@__Razer]

+Valent Turkovic Um, actually Valent, you paid for it.

[@KevinStCroix]

I wondered the same the thing. I am sure they went KZbin to go viral and get some revenue to fund the artistic portion of this project. Robots doing things by day for corporations and working at night for artists. Most companies will fund art studies and experiments for exposure. The best advertising they can get is to have a couple guys design this and go viral on KZbin. Best advertising in the world because the product speaks for itself.

[@kead_davidson]

What if BD Move can be used to animate 3d prints rigged with motors? Custom animatronics and stop-motion sets would be easier to animate, I'd assume.

[@diemc7a1]

Who did the music?

[@PogieJoe]

I'm flabbergasted. Absolutely stunned! :D WELL DONE.

[@KimberleyBianca]

Because he wanted to experiment with projection mapping, not fixed screens playing video. There is no need for any of it, robots, the projectors, etc. But it is an art project and there is a concept and a will to experiment. That's what matters and they were successful.

[@salmapaulinalopez5528]

what is the name of the song?

[@brettmclemore8175]

Actually it is groundbreaking. Don't think about from the aspect of that video, but think about it from an aspect of actually being able to do this live on moving canvases. You don't have to create something with perspective like they did. They just chose that because they are creating a film. I guess most people don't know that projection mapping is used heavily in a lot of live concerts.