How To Record Video At 660 FPS On A $6 Raspberry Pi Camera - Part 1

[@matthiaswandel]

Didn't know about the focus thing. Very useful! I do in fact have one of those white focus adjuster thingys that came with a camera, didn't know what it was for.

[@GaryMcKinnonUFO]

Very cool indeed. I like your no-nonsense presentation, liked and subbed.

[@ulianadu9411]

Hi Robert! Thank you for a cool video and an informative guide in your blog. I was trying to repeat all the steps you described with a Raspberry Pi 4B board and Raspberry Camera v.2.1. At first I had the same issue with "Don't know how to set the GPIO pins", but this was easily solved by cloning the last versions of raspiraw and dcraw from 6by9. However, when I am starting to capturing any frames (./raspiraw command from your blog) I got multiple errors "RaspiRaw: Reg: 3815 not found!". It cannot find regs for 3808, 3809, 380A, and 380B as well. Solution: I removed raspiraw which I cloned from 6by9 and cloned back a fork-raspiraw from your repo as you have it in your blog. Then, in file camera_i2c I changed the lines 94 and 95 to those, which were added by 6by9 to support Raspberry Pi 4B: 'a02082'|'a22082'|'a020d3'|'9020e0'|'a03111'|'b03111'|'c03111'|'a03112'|'b03112'|'c03112') echo "Raspberry Pi3B / Pi3+ /3A / 4B(1G/2G/4G)" The rest of your instructions run without problems. I hope it will be helpful for someone, who has the same issue.

[@RobertElderSoftware]

Hi, Thanks Uliana that's very informative. Yes, my instructions were written based on Raspberry Pi 3 and I haven't had time to test them with a Raspberry Pi 4 (Haven't even had time to buy a Raspberry pi 4 yet).

[@wiktec]

I followed the steps you described to run my v1.3 camera on pi 4 but l have this message: raspiraw: No sensor found.

[@shahbazali-ov1yh]

Hi Uliana thanks for a wonderful solution! I could also get this working on my raspi4 v2 camera. However the images captured are too dark almost black- greyish can you help me with the manual raspiraw configuration? thanks

[@roizenberg79]

Hi uliana thank you for your post. Unfortunately i did exactly everything but the camera_i2c didnt show me anything.What could i have missed? setting GPIO for board revsion: b03111 Raspberry Pi3B / Pi3+ /3A / 4B(1G/2G/4G) Set state of 133 to 1 ---------------------------------------- ---------------------------------------- ---------------------------------------- i didnt got any address

[@rosgori]

Tell me: is this spectacular? It is absolutely amazing

[@W00PIE]

Fascinating results, thanks for sharing! Do you know if there's some way to crank this up? Maybe with a CSI interface for a desktop computer (or a Jetson Nano) and a more powerful camera module with a higher bandwidth, so that you can get a higher resolution?

[@RobertElderSoftware]

I've thought about it, but not sure if there's an easy/low-cost way to get more resolution out of it. I think these modules brush up against some limits so more resolution might require a different way of doing things (specifically with faster/more local memory).

[@JeffGeerling]

Special visitor liked the MD5 hash at 2:28

[@parkerlreed]

I wonder if the PS3 Eyetoy could be pushed similarly on a regular computer. Stock it supports 320x240@187fps. It's gspca_ov534

[@RobertElderSoftware]

I'm not familiar with that camera, but I wouldn't be surprised if it could. I think these sensors are designed to be quite generic, and a lot of the full features end up getting hidden underneath the firmware layer, but no one motivated enough to dig deeper ever takes a look at it since there are other off the shelf alternatives (although they cost more).

[@parkerlreed]

@@RobertElderSoftware I did find the man himself (Hermann) experimented with it but only for the 187 fps. www.raspberrypi.org/forums/viewtopic.php?t=193033

[@parkerlreed]

@@RobertElderSoftware Despite what v4l2-ctl says, YUV does NOT support the 320x240@187. Had to use the bayer_grbg8 to get this result kzbin.info/www/bejne/aKq1iJyjr9iHY8U

[@parkerlreed]

@@RobertElderSoftware Also for giggles I overlayed the same shot between the Eyetoy at 187 and phone at 240 (I rendered the Eyetoy at the phone's 28.blah fps so the footage ends a little sooner and isn't quite in sync) kzbin.info/www/bejne/f5W1Zn1tn7Caprs

[@jordils6911]

Hi Robert! This is impressive! I am a university student, and I am working in a project where I need to project images over a moving object. So latency is crucial. To get the 3d object position, I am looking for a cheap way to get 2 live video streams without any latency, to process them with OpenCV. I tried connecting several cameras to my MacBook, but all of them have at least 100 milliseconds latency. Could I get better results with a pair of Raspberry Pi cameras at 660 fps and send a 3d vector to my MacBook on each video frame? Thanks a lot!

[@RobertElderSoftware]

Achieving low latency on a streaming video feed can be challenging, and you'll have to investigate what performance tuning variables are available to you to get an acceptable result. Raw uncompressed video takes up a lot of data, so naturally, we employ video encoding methods that use compression. In order for the compression to work, it has to look at several frames and pick out the repeated data. This requires having all the data over multiple frames in advance, and this is often done by buffering a few frames before they get processed/compressed and sent out. You might be able to find a way to configure how many frames are stored in the buffer. You may also get lower latency by using less compressed video, but then you run into the problem of having more data in the frames to move around, and more data will be slower to transfer. I don't know what software you're using, but to minimize latency, you could try going close to raw uncompressed video with a frame rate that is as low as possible, and a resolution that is as low as acceptable.

[@RobertElderSoftware]

I forgot to mention: In the raspberry pi, there is an entirely separate piece of silicon (that is surprisingly large) that handles video encoding. I don't think it is easy (or even possible?) for consumers to control the internal h.264 harware encoding (although I may be wrong).

[@jordils6911]

@@RobertElderSoftware Thanks a lot, Robert. That makes a lot of sense. I will try to get raw data from the camera. When you say 'with a frame rate that is as low as possible', what do you mean? I thought with a high frame rate, the projection over the object would be more accurate. By the way, I am using OpenCV and C++.

[@Abhidoesfirestuff]

​@@jordils6911hey ,I am just starting in my college project and am working on camera in motion too ,in a car ie , please help me out with what worked for you and any tips if possible

[@dadendoa]

Hi Robert thank you very much for the camera guide, I would like to ask a question about the ./camera,2c command, I am getting "setting GPIO for board revision: b03111 and Failed: don't know how to ser GPIO for this board!" error message in our Rasp pi4. Could it be possible to help us in this issue. On the other hand, the command of the config.txt and modules.conf file lines are also written in our case.

[@RobertElderSoftware]

Hi, the instructions I provided make reference to a static fork of the relevant code so that the results described here are repeatable. You should check out the original 'raspiraw' and 'dcraw' branches for updates by 6by9: github.com/6by9 and Hermann-SW github.com/Hermann-SW I specifically remember seeing some new commits related to Raspberry Pi 4 on these branches.

[@dadendoa]

​@@RobertElderSoftware thank you for your replay. By the way, I updated the camera_i2c file somewhere raspiraw_master and the message about this issue was solved. Additionally, I have a problem with the "ffmpeg" command right now. I am getting "Impossible to open 'dev/shm/out.000002.raw.tiff" /dev/shm/ffmpeg_concats.txt: No such file or directory" messages. Did you ever have any experience with the message I wrote above?

[@RobertElderSoftware]

Not sure, but in that error message, I would expect it to start with "Impossible to open '/dev/shm/..." instead of "Impossible to open 'dev/shm/...". Perhaps you're missing the leading '/' somewhere?

[@rhythmofmusic_]

Hi Robert, how can you set the sensor exposure time in microseconds ? ı can't understand what you did in 7:23 seconds. Can you explain ?

[@RobertElderSoftware]

If you don't explicitly set a value for the exposure time, you can implicitly set a max exposure time when specifying a frame rate. This is because it doesn't make sense to expose the sensor for more time than a single frame represents. I'm not sure what the exposure time defaults to, but you could probably poke around the code of raspiraw to find out. If you want to explicitly set the exposure time, see the raspiraw documentation: github.com/6by9/raspiraw -eus, --expus : Set the sensor exposure time in micro seconds.

[@luchornes]

Excellent guide. After executing the first command should there be 660 entries in tstamps.csv file?

[@RobertElderSoftware]

There will be approximately that many, I doubt it will be exact. The recording isn't real-time precise, so if the OS lags then it can skip frames or get off frequency by a bit.

[@dracleirbag5838]

wow amazing! Do you think you could do that for jetson nano?

[@RobertElderSoftware]

Good question, after some googling, it looks like the jetson nano has support for at least the IMX-219 sensor. You might have to do some software hacking with whatever library it uses to set the camera sensor's register values manually.

[@dracleirbag5838]

@@RobertElderSoftware Thanks.

[@rex25450105]

After following all the steps, entering the following command gives me "-bash: ./raspiraw: No such file or directory ". I follow the process you wrote in the blog. ./raspiraw -md 7 -t 1000 -ts /dev/shm/tstamps.csv -hd0 /dev/shm/hd0.32k -h 64 -w 640 --vinc 1F --fps 660 -sr 1 -o /dev/shm/out.%06d.raw

[@rex25450105]

When using buildme after raspiraw clonning, I got an error message. In file included from raspiraw.c:40:0: /usr/include/linux/i2c-dev.h:37:8: error: redefinition of ‘struct i2c_msg’ struct i2c_msg { ^~~~~~~ In file included from raspiraw.c:39:0: /usr/include/linux/i2c.h:68:8: note: originally defined here struct i2c_msg { ^~~~~~~ In file included from raspiraw.c:40:0: /usr/include/linux/i2c-dev.h:89:7: error: redefinition of ‘union i2c_smbus_data’ union i2c_smbus_data { ^~~~~~~~~~~~~~ In file included from raspiraw.c:39:0: /usr/include/linux/i2c.h:131:7: note: originally defined here union i2c_smbus_data { ^~~~~~~~~~~~~~

[@RobertElderSoftware]

@@rex25450105 I didn't encounter this error, so I'm not sure what the issue would be. I see that there are 'redefinition' errors, so perhaps that means that multiple versions of something i2c are installed (possibly stuff from other projects too?). If anyone else sees this error, add another comment here and I'll investigate to make sure my documentation isn't wrong.

[@ParthaPratimDas-XX]

Hello! Can we record a 64*640 video? I am just swapping the height and width as I need some extreme slo mo for my purpose for which I need to capture a vertical object. If that doesn't work I might need to set up the camera 90 oriented. btw, Thanks for the tutorial man!

[@RobertElderSoftware]

I can't remember if I ever actually tried 64x640 instead of 640x64. Having said that, I don't think it will work since the reason that it works with wide frames and not square frame with the same area is (I believe) due to physical limitations involving how the pixels are read out. I did at least one video vertically by just rotating the camera lens.

[@ParthaPratimDas-XX]

Thanks man! Another question. I am really noob in this field. Do you think this way would work on a nvidia jetson NX?

[@DoubleF3lix]

How hard would it be to lower the framerate at the cost of resolution?

[@RobertElderSoftware]

There definitely is a trade-off between frame rate vs. resolution (higher frame rates require capturing less pixels/lower resolution).

[@diygolfguy3792]

Do you think this would work with the HQ camera?

[@RobertElderSoftware]

I haven't tried this myself, but here is a forum thread where people seem to be discussing that they got 'raspiraw' running with the HQ camera: forums.raspberrypi.com/viewtopic.php?t=276047