Tutorial 11 for Arduino: SD Cards and Datalogging

[@sciguy14]

@RosauraVelazco You can download youtube videos using a service like keephd (dot) com if you want. All the source code ,schematics, etc are available on my website. Visit the link in the video description.

[@laragrimes855]

Best tutorial yet. I've been following the series and working my way through the book. It is so helpful to read the section then see how it applies in your tutorials. Great job! Keep 'em coming. Thanks.

[@sciguy14]

@lolypopboy777 Are you using an SD card shield? If not you need to do level shifting.

[@ofereliassaf]

Hi, great lectures!! few issues - 1. for parsing the integer 1000 you can read it to a buffer and use atoi/itoa functions... 2. return does not close the program - it just exists the function you are in...

[@jamescollins1259]

Hey Jeremy, Your awesome! Thanks so much for taking the time to put together the best tutorials on Arduino. I'm a total beginner and you make it possible to understand whats going on!

[@juliovelarde1]

hey I’m from Peru and arduino is new for me however you’ve caught my attention with this very well made tutorial I really liked them, and I hope you’ll keep on making them and thank you very much for the time you take to teach us all this its really helpful. Im sorry if there are mistakes my English is not that good.

[@sciguy14]

@RichyBTheking There are more coming - I've just been too busy with school to put them together. I'm in the middle of finals now, I have to make that my first priority. I have 4 new arduino tutorials lined up though.

[@jimroskind5301]

Spectacular intro series! I really like them. FYI: Much simpler (and pretty standard) code for converting the ASCII file value to an integer (than what you suggested around 13 minute mark, using pow() etc. etc.) is: int refresh_rate = 0; while (commandFile.available()) { refresh_rate = 10 * refresh_rate + commandFile.read() - '0'; }

[@sciguy14]

@ONixaO That's not a question... what's the issue?

[@FutureJacket]

Element 14 is really good in general. I was buying form them because of price and express shipping long before i found out they sponsored small projects like this.

[@miketodesco6915]

Thanks for all your effort on these tutorials, great review for me, since I have been out of the industry for a while

[@RichyBTheking]

@sciguy14 Thanks mate, iv just been following your vid series and your showing me alot about arduino and helping me get my head around it. Good luck with your finals and i look forward to you posting the next set of vids. Richy

[@johannfr]

No problem. Make sure you get the cable as well. It's listed under related products.

[@sciguy14]

@lilinghiew You need to initialize it as FAT16 with your computer first. (Format it).

[@aerofart]

Lol. "Whats the fun if we have it plugged into USB." (21:26). Enjoyed this video very much. Please keep them coming!

[@ImaginationToForm]

Also good tip, thanks. When I first got my Arduino Uno a couple months ago I couldn't believe it only had 2kb. I used a ZX81 computer and it had 1kb ram way back then. I might have to upgrade to a Mega someday but so far my clock fits in. I've seen it somewhere but I'll have to look for the function to return memory to check how full its getting. But I seen that someone else done a clock program he had to switch to a mega644 chip to get it to fit. Good day.

[@ImaginationToForm]

Oh good information. I look into those atoi/itoa functions. Want to read some values off of sd card for my clock project.

[@xsirfr1958]

Hey Jeremy - using a GPIO (pin 8) to power a (low power) device is a cool idea, as you have control to completely power down if needed. But if the SD card needs a big current to write, you could have a power integrity issue.

[@CalgaryCalamari]

Great series Jeremy! So much so that you inspired me to go buy a few kits and try to build a camera control rig I've always wanted to! Can't wait to get the kits! Quick question on this piece, why didn't you log the time & date on the data you acquired? Does the board not supply a clock to read from?

[@ofereliassaf]

Hi, make sure not to create too large buffer since you only have 2kb memory. If you need a large buffer - create a function for reading the input and place the buffer on the stack inside the function. this way it will be freed when the function will return,..

[@tharshan09]

I do a very similar course to you i think, however im in the UK. First year, computer systems engineering :). Anyways, great videos and awesome explanations on the electronics side of the components. It would be great if the next few videos developed on the EE side of things. Thank you for the tutorials!

[@devopssimon]

First of all, I love the tutorials, they are very easy to understand. Well done and hope you did OK in your school exams. I agree with hjjavaher could you read the bytes into a string and then convert to an integer?

[@001marselle]

Yes but you miss a point that is more command and more variable mean more memory .in real project your concern always will be the used memory .

[@friendsonearth]

Rabih Brahim not necessary, few byte, make the code more easy understand and maintenance, opposite in real project, documentation is very important, including coding, self explain is very important

[@milobard]

Jeremy, great video! I'm in the process of putting together a data logger to measure acceleration data for vibration testing. I'm probably going to need a pretty high sample rate like 1kHz. Have you tested the limits of sample rate for writing data to the SD card?

[@sciguy14]

@karandex There are good and bad classes for sure. The lab and project classes are great for teaching stuff, and the book classes are good for imparting knowledge, though I'm not a huge fan of them. At least at Cornell, the engineering students take their work seriously - there isn't much cheating, and people genuinely want to learn the material. I'm not a fan of people who do an engineering degree just to end up in finance or business.

[@ImaginationToForm]

Thanks for your videos. I first started watching them before I even had Arduino. Now I have the Uno and soon a Nano for a little robot project. Using this one to learn about using my SD Card shield for a clock project im working on. So did you graduate? Hope you did/do well and get a good job.

[@virtlink]

Note that `return` will NOT terminate the Arduino program (e.g. when an error occurs). To stop further processing of your code, use `while (1);` instead. This puts the Arduino in an infinite loop, stopping it from doing anything else.

[@hjjavaher]

Hey Jeremy, Great job my man!!! I was wondering why wouldn't you assemble the bytes coming in into an string and then use a cast or ".toInt()" function to convert it into an int?

[@kevinlutzer1616]

Thanks Jeremy! Your code helped me with a problem I was having in a project.

[@barneycarparts]

Excellent tutorial Jeremy. Thank you for making the series

[@weirdnerd10]

I have loved all of your tutorials! So helpful, thank you! But I had a question regarding reading from the SD card. How can I read multiple numbers from the SD card? I have a veryy long .csv file which has readings from a weather station but I want to be able to read these values line by line and store them into ints, floats, and longs.

[@arkadianriver]

That would definitely be easier to understand, but you'd have to convert the final string to an int (using stdlib's atoi function I think). Jeremy's math method might compile to a smaller program? I dunno.

[@chinmayb7069]

Hi. What are the changes do i have to make to interface SD card shield through ICSP header?? I think ICSP header does not have a slave select pin. How to connect in that case?? By the way, nice tutorials. These made to understand about Arduino within 3 days...

[@tromtrom009]

Question hey, thanks for that great video! I am looking for an example to read one column of the csv-file (only once a month) and sum it up to use the result in my code. Do you have this case in any other video? Greatings and thanks again

[@sciguy14]

@jacgoudsmit I'm working on the assumption that a number is passed.

[@JacGoudsmit]

Great video, but why the strange way of reading the number from the command file? refresh_rate = 0; while (commandFile.available()) { refresh_rate = refresh_rate * 10 + (commandFile.read() - '0'); } Serial.print("Refresh rate = "); Serial.print(refresh_rate); Serial.println("ms"); This is assuming that the Arduino library doesn't already have a function to do this such as strtoul or sscanf. You should probably also check that each charcter in the file is a number before you process it.

[@sinisaveir]

Hi, I have 2 questions.. 1st: Can i do the data logging with the SD card on ethernet shield as well? 2nd: I am planning to use Arduino MEGA in a project which would involve some LCDs, tons of sensors, RTC and networking, and i was wondering can i use the additional external 5V power supply just to supply sensors with enough power (with common ground, of course)? Or you think that 5V power output on arduino could handle this? I am planning to power the arduino from 9V/1A power supply.

[@elnapa091]

hello, congratulations on this video. He has helped me a lot with my graduate work. I'm having a problem: The .csv file is recording the data in one column, not into two columns, as in your video. My code looks a lot like yours.

[@6XMX9]

Hi, i have seen your 11 tutorial and is really helpful. I need to handle the data from a .csv file and i have to put each data of each column's .csv file in a string, i figured out how to build a string but... ¿How to know when a data is of another column and start a new string, there is any caracter like '\0' or something like this?, i hope you can help me... meanwhile i keep searching, thank you in advance.

[@bet28]

Hi Jeremy, I noticed that everytime I push reset button ID will start again from 1# and that is perfect for what I need, but I would like that instead of keep writing the same file every time I press reset it will create a new log file..can you help me with that? Thank you for the great tutorial!!

[@80amnesia]

Besides storing data in a sd card, is there any other solution to store data that is not removable? I’d like to log data from a temperature sensor every second for 24h. Clearly I’d need a high capacity memory chip, maybe more than 64Mbytes. Are “flash” DIP chips a good solution? Can you please advise me what to look for? Thanks, your vdos are great

[@Bob_Burton]

I have found these tutorials very helpful but your method of reading the refresh rate from the SD card seems very convoluted. It seems to me that it would have been better to read a character and concatenate it to the end of a variable until the EOF was reached or the character read was a carriage return or newline. This would allow data of arbitrary length to be read and could be implemented as a function that returns the value for further manipulation.

[@lilinghiew]

What software are you using to initialize the sd card? I did some survey on the logging stuff where the Fatfs library is kind of confusing me now. I don't really understand purpose of including the Fatfs library and is it necessary for the logging system where i didn't see any on this code appear in your tutorial. Thanks and looking forward to your reply.

[@FibonacciEngineering]

Hey Jeremy, Will the code be essentially the same for other SD shields out there (maybe just some pin changes)? I am looking at some other shields that fit my needs better. Also, is 2GB the limit of FAT16?

[@jpedamen]

Jeremy; Thanks for the video. I have a problem; whenever i run the Sd_write code and put the sd card into my computer i the sd card has corrupted filed. I have formated the sd card twice and i get the same errors; Do you know why ?

[@RichyBTheking]

@sciguy14 have you stopped making these mate its been over a month or have you not added it to the playlist yet?

[@suryasubbarao]

Hey Jeremy. I'm using a datalogger to store accelerometer data with a refresh rate in the order of micro/milliseconds. Wield this code be applicable, as you said it takes time to do the logging? Thanks in advance!

[@vedantbarje134]

Nice tutorial. I want to log data in the sd card and make a graph of it in real time, what do i do for that

[@lolypopboy777]

why do i get "Card Failture" at the first run? The input and output is called DI & DO. my pins: CS -> 10 DI -> 11 vcc.-> 8 Clk -> 13 Gnd -> Gnd DO-> 12 (I tried to change the IN/OUT put, but it didn't work. There is also pins called: IRQ, P9, SD, WP, COM)

[@will9twl]

I noticed all the files created have a date 1/1/2000 I assume that's the default setting in the SDFatLib?

[@anuthami4142]

hey Jeremy suppose i wnt to edit a data already written in the csv file, how do u do it?? hpw can i goto a particular position of data character n rewrite it??

[@OKPRODS]

what file connects the SD read/write instructions with what actually happens with the SPI interface pins?

[@suckitcidem]

Is there a reason you cant use a SD card reader through USB serial? I have a MEGA2560 and wasn't sure if i could just use my SD card reader I already have in the USB connection or another serial connection. If this is possible how would i go about doing it? Thanks in advance.

[@piotrnazarewicz2765]

The way you calculate the refresh_rate (around minutes 15-17) using decade is unnecessarily complicated (I wouldn't think of doing it this way). The straightforward solution is: while (...) { float temp = commandFile.read()-'0'; refresh_rate=refresh_rate*10 + temp; }

[@tsunami1215]

Indeed this is simplier. Saves me a little bit of storage space aswell. Thanks

[@harshitagarwal5188]

in second program don't we need to close the commands file after reading the refresh rate ?

[@antongrobbelaar8040]

Hi Jeremy. Quick question - if I wanted to connect an OLED display that utilizes digital pins 8 to 13 and also use this SD reader/writer, how do I connect them as there is only one connector for each pin? Thanks

[@SamnissArandeen]

Is there any way to get an Arduino to plug into an OBDII port in a car? I'd love to datalog oil pressures, coolant temps, and write to text any Check Engine codes.

[@nurulsurayah]

I've seen most ethernet shield have a micro sd card slot. Can that slot be used for this or I need this particular module?

[@smitamohanty6386]

Thank you so much..... Could we follow same logic for data logging from modbus sensors.

[@fifavids3101]

Hey, Super Video Man. Is there a possibility to integrate a realtime Clock in this projekt? If yes, which shield is needed for this, can i handle it with "normal" tutorial, so that i can complete it with your projekt!? Thanks so far!

[@alchemy91]

File myFile; int k; int x=0; . . . myFile = SD.open("test.txt"); if (myFile) { Serial.println("test.txt:"); do { k=myFile.parseInt(); Serial.println(k); }while(x!=10); This will get you the number in file directly. Also we can read a text file with different numbers in each line. x= number of lines Can use for loop also inplace of do...while. parseInt() terminates when a non integer value is reached.

[@powertran6755]

Awesome Jeremy, Can this be done with arduino uno lcd screen - SD equipped?

[@sukhdev79]

what is that string variable I searched for it but could not find info about it,what others do for storing string ,is they use it as array,but can not find about string variable,what is it?

[@FutureInventions]

When I try to compile my code, I get errors pertaining to the SD card library. Is there any way I can replace the library to fix it? I'm positive that that is the problem.

[@SinCitiesSin]

if we do not have that sd connector is there any other way to connect the SD card for data logging?

[@יוסיבןעזרא-צ1ד]

How can the while loop finish? As long as there bytes in the file the condition will stay true

[@salconelectric3166]

I am Learning so much from you M. Thank You

[@nazirulhaziq46]

Hi Jeremy, I implent your code to my project. the code was able to create the file name but could not write any stuff inside the file. any idea how to sort it out?

[@angelgarcia4963]

Jeremy great your tutorial only one thing if i want to read the values of my sycard and each one are separated by commas, how i can get each one without commas thank you

[@001marselle]

use if statement then compare the byte by the hex or the asci of the comma . if the the value is comma jump to the next !

[@juanmanuelgarciahernandez8730]

Rabih Brahim thank you

[@JGunlimited]

FYI, for anyone runs into similar problem where can't create a new file. Filenames with the SD.h library can be a max 8 characters in length.

[@raffa88dk]

I like your tutorial, one question though. How do i make it understand decimal numbers ? I've tried to make the refresh rate"5000.50ms" but the result in arduino becomes: "49849.99ms" when 50.50 is written in the file.

[@smoguli]

great tutorial, but you could have read the file one character at a time in a string until you encounter a blank or newline, then to a StringToInt()

[@jpedamen]

Would one be able to connect a ps2 keyboard, lcd display, and an sd card to the arduino and use it as a word processor?

[@EthanMageMao]

Hey Jeremy. I got a problem between MsTimer2 and I2C. I want to use MsTimer2 to print the value i got via I2C every sec. The timer works good without I2C, print value works without timer. But when I use them to get my goal, it doesnt work; I cant print any thing. I dont know why, can you help me ? Thanks!

[@dragoninfire123]

@sciguy14 Hey Jeremy :) I wanted to know: is there a possible way for the Arduino to execute code from an SD card? If so, can you tell me please? Thank you :)

[@田田-g8c]

can't read the number 1000 from the COMMAND.txt, shows refresh rate= nanms but I'm sure the code is no different with yours

[@chohilary1420]

Please how can you help me with the code that can enable save my data from the Arduino to an excel file using Pir sensors. i will be happy to read from you.

[@ChaplainDaveSparks]

I'm curious about something. As a former C programmer, I used the "printf" function a lot for formatting output text strings. Is this available for the Arduino, or is the library too large for the available memory?

[@wilnalynnalbarida3920]

Do you have a tutorial on how to save a fingerprint from Adafruit to SD Card?

[@rocketman4885]

Awesome video! Thanks for the help in my data caching!

[@lolypopboy777]

@sciguy14 No, i dont have the shield( i use a SDcard sniffer not MicroSD). And how do i make a 'level shifting'? Got a tutorial? :D

[@adamprzybylski5870]

Very good video. I would like to write to a new file every time I log the data, as my datalogging is a function of a button. Could anyone point me in the right direction because my attempts aren't working?

[@bet28]

did you find a way? exact same problem here..

[@terraria103]

Hi I need my arduino to write a log on my computer. Can you tell me how to make it log to the computer and also how to make it write to a log that is compatible with task scheduler? I want a button on the arduino to trigger a task. Please help :)

[@SamnissArandeen]

That was exactly the part I needed. Thank you for your help, kind sir.

[@heaanlasai866]

Confused student here: I'm planning a project where I need the speed and multiple I2C channels of a Teensy 3.6. Could I ask you: How well does Arduino coding experience translate to other platforms like the Teensy?

[@sciguy14]

@patrickjmcdowell Thanks!

[@pacsmile]

Nice tutorial, but i have some questions; is there a way to make different files in the sd card by using a variable to set the name of the file?, let's say i'm using a real time clock, and i'd like to make a datalog everyday on different files using the data from the rtc to set a name... for example: 2014_04_28.txt 2014_05_28.txt and so on....

[@soundcrane]

yea its possible.......... i used the following setup for creating a sequence of txt files named from 000-999 make and use a char array; eg void getFilename(char *filename); char filename[]="000.txt"; //put these above setup //this function for variable name void getFilename(char *filename) { if(a==10) { a=0; b++; } if(b==10) { b=0; c++; } filename[0] = c+'0'; filename[1] = b+'0'; filename[2] = a+'0'; filename[3] = '.'; filename[4] = 't'; filename[5] = 'x'; filename[6] = 't'; a++; return; } call the function before calling the SD.open function and use the *filename(variable name,i.e,''filename'' in this case) and voila :D hope this helped

[@mgtan8900]

soundcrane hi can you please be more specific...I am quite new to this and I need assistance...So what you mean is save this code: //this function for variable name void getFilename(char *filename) { if(a==10) { a=0; b++; } if(b==10) { b=0; c++; } filename[0] = c+'0'; filename[1] = b+'0'; filename[2] = a+'0'; filename[3] = '.'; filename[4] = 't'; filename[5] = 'x'; filename[6] = 't'; a++; return; } as separate file and I need to call this function: void getFilename(char *filename); char filename[]="000.txt"; on the main text??

[@harshitagarwal5188]

in second program when we are reading the refresh rate from commands.txt file, why we have taken the vaiable decade as float can we take it as int ?

[@Motorvator]

12:34

[@SeanDIDK]

Another great tutorial.

[@LynkedVideos]

Why wouldn't you just multiply the current refresh rate by 10, add the read value, and advance the read index by one until through with the string? Your 'decade' float adds confusion where unneeded. Initialize an int (not float) at zero, iterate each character in the read line with validation, and modify the existing int. Easy, huh? PS. Your dorm window faced West, which is why the temperature and light levels peaked late in the day.

[@001marselle]

I believe this is for demonstrator and eduction only , in the fact you can forget about lot of things , and do what you want .the idea is to get what to you want from the tutorial and leave the things which you think it is not required (and that is not wrong).

[@artur2277]

Can anybody suggest what to do if I combine multiple examples to have a TRH logger with mux, oled, SD card and when I try to verify them then it wants to be declared in scope even though it worked before with oled, mux and the sensor. Since I'm a civil engineer student and my c language skills are close to nothing then can anybody help?

[@ryanmonahan3824]

Can you do a similar video with a HC-SR04 ultrasonic sensor?

[@koustubhsahu1703]

Well, tutorial was good. I was wondering if some 1 could tell me how that while loop ( while reading the data ) checks the condition n stops exucuting ?????

[@happyduck70]

Hi Jeremy, when i try to write to my sd card with your sd_write it says: couldn't open log file. I can't create any files on my sd. can you help me?

[@001marselle]

2 reason , or the memory card is corrupted , or it is locked , or it is already open in your code .

[@JeppeJazzHarmonica]

Hello Jeremy. I'm having trouble coding an IF statement on my Arduino UNO. I want to datalog my variable "count" every hour, but I can't figure out how do that? I got the Adafruit Datalogging shield with RTC. can you help me with that?

[@SharonFSmith-gq5es]

+Jeppe Andersen *Hello Jeppe, download here. Arduino Microcontroller Guide. Learn how to design and make programming Arduino. FREE DOWNLOAD HERE* plus.google.com/116428027560638976608/posts/96r9KFpBur9

[@sciguy14]

@Shackdaddy1337 yes

[@sciguy14]

@JhonssonCOL85 You should be able to accomplish that.

[@crims0n.x]

Awesome tutorial as always, cheers!

[@JhonssonCOL85]

is'nt for sismic acivity.... i need store data of vibration in a truck, and i need (i think) a sample rate of 500Hz(minimum)

[@Kosmonooit]

ReadLine()? no need to close the file?

[@nonononono352]

if i dont have arduino SD card reading, iam just hav arduino uno, what can i do??