Tutorial 11 for Arduino: SD Cards and Datalogging

[@laragrimes855]

Best tutorial yet. I've been following the series and working my way through the book. It is so helpful to read the section then see how it applies in your tutorials. Great job! Keep 'em coming. Thanks.

[@RichyBTheking]

@sciguy14 Thanks mate, iv just been following your vid series and your showing me alot about arduino and helping me get my head around it. Good luck with your finals and i look forward to you posting the next set of vids. Richy

[@jamescollins1259]

Hey Jeremy, Your awesome! Thanks so much for taking the time to put together the best tutorials on Arduino. I'm a total beginner and you make it possible to understand whats going on!

[@juliovelarde1]

hey I’m from Peru and arduino is new for me however you’ve caught my attention with this very well made tutorial I really liked them, and I hope you’ll keep on making them and thank you very much for the time you take to teach us all this its really helpful. Im sorry if there are mistakes my English is not that good.

[@jimroskind5301]

Spectacular intro series! I really like them. FYI: Much simpler (and pretty standard) code for converting the ASCII file value to an integer (than what you suggested around 13 minute mark, using pow() etc. etc.) is: int refresh_rate = 0; while (commandFile.available()) { refresh_rate = 10 * refresh_rate + commandFile.read() - '0'; }

[@miketodesco6915]

Thanks for all your effort on these tutorials, great review for me, since I have been out of the industry for a while

[@FutureJacket]

Element 14 is really good in general. I was buying form them because of price and express shipping long before i found out they sponsored small projects like this.

[@aerofart]

Lol. "Whats the fun if we have it plugged into USB." (21:26). Enjoyed this video very much. Please keep them coming!

[@sciguy14]

@lolypopboy777 Are you using an SD card shield? If not you need to do level shifting.

[@barneycarparts]

Excellent tutorial Jeremy. Thank you for making the series

[@sciguy14]

@RichyBTheking There are more coming - I've just been too busy with school to put them together. I'm in the middle of finals now, I have to make that my first priority. I have 4 new arduino tutorials lined up though.

[@ofereliassaf]

Hi, great lectures!! few issues - 1. for parsing the integer 1000 you can read it to a buffer and use atoi/itoa functions... 2. return does not close the program - it just exists the function you are in...

[@sciguy14]

@RosauraVelazco You can download youtube videos using a service like keephd (dot) com if you want. All the source code ,schematics, etc are available on my website. Visit the link in the video description.

[@xsirfr1958]

Hey Jeremy - using a GPIO (pin 8) to power a (low power) device is a cool idea, as you have control to completely power down if needed. But if the SD card needs a big current to write, you could have a power integrity issue.

[@kevinlutzer1616]

Thanks Jeremy! Your code helped me with a problem I was having in a project.

[@johannfr]

No problem. Make sure you get the cable as well. It's listed under related products.

[@devopssimon]

First of all, I love the tutorials, they are very easy to understand. Well done and hope you did OK in your school exams. I agree with hjjavaher could you read the bytes into a string and then convert to an integer?

[@001marselle]

Yes but you miss a point that is more command and more variable mean more memory .in real project your concern always will be the used memory .

[@friendsonearth]

Rabih Brahim not necessary, few byte, make the code more easy understand and maintenance, opposite in real project, documentation is very important, including coding, self explain is very important

[@tharshan09]

I do a very similar course to you i think, however im in the UK. First year, computer systems engineering :). Anyways, great videos and awesome explanations on the electronics side of the components. It would be great if the next few videos developed on the EE side of things. Thank you for the tutorials!

[@CalgaryCalamari]

Great series Jeremy! So much so that you inspired me to go buy a few kits and try to build a camera control rig I've always wanted to! Can't wait to get the kits! Quick question on this piece, why didn't you log the time & date on the data you acquired? Does the board not supply a clock to read from?

[@sciguy14]

@karandex There are good and bad classes for sure. The lab and project classes are great for teaching stuff, and the book classes are good for imparting knowledge, though I'm not a huge fan of them. At least at Cornell, the engineering students take their work seriously - there isn't much cheating, and people genuinely want to learn the material. I'm not a fan of people who do an engineering degree just to end up in finance or business.

[@ImaginationToForm]

Also good tip, thanks. When I first got my Arduino Uno a couple months ago I couldn't believe it only had 2kb. I used a ZX81 computer and it had 1kb ram way back then. I might have to upgrade to a Mega someday but so far my clock fits in. I've seen it somewhere but I'll have to look for the function to return memory to check how full its getting. But I seen that someone else done a clock program he had to switch to a mega644 chip to get it to fit. Good day.

[@piotrnazarewicz2765]

The way you calculate the refresh_rate (around minutes 15-17) using decade is unnecessarily complicated (I wouldn't think of doing it this way). The straightforward solution is: while (...) { float temp = commandFile.read()-'0'; refresh_rate=refresh_rate*10 + temp; }

[@tsunami1215]

Indeed this is simplier. Saves me a little bit of storage space aswell. Thanks

[@sciguy14]

@lilinghiew You need to initialize it as FAT16 with your computer first. (Format it).

[@sciguy14]

@ONixaO That's not a question... what's the issue?

[@LynkedVideos]

Why wouldn't you just multiply the current refresh rate by 10, add the read value, and advance the read index by one until through with the string? Your 'decade' float adds confusion where unneeded. Initialize an int (not float) at zero, iterate each character in the read line with validation, and modify the existing int. Easy, huh? PS. Your dorm window faced West, which is why the temperature and light levels peaked late in the day.

[@001marselle]

I believe this is for demonstrator and eduction only , in the fact you can forget about lot of things , and do what you want .the idea is to get what to you want from the tutorial and leave the things which you think it is not required (and that is not wrong).

[@powertran6755]

Awesome Jeremy, Can this be done with arduino uno lcd screen - SD equipped?

[@ofereliassaf]

Hi, make sure not to create too large buffer since you only have 2kb memory. If you need a large buffer - create a function for reading the input and place the buffer on the stack inside the function. this way it will be freed when the function will return,..

[@fifavids3101]

Hey, Super Video Man. Is there a possibility to integrate a realtime Clock in this projekt? If yes, which shield is needed for this, can i handle it with "normal" tutorial, so that i can complete it with your projekt!? Thanks so far!

[@antongrobbelaar8040]

Hi Jeremy. Quick question - if I wanted to connect an OLED display that utilizes digital pins 8 to 13 and also use this SD reader/writer, how do I connect them as there is only one connector for each pin? Thanks

[@rocketman4885]

Awesome video! Thanks for the help in my data caching!

[@tromtrom009]

Question hey, thanks for that great video! I am looking for an example to read one column of the csv-file (only once a month) and sum it up to use the result in my code. Do you have this case in any other video? Greatings and thanks again

[@virtlink]

Note that `return` will NOT terminate the Arduino program (e.g. when an error occurs). To stop further processing of your code, use `while (1);` instead. This puts the Arduino in an infinite loop, stopping it from doing anything else.

[@bet28]

Hi Jeremy, I noticed that everytime I push reset button ID will start again from 1# and that is perfect for what I need, but I would like that instead of keep writing the same file every time I press reset it will create a new log file..can you help me with that? Thank you for the great tutorial!!

[@ImaginationToForm]

Thanks for your videos. I first started watching them before I even had Arduino. Now I have the Uno and soon a Nano for a little robot project. Using this one to learn about using my SD Card shield for a clock project im working on. So did you graduate? Hope you did/do well and get a good job.

[@raffa88dk]

I like your tutorial, one question though. How do i make it understand decimal numbers ? I've tried to make the refresh rate"5000.50ms" but the result in arduino becomes: "49849.99ms" when 50.50 is written in the file.

[@elnapa091]

hello, congratulations on this video. He has helped me a lot with my graduate work. I'm having a problem: The .csv file is recording the data in one column, not into two columns, as in your video. My code looks a lot like yours.

[@SamnissArandeen]

That was exactly the part I needed. Thank you for your help, kind sir.

[@田田-g8c]

can't read the number 1000 from the COMMAND.txt, shows refresh rate= nanms but I'm sure the code is no different with yours

[@milobard]

Jeremy, great video! I'm in the process of putting together a data logger to measure acceleration data for vibration testing. I'm probably going to need a pretty high sample rate like 1kHz. Have you tested the limits of sample rate for writing data to the SD card?

[@salconelectric3166]

I am Learning so much from you M. Thank You

[@ChaplainDaveSparks]

I'm curious about something. As a former C programmer, I used the "printf" function a lot for formatting output text strings. Is this available for the Arduino, or is the library too large for the available memory?

[@alchemy91]

File myFile; int k; int x=0; . . . myFile = SD.open("test.txt"); if (myFile) { Serial.println("test.txt:"); do { k=myFile.parseInt(); Serial.println(k); }while(x!=10); This will get you the number in file directly. Also we can read a text file with different numbers in each line. x= number of lines Can use for loop also inplace of do...while. parseInt() terminates when a non integer value is reached.

[@RichyBTheking]

@sciguy14 have you stopped making these mate its been over a month or have you not added it to the playlist yet?

[@pacsmile]

Nice tutorial, but i have some questions; is there a way to make different files in the sd card by using a variable to set the name of the file?, let's say i'm using a real time clock, and i'd like to make a datalog everyday on different files using the data from the rtc to set a name... for example: 2014_04_28.txt 2014_05_28.txt and so on....

[@soundcrane]

yea its possible.......... i used the following setup for creating a sequence of txt files named from 000-999 make and use a char array; eg void getFilename(char *filename); char filename[]="000.txt"; //put these above setup //this function for variable name void getFilename(char *filename) { if(a==10) { a=0; b++; } if(b==10) { b=0; c++; } filename[0] = c+'0'; filename[1] = b+'0'; filename[2] = a+'0'; filename[3] = '.'; filename[4] = 't'; filename[5] = 'x'; filename[6] = 't'; a++; return; } call the function before calling the SD.open function and use the *filename(variable name,i.e,''filename'' in this case) and voila :D hope this helped

[@mgtan8900]

soundcrane hi can you please be more specific...I am quite new to this and I need assistance...So what you mean is save this code: //this function for variable name void getFilename(char *filename) { if(a==10) { a=0; b++; } if(b==10) { b=0; c++; } filename[0] = c+'0'; filename[1] = b+'0'; filename[2] = a+'0'; filename[3] = '.'; filename[4] = 't'; filename[5] = 'x'; filename[6] = 't'; a++; return; } as separate file and I need to call this function: void getFilename(char *filename); char filename[]="000.txt"; on the main text??

[@heaanlasai866]

Confused student here: I'm planning a project where I need the speed and multiple I2C channels of a Teensy 3.6. Could I ask you: How well does Arduino coding experience translate to other platforms like the Teensy?

[@smoguli]

great tutorial, but you could have read the file one character at a time in a string until you encounter a blank or newline, then to a StringToInt()

[@sinisaveir]

Hi, I have 2 questions.. 1st: Can i do the data logging with the SD card on ethernet shield as well? 2nd: I am planning to use Arduino MEGA in a project which would involve some LCDs, tons of sensors, RTC and networking, and i was wondering can i use the additional external 5V power supply just to supply sensors with enough power (with common ground, of course)? Or you think that 5V power output on arduino could handle this? I am planning to power the arduino from 9V/1A power supply.

[@adamprzybylski5870]

Very good video. I would like to write to a new file every time I log the data, as my datalogging is a function of a button. Could anyone point me in the right direction because my attempts aren't working?

[@bet28]

did you find a way? exact same problem here..

[@FibonacciEngineering]

Hey Jeremy, Will the code be essentially the same for other SD shields out there (maybe just some pin changes)? I am looking at some other shields that fit my needs better. Also, is 2GB the limit of FAT16?

[@harshitagarwal5188]

in second program don't we need to close the commands file after reading the refresh rate ?

[@hjjavaher]

Hey Jeremy, Great job my man!!! I was wondering why wouldn't you assemble the bytes coming in into an string and then use a cast or ".toInt()" function to convert it into an int?

[@sciguy14]

@jacgoudsmit I'm working on the assumption that a number is passed.

[@nazirulhaziq46]

Hi Jeremy, I implent your code to my project. the code was able to create the file name but could not write any stuff inside the file. any idea how to sort it out?

[@Bob_Burton]

I have found these tutorials very helpful but your method of reading the refresh rate from the SD card seems very convoluted. It seems to me that it would have been better to read a character and concatenate it to the end of a variable until the EOF was reached or the character read was a carriage return or newline. This would allow data of arbitrary length to be read and could be implemented as a function that returns the value for further manipulation.

[@JacGoudsmit]

Great video, but why the strange way of reading the number from the command file? refresh_rate = 0; while (commandFile.available()) { refresh_rate = refresh_rate * 10 + (commandFile.read() - '0'); } Serial.print("Refresh rate = "); Serial.print(refresh_rate); Serial.println("ms"); This is assuming that the Arduino library doesn't already have a function to do this such as strtoul or sscanf. You should probably also check that each charcter in the file is a number before you process it.

[@MichaelSmith-gd4gv]

It could represent your parents snooping around your room.

[@saptarshiganguly1683]

suggest me ways to increase data logging speed on the sd card. i have downloaded the sdfat library but i'm finding it difficult to understand the code. how to solve this prob ?

[@JGunlimited]

FYI, for anyone runs into similar problem where can't create a new file. Filenames with the SD.h library can be a max 8 characters in length.

[@crims0n.x]

Awesome tutorial as always, cheers!

[@sukhdev79]

what is that string variable I searched for it but could not find info about it,what others do for storing string ,is they use it as array,but can not find about string variable,what is it?

[@vedantbarje134]

Nice tutorial. I want to log data in the sd card and make a graph of it in real time, what do i do for that

[@lilinghiew]

What software are you using to initialize the sd card? I did some survey on the logging stuff where the Fatfs library is kind of confusing me now. I don't really understand purpose of including the Fatfs library and is it necessary for the logging system where i didn't see any on this code appear in your tutorial. Thanks and looking forward to your reply.

[@JeppeJazzHarmonica]

Hello Jeremy. I'm having trouble coding an IF statement on my Arduino UNO. I want to datalog my variable "count" every hour, but I can't figure out how do that? I got the Adafruit Datalogging shield with RTC. can you help me with that?

[@SharonFSmith-gq5es]

+Jeppe Andersen *Hello Jeppe, download here. Arduino Microcontroller Guide. Learn how to design and make programming Arduino. FREE DOWNLOAD HERE* plus.google.com/116428027560638976608/posts/96r9KFpBur9

[@TTGSNOWY]

*watching* *watching* HOLY FUCKING BREADBOARD *walks away*

[@smitamohanty6386]

Thank you so much..... Could we follow same logic for data logging from modbus sensors.

[@ImaginationToForm]

Oh good information. I look into those atoi/itoa functions. Want to read some values off of sd card for my clock project.

[@arkadianriver]

That would definitely be easier to understand, but you'd have to convert the final string to an int (using stdlib's atoi function I think). Jeremy's math method might compile to a smaller program? I dunno.

[@wilnalynnalbarida3920]

Do you have a tutorial on how to save a fingerprint from Adafruit to SD Card?

[@happyduck70]

Hi Jeremy, when i try to write to my sd card with your sd_write it says: couldn't open log file. I can't create any files on my sd. can you help me?

[@001marselle]

2 reason , or the memory card is corrupted , or it is locked , or it is already open in your code .

[@chohilary1420]

Please how can you help me with the code that can enable save my data from the Arduino to an excel file using Pir sensors. i will be happy to read from you.

[@SeanDIDK]

Another great tutorial.

[@artur2277]

Can anybody suggest what to do if I combine multiple examples to have a TRH logger with mux, oled, SD card and when I try to verify them then it wants to be declared in scope even though it worked before with oled, mux and the sensor. Since I'm a civil engineer student and my c language skills are close to nothing then can anybody help?

[@will9twl]

I noticed all the files created have a date 1/1/2000 I assume that's the default setting in the SDFatLib?

[@chinmayb7069]

Hi. What are the changes do i have to make to interface SD card shield through ICSP header?? I think ICSP header does not have a slave select pin. How to connect in that case?? By the way, nice tutorials. These made to understand about Arduino within 3 days...

[@weirdnerd10]

I have loved all of your tutorials! So helpful, thank you! But I had a question regarding reading from the SD card. How can I read multiple numbers from the SD card? I have a veryy long .csv file which has readings from a weather station but I want to be able to read these values line by line and store them into ints, floats, and longs.

[@ryanmonahan3824]

Can you do a similar video with a HC-SR04 ultrasonic sensor?

[@יוסיבןעזרא-צ1ד]

How can the while loop finish? As long as there bytes in the file the condition will stay true

[@angelgarcia4963]

Jeremy great your tutorial only one thing if i want to read the values of my sycard and each one are separated by commas, how i can get each one without commas thank you

[@001marselle]

use if statement then compare the byte by the hex or the asci of the comma . if the the value is comma jump to the next !

[@juanmanuelgarciahernandez8730]

Rabih Brahim thank you

[@suryasubbarao]

Hey Jeremy. I'm using a datalogger to store accelerometer data with a refresh rate in the order of micro/milliseconds. Wield this code be applicable, as you said it takes time to do the logging? Thanks in advance!

[@edgarsprieditis7070]

I use "catalex" micro sd card adapter and when try to run this code, it says "Initilizing Card Card Filed Couldn't access file"

[@edgarsprieditis7070]

found the problem- it was a wrong wiring.

[@rubenesfuel]

Edgars Priedītis Hello Edgards, I am using the same SD card shield than you (catalex). Could you please send me how you connect it to the Arduino? I have the same error you had. My email is [e-mail]. Thanks a lot for your help.

[@edgarsprieditis7070]

Ana Belén Sancho Pareja One of the built in example codes in Arduino software helped me to solve the problem: 1)Open up Arduino software on your computer. 2)From menu in upper left side of the screen choose File-> Examples -> SD -> CardInfo 3) Now you should have some code there. In the code it says how you should connect your pins. the following is a small part taken from that example code: The circuit: * SD card attached to SPI bus as follows: ** MOSI - pin 11 on Arduino Uno/Duemilanove/Diecimila ** MISO - pin 12 on Arduino Uno/Duemilanove/Diecimila ** CLK - pin 13 on Arduino Uno/Duemilanove/Diecimila ** CS - depends on your SD card shield or module. Pin 4 used here for consistency with other Arduino examples 4) Connect your shield, change the example code to mach the pins and upload it to Arduino. If you managed to connect your SD card shield- the example code will display the info of the SD card. If you did not manage to connect it, it will display that also and show some tips on how to solve the problem. I hope this helps.

[@jamesaddison81]

I've got to 4:23, but when i run the code, it returns an error saying "SD was not declared in the scope". Thoughts? thanks.

[@eslamali180]

did you include the SD header file "#include " ??

[@80amnesia]

Besides storing data in a sd card, is there any other solution to store data that is not removable? I’d like to log data from a temperature sensor every second for 24h. Clearly I’d need a high capacity memory chip, maybe more than 64Mbytes. Are “flash” DIP chips a good solution? Can you please advise me what to look for? Thanks, your vdos are great

[@lolypopboy777]

why do i get "Card Failture" at the first run? The input and output is called DI & DO. my pins: CS -> 10 DI -> 11 vcc.-> 8 Clk -> 13 Gnd -> Gnd DO-> 12 (I tried to change the IN/OUT put, but it didn't work. There is also pins called: IRQ, P9, SD, WP, COM)

[@6XMX9]

Hi, i have seen your 11 tutorial and is really helpful. I need to handle the data from a .csv file and i have to put each data of each column's .csv file in a string, i figured out how to build a string but... ¿How to know when a data is of another column and start a new string, there is any caracter like '\0' or something like this?, i hope you can help me... meanwhile i keep searching, thank you in advance.

[@koustubhsahu1703]

Well, tutorial was good. I was wondering if some 1 could tell me how that while loop ( while reading the data ) checks the condition n stops exucuting ?????

[@sciguy14]

@rlameiro Haha thanks :) I'm not in danger of failing anything - at least not yet ;)

[@dragoninfire123]

@sciguy14 Hey Jeremy :) I wanted to know: is there a possible way for the Arduino to execute code from an SD card? If so, can you tell me please? Thank you :)

[@steffensande975]

I have mye file and everything is stored properly, but when i open it in excel all the data is in colon A not just the ID number Can someone help me out whit how its stored in different columns? and not all just i column A

[@samtam721]

While writing to the sd card make sure you use the println function only after you've written one complete line

[@EthanMageMao]

Hey Jeremy. I got a problem between MsTimer2 and I2C. I want to use MsTimer2 to print the value i got via I2C every sec. The timer works good without I2C, print value works without timer. But when I use them to get my goal, it doesnt work; I cant print any thing. I dont know why, can you help me ? Thanks!

[@roybear6850]

thanks so much for your time,,,making these videos

[@nurulsurayah]

I've seen most ethernet shield have a micro sd card slot. Can that slot be used for this or I need this particular module?

[@harshitagarwal5188]

in second program when we are reading the refresh rate from commands.txt file, why we have taken the vaiable decade as float can we take it as int ?

[@Motorvator]

12:34

[@jpedamen]

Jeremy; Thanks for the video. I have a problem; whenever i run the Sd_write code and put the sd card into my computer i the sd card has corrupted filed. I have formated the sd card twice and i get the same errors; Do you know why ?

[@suckitcidem]

Is there a reason you cant use a SD card reader through USB serial? I have a MEGA2560 and wasn't sure if i could just use my SD card reader I already have in the USB connection or another serial connection. If this is possible how would i go about doing it? Thanks in advance.

[@OKPRODS]

what file connects the SD read/write instructions with what actually happens with the SPI interface pins?

[@ankege2682]

Hey Jeremy, Does this SD shield disable pin 11 to 13?

[@VinnyXL420]

I was wondering it myself. Depends on what shield youre using. If youre using the ethernet shield with SD card than yeah its 11-13 and pin 4 for the SD card as far as i know. Im trying to figure it out myself right now so i'll keep digging... Yep, just confirmed. 4 is the SS pin for the SD card, pins 10-13 are used for the shield.

[@sandermans15]

yes and no, it uses those pins but only if the chip select pin is enabled, if the chip select pin is disabled and the sd card isn't activated you can use these pins to tranfer data to other shields or IC's the SD card will ignore the data being send because it isn't activated and you are free to do with those pins what you want. However if you are trying to use these pins while the sd card is running and there is data being send and recieved, everything you try to do with those pins like digitalWrite will not work or cause errors on the SD card. I haven't tried it though my information above was based on working with the SPI library and not the SD card library so i can't garantee it but it also works based on SPI so it should be the same!

[@VinnyXL420]

sandermans15 Tested it today. didnt work. Either its because i didnt define pin 4 as an output, or because it just blocks it from being used. I didnt define any of the other pins as outputs or inputs and they work fine.

[@SamnissArandeen]

Is there any way to get an Arduino to plug into an OBDII port in a car? I'd love to datalog oil pressures, coolant temps, and write to text any Check Engine codes.

[@KuldeepYadav-ov7uw]

Can please give a tutorial on Multi-Tasking using arduino

[@anuthami4142]

hey Jeremy suppose i wnt to edit a data already written in the csv file, how do u do it?? hpw can i goto a particular position of data character n rewrite it??

[@sciguy14]

@patrickjmcdowell Thanks!