Project Setup: Automated Testing with Make (using queue example)

[@TheCocoaDaddy]

Bravo!!!! At my first programming job, we used make for a lot of our regression testing along with a ton of other stuff. We didn't use a framework, so we wrote unit test programs that tested all of our base classes. We wrote integration test programs to test our application and we used it extensively for environment setup (e.g. database "priming") and for test result comparison (e.g. report comparison, in some cases). I don't use make at all now, but sometimes I miss the "old days". Given the right targets, I could make a code change and run only the related tests OR I could run make "clean" and run the full regression suite. At one point, we had so many test cases, we had to split our build into two: one that ran the regression tests deemed most important to run and one that ran absolutely everything. I believe the second one had close to 300 tests (maybe a little more). Anyway, GREAT video and thanks for posting!

[@JacobSorber]

You're welcome.

[@codebreakthrough]

This video has been tremendously helpful, thank you!

[@neillunavat]

I didnt even know you can do testing with make :o I owe all my C and C++ knowledge to you. Thank you so much.

[@JacobSorber]

Any time!

[@neillunavat]

@Newtube link me resources. I am a bit naive being self taught.

[@neillunavat]

@Newtube im just 15 bruv. But yeah, i know what i need to know and i only watch those that i find informative like Mr. Jacob. Just love his videos...

[@KarlKatten]

@Newtube Imagine complaining about the quality of someone's work when you literally just post stuff other people have made

[@fillempie1501]

Good tip!. I found testing frameworks always clumsy to use. This one is rather easy. Have already used it and found some bugs in my functions. Implemented the tests it in my autotools setup.

[@neerajkale]

Always.... Always learning something new and useful on your channel....

[@JacobSorber]

Wonderful. That's the goal.

[@neerajkale]

@@JacobSorber FYI, in one of my previous companies, we used Robot Framework for Integration/Feature testing for an ECU for cranes. I never got a chance to look into how things have been setup, probably because I joined the project when 70% work was done, but all I remember is that we wrote few test keywords in a python file, write test cases in a txt file using those keywords and test the new Application feature that we wrote in C. It was pretty easy to use and would give a nice result output in an html file.

[@timokkko]

Could you make a follow up video about how to ensure that a test is not actual using periferal hardware but uses mocks or stubs?

[@lucasteixeira5346]

Extremely useful video. Clear and meaningful info. Thanks a lot

[@JacobSorber]

You're welcome. Glad it was helpful.

[@jessiehopper]

I've been using the Catch2 framework to test my C code lately. It requires only that you add one header file and you're good to go.

[@DanielMuringe]

You post some really legit content.

[@jerrycrane6830]

Great video,. I notice that Criterion doesn't support mocking. What mocking package do you use with it?

[@edgeeffect]

This is cool! I've often wondered how "you people' do automated testing in C.

[@vvinoth514]

❤️ From India

[@JacobSorber]

❤ from South Carolina. Thanks for being here.

[@rohitjoshi4341]

I really like your videos on make. I would appreciate if you could provide videos where we can use makefile to link 3 party libraries like boost (header files libraries) and unlike boost lib. In both way shared or dynamic linking and static linking. Also, I would like if you can make videos on best practices while making a makefile. Also, my personal question can we use a nested makefile. I mean we add makefile of small projects then combine them to make a bigger one while making the project. If we can can you add how to do it. Also, best practices for it.

[@muhammedthahirm423]

is it fine if we haven't freed memory in test file. will it automatically be freed if any exception occurred?

[@JacobSorber]

Any time a process ends, any non-shared memory will be unmapped.

[@mrafayshams3089]

How can I write my unit tests when using an IDE like MPLAB X? If I create a new folder named tests and in it, I write my unit tests; I don't want them to be compiled and be part of the actual code that will go into the microcontroller. I only want to run the unit tests when I need to or whenever I build the main code. Please guide as I'm very new to this.

[@benjaminshinar9509]

really great!

[@dzious]

Hi, Just a little warning, I've got a ln issue few years ago with criterion. When I used a structure with more than 12 elements (if I remember correctly) it used to make tests fail. An other library that you can use to test your code which doesn't have this issue is Gtest (from Google) Overall this is a good video as always. Really good introduction to tests and how to implement them in a project. Maybe you should just add a link to Criterion in the description to make it easier for people to find the lib and the assertion references. I'll add few commands to my comment later on which can be useful with criterion

[@JacobSorber]

Interesting. Thanks. I haven't run into this issue, but I might just not put enough things in my structs. 🤔 I'll have to check it out.

[@michealking6425]

Where can I get the criterion library on windows?

[@sleepib]

Any ideas for automated testing of embedded code? The only thing I can think of that sounds like a reasonable use of time is to choose a platform like raspberry pi, where you can develop on the target hardware, or at least something close enough to run natively. Though I don't have any ideas for something like an 8051 microcontroller.

[@t0m0b0nes]

Great video as always 🙏 Now there is no excuse anymore to not use unit testing in my current/following project 👍 One general question about unit testing: Would you only test the interfaces (.h files) or also functionality in the actual implementations (.c files) with unit testing? For me, my naive concept of unit testing is just testing the interfaces. Is that too narrow? Best wishes from Munich! ☀️

[@JacobSorber]

Hi Thomas! I would recommend testing everything you can. The smaller the "units" are that you are testing, the easier it is going to be to make sure you're testing everything. Testing the external-facing interfaces is usually easier than testing internal auxiliary functions. And, if testing those internals becomes too painful, that's ok. The point is to make sure that your tests exercise all of your code - all of your functions/lines of code as well as each of your interesting edge cases. So, the bigger the "unit" the more thought needs to go into testing it thoroughly. I hope that helps. Best of luck with the project.

[@ivankryvosheia446]

Extremely useful! Question: is it a good method for cross-compilation? I mean, I develop for MCU but compile on my linux desktop. Or I have to somehow run tests on a target? thanks

[@LeFede]

how can I use it on windows?

[@rajamchakradar6711]

hi Iam new to automated unit testing , i have followed you in this vedio and try to frame this framework,but dont know why i cant able to get output as it shows in your system .In my case makefile is not linked with test file and getting this in output screen ar -cvrs lib/hello.a obj/hello.o a - obj/hello.o for test in ; do ./$test; done can you please help me with this..

[@KangJangkrik]

Any idea how to use CMAKE instead? My team currently using CMAKE as a build tool but we use python instead to test because nobody knows how to do that in CMAKE

[@neerajkale]

Looking forward to CMake.

[@chonchjohnch]

My habit is to create my first make statement with all my settings and files and whatnot and then factor things out

[@honey-py9pj]

I know it's called unit testing but in order to check my result for a school assignment which may include multiple functions (or any expect of main) , can I still use a unit testing framework and treat my whole program as a function? to test?

[@k8236]

Great video Jacob! How would I go about using criterion on another machine-VM gets the code from a git repo? Any help would be greatly appreciated!

[@JacobSorber]

Yes, you could do it that way. I often use vagrant to create the VM and then just share a folder between host and guest. But, a git repo would work, as well.

[@k8236]

@@JacobSorber I ended up using a modified minunit setup! Just made it fancy with colours and more testing options! Thank you.

[@wilfreddv]

The "Test" functions (?) look weird to me, and especially the TestSuite one with the `.init=suitsetup`. Can someone tell me what is going on? Heavy preprocessor use?

[@killermonkey1392]

Without having looked at it at all, I'll take a wild guess and say that most of it are infact preprocessor macros, and that the `.init=suitesetup` are probably copy-pasted into a Struct initialization.

[@sumitbhosale3401]

Nice Explantion. But Can You Please Make On Video In Graph. Thanks

[@pseudopseudo3679]

Is it a poor idea to just use a set of standard assertions to test rather than criterion?

[@daniellowry]

Word of warning with the $(RM) it can and will remove your OS and code if you're not careful. I lost a week of work and a bunch of OS configuration when i went to make and it ran the clean step. Be very careful to make sure you've got it right before running make as you won't know that it's destroying everything before its too late

[@diceandbricks]

Nice. How does criterion detect and call each test case?

[@dzious]

Criterion uses macros to "detect" each test case and build its function at compile time if you use VScode you han hover the Test keyword and you'll see the implementation of it

[@sumdim6427]

Nice!

[@DipankarDas-cm1zs]

Which is the best testing framework?

[@DipankarDas-cm1zs]

for C

[@abcddd580]

Dr.Sorber, do you use CMake? If so, I'd appreciate if you could do a series on it.

[@rmando_v]

Wow :0

[@cernejr]

Please do not use make for testing in 2021, just don't. Use python, or some other readable and maintainable language/platform. For building projects make is still fairly competitive (especially when used with cmake), but for testing it is a really poor choice.