justforfunc #22: using the Go execution tracer

[@joecarter4512]

One of your best episodes so far. This was really interesting, I'm up for more episodes like this!

[@Danielshenking]

Every step of inference of optimizing is amazing. Particularly love this episode by far.

[@azzalos]

I watched this again and it was super helpful. I wish there were more of these. I also find it funny how you get excited about performance improvements as if you've just discovered them for the first time.

[@sdero499]

This was the best func yet. Seriously, breaking down code and just tracing through while optimizing is insanely helpful. More observability always!

[@markmcdonnell]

I've been needing to understand profiling in go for a while. I've never really grok'ed it properly before but you've made it so super simple. Thank you ❤️

[@Dolanor]

Wanted to exactly say that. That was an amazing episode!

[@nikhilchoudhary6891]

Watching from India.The comparison is really cool and would love to watch more such videos. Thanks

[@JeffRAllenCH]

You can also tell it's working better with createRow because time said that CPU was over 600% (theoretical max was 800%). Also, beginners needed a better description of why none of your (working) programs had data races. There are two cardinal sins when using goroutines: 1. communicating by sharing (unless you know why it's safe) and 2. not knowing when your goroutines exit (thanks Uncle Dave). You did a great job of showing that even in toy programs your goroutines have to exit correctly.

[@joonasfi]

I love to learn about perf / tooling videos like these!

[@riftitsolutions2372]

I actually enjoy the go tooling because from every video I have seen and tutorials I followed on benchmarking performance, after watching your video I finally get it. Thanks.

[@chrisvaughn01]

Great episode. I really appreciate videos like this showing the tools available and how to approach performance problems.

[@pawesajnog2248]

Wow, trace tool is impressive! Another great just for func episode! Thanks Francesc :)

[@lawrencehoffman3517]

Great rundown. Immediate subscription. Going to go back and watch some of your other videos. Please keep these up!

[@hugoschmitt1772]

Love the use of music for comedic effect ;)

[@merlin2600]

Very interesting. I will immediately apply it to one of my first apps that is extremely CPU intensive and I did use the "let's create millions of routines and let Go handle it" approach.

[@JeffDupont38]

Awesome! Love seeing how to effectively use these tools

[@robertoferrer535]

Cada video te superas, gracias Francesc :)

[@chneau]

That is a super episode, I knew pprof but now I know trace ! Cheers

[@radoslavtomov9121]

Great video. Keep them coming :). Tooling and code reviews are spot on.

[@KasisnuSingh]

Love the style of presentation.

[@JammyJay86]

Great video! I like both types of videos, and like how you alternate them.

[@shadowfaxenator]

Thanks! More videos like this please

[@igustingurahokaprinarjaya7482]

AMAZING! love it! Thank you for your effort creating this awesome amazing knowledge video

[@jhaals]

Tooling and performance is great! Thanks

[@kimnielsen7818]

I really enjoyed this .. Thanks :) I wouldn't mind more about tooling or ways to optimize so please keep'em comming

[@rajendragosavi2233]

That was crazy. Thanks for explaining such. a complex topic. I am late in the race, trying to learn as much I can. :)

[@llgmusic]

Thanks, good demonstration on how to optimize code with go tools

[@caeliferog]

Great video! It would be nice if you mentioned navigation keys you use to zoom on the trace.

[@ezg5221]

Oh boy, I'll be coming back to this one

[@quillaja]

I'd like to know more about how to do optimizations in Go such as avoiding cache misses and aligning (slices/arrays of) data in memory. Also, and probably more importantly, how to reduce allocations and garbage collection.

[@morwar_]

Amazing video, the tracer tool is really good and you explained it very well. Twenty days for GopherconBR, see you there Francesc!

[@MAtloi]

This video gives me a good feeling that using the worker pools is most of the time the right pattern I use! The video was awesome! Keep these coming, because I feel like a lot of stdlib things aren't very well documented, explained or advertised. One question I have for long running programs: Can you create traces while running e.g. in a container, like it's possible with pprof via http? Thanks! :)

[@RandomShowerThoughts]

this video was incredible!

[@ronanlanigan]

Excellent video, keep them coming :-)

[@jepkofficial]

I def would enjoy more videos using the go tooling for A/B testing code performance.

[@MaisonArmani]

Thanks bro.... This is crazy helpful...

[@shaikmuzakkir]

Very informative. Can you please let me know how I can zoom into the graphs on my mac?

[@shaikmuzakkir]

I figured it out. Thanks!

[@MichaelYinger_Beginner]

Very informative. Thanks.

[@hualetwang3105]

Nice video, very useful.

[@baranda1985]

Amazing video, really enlightening!

[@soumyaprakashdatta]

This was really awesome ... loved the video :)

[@MarEkkertsen]

A really good one. Thank you sir!

[@b4ux1t3-tech]

So, you going through this helped me _finally_ grok goroutines and channels. Plus, I know about trace now (I didn't even know it existed).

[@ilyushka21st]

First of all, I want to say: Thank you, for a good job. Want to ask, what the shortcut do you use to enlarge the trace in a browser?

[@lovekeys1908]

Thanks! Great help! More vids like this, please :)

[@ugur_dincer]

Me and the boys gonna watch this again, reserved.

[@tjholowaychuk4646]

Awesome as usual!

[@arthurnn]

gracias .. great work!

[@RealShabana]

This is amazing really

[@konstantin8105L]

Thanks.

[@nxxxxzn]

I guess the best would be to divide the image into 8 parts, for each go routine..? So that each go routine gets ~500k pixels

[@valentindeleplace7860]

Great video What you call createRow, I would call createColumns. Disclaimer: I have ~50% chance of being wrong...

[@Daniel-wt9bh]

Thanks!

[@vladimirbauer6604]

Thank you Francesc! Some of patterns you've showed, have well established names, so it would be nice if you'll refer to "official" name where appropriate. For example, last pattern you've used is called bounded parallelism, as far as I know. PS: nice caveat561 gotcha :)

[@dineshdevaraj1844]

Great Man

[@andreaswestberg536]

Can the videos be watched offline? Like watching in areas where Wi-Fi not accessible

[@JustForFunc]

+Andreas Westberg KZbin Red makes it easy, but I'm sure there's other ways

[@simonarcher1510]

This is helpful, but I need to mention how challenging this is when you have a real program. It’s much harder “in the wild”. Having unit tests to ensure that your optimization did not break anything is recommended over manual inspection.

[@theyruinedyoutubeagain]

"Top see you em" for cumulative. I loved how awkward that was! xD

[@abhijitiitr]

This somehow shows that blocking send/receive on a channel(2048 times) has more overhead than the scheduler scheduling 2048 goroutines on 8 cores. Since there are only 8 goroutines in the first case and the process is CPU heavy, so I don't expect any of the 8 goroutines getting scheduled on/off from the CPU. Please feel welcome to critique this hypothesis.

[@SeshachalamMalisetti]

how should one decide on the number of go routines to spawn(here 8 routines) ? programs run on different machines.

[@JustForFunc]

+Seshachalam Malisetti you could use the runtime package to figure out how many CPUs are available

[@SeshachalamMalisetti]

yeah, runtime.NumCPU() but to find the optimal number of worker routines how to proceed ? somewhere in irc channel i heard someone saying dont use numcpu

[@patenlikoyun]

What's that text editor?

[@MarcoCarmonaT]

Visual Studio Code.

[@patenlikoyun]

Thanks

[@alexandersvensson1253]

Why do you build the program instead of using "go run" ?

[@oliverbutterfield9844]

Alexander Svensson because he was trying to time the performance of the thing. Some of the time spent in go run is compiling stuff, so it adds overhead to each test.

[@JimmyStewpot]

Alexander Svensson because the time command would also so the compile time for the binary. He mentioned briefly at the start.

[@alexandersvensson1253]

Ah, makes sense, thanks for the answers :)

[@ulissemini5492]

i ' m a l r e a d y t r a c e r

[@theyruinedyoutubeagain]

50th! Now, to watch the video.

[@jjanx4820]

That command flag lol

[@zelimirfedoran9720]

So... "don't use channels". Got it ;p

[@LuisFigueiredoz]

go channels have some overhead and can get a bit slower than a simple mutex, github.com/golang/go/wiki/MutexOrChannel A discussion on the topic: news.ycombinator.com/item?id=11210578 (I disagree that channels are bad)

[@theyruinedyoutubeagain]

"My mac has 8 cores". It's threads, not cores. Literally unwatchable.

[@theyruinedyoutubeagain]

Just kidding of course. Love your videos ❤️

[@theyruinedyoutubeagain]

Also, is the correct nomenclature hyperthreads? Otherwise it's confusing with OS threads.