Advancing Spark - Getting Started with Ganglia in Databricks

[@bramlangelaar]

Thanks for the video! I just started working with Databricks, and used Ganglia a bit. Now I am walking into some out of memory issues, I should definitely use it more.

[@mamamiakool]

Belated new year wishes Simon !! Great to see you back and sharing incredible detailed insights into Advanced Performance tuning with Spark. More power to you for doing the good work :)

[@jaimehernandez4333]

I appreciated this video. Thanks!

[@anonymouslyyours5605]

Very nicely explained. Can you add how we can get data from ganglia to generate alerts

[@hubert_dudek]

Ganglia is nice but in databricks it really should be replaced by some databricks native solution as it is quite "funny" to include smth where most tabs are not working and include png screenshots of previous states

[@thiagojuliao2214]

Is there a way to hit some ganglia endpoint to collect the metrics without consulting the frontend page? Great video btw!!

[@ravirajuvysyaraju123]

Very informative

[@StartDataLate]

just a question from 2024, is this Ganglia still in use??

[@RodrigoCastroHdz]

I thought Ganglia was no longer maintained in favor of other tools like grafana or Prometheus

[@AdvancingAnalytics]

We don't see any updates to Ganglia, but it remains baked into the Spark UI, so it's worth understanding how it works if you don't want to sync your logs to another store and build your own dashboards!

[@omarcubano2698]

Anything about gpu monitoring?

[@SatyamShukla-z7o]

this guys is so very irritating '

[@AdvancingAnalytics]

Hah, I aim to please 😅

[@kriandir]

@@AdvancingAnalytics Don't take a random internet comment to heart, you did a very well job of explaining it in an easy to follow and entertaining way