Mapping over 1 Million points with Plotly Datashader

[@CharmingData]

Hey everyone, don't forget to download the data and code here: github.com/Coding-with-Adam/Dash-by-Plotly/tree/master/Plotly_Graphs/BigData Join me on LinkedIn: www.linkedin.com/in/adam-schroeder-17b5a819/

[@caroheymes3982]

Hello, this is a great tutorial. great job ! But when we go to the scale of a country or a continent, there is a problem of projection between plotly and datashaser. The image appears twisted. I tried to solve the problem with : # Project longitude and latitude onto web mercator plane. from datashader.utils import lnglat_to_meters as webm data['lat'], data.loc[:, 'lon'] = webm(data['latitude'],data['longitude']) But it doesn't work. If you have an idea it would be great !

[@gatorpika]

Nice explanation and solution. Thanks for making this!

[@CharmingData]

You're welcome @Gatorpika. Are you liking dash so far?

[@gatorpika]

@@CharmingData I was just interested in datashader, but the dash stuff looks interesting. I have a need to map millions of points for my company and our expensive commercial package doesn't do it well. I found datashader before and watched some videos of guys walking through the taxi example, but you explained more of what the code was doing which was useful.

[@leo1018]

Great Video！TKS for your sharing！

[@amir_forooghi]

Your videos should be on official dash docs. Well done =)

[@CharmingData]

Thanks Amir

[@gazul05]

Great! Thanks a Lot... Greetings from México. 💯👍

[@CharmingData]

Saludos Guadalupe. Muchas gracias por ver mi canal de Charming Data.

[@pankajsc96gtl]

Thank you

[@CharmingData]

You're welcome, @Pankaj I hope you get to work with Big Data :)

[@pankajsc96gtl]

@@CharmingData Actually I am a Telecom engineer and I deal with KPI data of telecom cells on hourly basis. With the help of your tutorials now I have my own dashboard. Things which used to took hours for me earlier now I'm doing in seconds. Many Many Thanks to you :))))

[@CharmingData]

@@pankajsc96gtl I'm glad they have been helpful, @Pankaj. In case you're interested in supporting me in making more of these tutorials, it's just a couple of coffee cups a month 🤗 www.patreon.com/charmingdata Thank you,

[@gouthamansaravanan7692]

When I try this my image is not getting overlayed on the map and there is an offset issue can anyone help?

[@Jessica112197]

Truly an awesome video as always, thanks for all your work. Saludos desde México :)

[@CharmingData]

Saludos Jessica. Gracias por tu comentario. Como te va con Dash Plotly? Lo usas para el trabajo o estudios?

[@Jessica112197]

@@CharmingData Bastante bien, lo uso para estudios. La mayoría de tus videos (si no es que todos) me sirven para trabajar en mi proyecto de tesis. Terminando veré la forma de apoyarte por todo tu trabajo :)

[@CharmingData]

@@Jessica112197 gracias Jessica. Suerte con tu tesis

[@alhdlakhfdqw]

thank you very much for sharing this great content please keep it up :)

[@ankurkumar9660]

Hey! Adam data is not available , please provide me the data

[@alhdlakhfdqw]

how does it open on a local server when you run the code? thank you

[@AugustoGeografo]

As a Geographer, I would love to use datashader, but no Geography journal will accept cartographic products without legends. Not even in my consultancies this will be well received. For the life of me, I can’t understand why the good people in holoviews would make such an incredible tool but leave it without color ramp legends.

[@משהדזיין]

hey, how does the datashader know that the data is from chicago without enter lon & lat (empty cells)

[@CharmingData]

Hi Moshe, What do you mean? I thought it does use lat lon, here on code line 22: aggs = cvs.points(dff, x='Longitude', y='Latitude')

[@משהדזיין]

@@CharmingData in my data there is also lat lon and he is not reading and show me blue screen after running the pro

[@anantharamaniyer9135]

Thank you for this one too! I have been using Plotly Dash and slowly scaling the amount of points to be plotted on a map from a few hundreds to a few million points of IoT data which is where I discovered its best to use Datashader rather than plotly maps directly as the browser has limitations of the amount of data it can handle. I also noticed Dash providing support for Holoviews and within it, also integrating Datashader (link - > dash.plotly.com/holoviews). Can you please do a tutorial on Holoviews and Datashader together? I am especially interested in Linked Selection where you can select a map over an area and the other chart with an associated graph updates based on the points selected...Thanks.

[@CharmingData]

Hi @Anantharaman Yes, Dash with Holoviews is supposed to be great for large data. I'll try to make time in the future to learn it more, and see if there is space for a tutorial on the topic. Thanks for the suggestion :)

[@anantharamaniyer9135]

@@CharmingData - i have tried the code out and also used my own dataset but found one thing missing. On Plotly Dash's page introducing Holoviews and Datashader, the code for linked selction with a map and a bar chart is incomplete. I have raised it on the community page on Plotly announcements but haven't had any response yet. Perhaps I I missing something.... would be good to have your views too. Equally you should also check out Panel by PyViz which allows for quite a pythonic way of creating dashboards without having to deal with the topic of Callbacks. If you make any tutorials on that I would be the first to subscribe!

[@luggieToast]

unfortunately the data is 404

[@JonVlogs123]

Nice graphs! 👍 MAGA