How to Turn a Ton of Data into a Hexagon Map

What happens when you have so much data that it's impossible to know how many features are stacked up on top of each other? Is there a way to quantify them? Yes! And also make a cool hexbin map.
This video is brought to you by Dr. Bin Li, an outstanding geography professor at Central Michigan University, who taught me much about GIS, data management, intellectual curiosity, and just being cool. Thanks Dr. Li!
0:00 Rousing introduction
0:21 Insanely dense data
1:07 Creating a hex mesh
2:58 Spatial join merges layers based on location
4:26 Applying a graduated color visualization
6:10 Applying a symbol-size visualization
7:40 Adding color back into the scaled symbols
8:29 Hacking a 3D surface with the data
11:02 Thanks Dr. Li!
Here's a companion blog post to accompany this video. www.esri.com/arcgis-blog/prod...
Check out some other social channels where I share how-to's and updates on random map adventures:
adventuresinmapping.com
www.esri.com/arcgis-blog/auth...
/ john_m_nelson
/ johnmnelson
/ johnmnelson

[@JohnNelsonMaps]

This data is two weeks of flights over the upper Midwest, focused around Chicago. It came from FlightAware. sometimes there's so much overlapping content in a GIS layer, that it's a challenge to do any quantitative work on it. Here's one way to quantify super dense highly-overlapping data.

[@Trashami]

I used this a few years ago for a Bee Hive Management project where we shared estimated location data with the public. I used the one-to-many join option. Then I made the hexagons a honey color and set the opacity to really low. The result was a stack of honeycomb beehives that, if I don't say so myself, looked amazing.

[@JohnNelsonMaps]

Nice!

[@dr.christophersutton4980]

Love it! I have a similar exercise for my students where I do hexbin tessellations with tornado data.

[@JohnNelsonMaps]

ah that sounds like a cool assignment.

[@heatmoon]

Most important math I ever learned, statistics. I understand quickly once into Geography professionally why it was critical, not just in remote sensing.

[@Zephirus10]

Couldn't agree more. The other areas are definitely still important but for understanding almost anything critically stats has helped me in almost every part of my life. (also useful to know when you're being lied to with stats as well)

[@JohnNelsonMaps]

i feel like statistics is under-taught in high schools, in favor of calculous and trig.

[@rodanvsandrew]

Awesome John, good tips here! Where'd you find the flight data?

[@JohnNelsonMaps]

thanks! this is FlightAware data.

[@doribellan]

So much fun. Thanks!

[@JohnNelsonMaps]

Thanks Dori!

[@doribellan]

@@JohnNelsonMaps I just subscribed. I didn’t know you worked at Esri. Cool. I’m often in meetings discussing Esri tickets and I was a speaker at the UC a few years back (when it was in person, must have been 2019). Great fun! Last year I was just in the online map gallery. I’m going to use your channel for tips and tricks, thank you!

[@Geotechdev]

Amazing John🎉❤

[@JohnNelsonMaps]

Thanks!

[@BlueMoonday19]

Awesome hacks thanks John

[@JohnNelsonMaps]

Thanks!

[@danwdoo]

How could I have missed format all symbols all this time? Amazingly informative video as always!

[@JohnNelsonMaps]

yeah, it's a huge time saver! thanks Danny.

[@robbiebowman752]

I've got to say, I love your videos! I'm currently working on a wildlife research project with a bunch of telemetry data which is usually so boring to map, but these hex's have really brought it to life! I have a question though. When I tried to run it as a 3d scene (not much use for research but thought I'd play it out and have a funk!) the "appearance" ribbon doesn't appear so I've not been able extrude the elevation. Any idea of what I've done wrong? Thank again, Your videos are top notch!!

[@JohnNelsonMaps]

Hi Robbie! That’s great to hear! Thanks for the kind words. In 3D the height of data is done using an extrusion. I’ve not done much in 3D so I can’t provide specifics.

[@Peterincan]

Hexagons are indeed the bestagons.

[@JohnNelsonMaps]

that they are. that they are.

[@JeffersonWhovianFernandes]

I just LOVED it

[@JohnNelsonMaps]

thanks Jeff!

[@gunpowder4697]

Yo-Bro John, Funny AF, and interesting at that. Thanks for the video and, will sub for sure! Hope to find more videos like this :-D

[@JohnNelsonMaps]

great, thanks!

[@gunpowder4697]

@@JohnNelsonMaps If I'd like to try this myself, is the data for the flights open and where do I get it? :- ) I'm studying my first year in GIS Engineering but the things we've learned so far in ArcGIS Pro ain't even close to look this good :-)

[@JohnNelsonMaps]

@@gunpowder4697 youtube teaches all! unfortunately, i can't share this data since it's part of a service.

[@heatherrobbins6240]

Really cool, will have to try this soon!

[@JohnNelsonMaps]

warning, it's addictive!

[@heatherrobbins6240]

@@JohnNelsonMaps symbology pane is my favorite too 😃

[@JohnNelsonMaps]

@@heatherrobbins6240 pandora's pane

[@heatherrobbins6240]

@@JohnNelsonMaps you rock sir! Look forward to checking out your other videos. Thanks for sharing and being funny at the same time.

[@JohnNelsonMaps]

@@heatherrobbins6240 thanks for the kind words and for watching!

[@StarshipsSteel]

THAT DATA IS NUTS!

[@JohnNelsonMaps]

right? it's pretty common to come across data that has a lot of overlapping features. this was a pretty cool example.

[@bobsage]

Strap in for this one!

[@JohnNelsonMaps]

appreciate it, Bob!

[@simaTeimourianmotlagh]

I would be thankful if you tell how I can solve my problem . Actually, after doing spatial join the result layer is full of Null data and I can not do symbology based on it.

[@RaisulBariShifat]

Thank you for sharing. I have question. Is there any way to calculate/visualize the Sum of shape length in each hexagon?

[@JohnNelsonMaps]

absolutely! it's a bit complex, but you can read about how i did something similar (adding up areas) here: www.esri.com/arcgis-blog/products/arcgis-pro/mapping/100-years-of-wildfire-gis-ification/ should be the nearly the same process for adding up line lengths.

[@StarshipsSteel]

It would be cool if you could apply a color ramp to the borders of a series of polygon classes. Love the graduated hex symbology.

[@JohnNelsonMaps]

thanks! you can apply thematic rules to the outlines as well as the fills, if that's what you mean. here's a video that shows it: kzbin.info/www/bejne/moOponWVhd59idk

[@StarshipsSteel]

@@JohnNelsonMaps Sorta, but independently of each other. So you could color your polygons with color ramp 1, and your borders with a color ramp with slightly darker versions of the colors. To make the lines exist but be unobtrusive..

[@JohnNelsonMaps]

@@StarshipsSteel yes, nice idea. definitely doable. and whenever it gets too unwieldy to push lots of things into one layer, i make a copy of the layer on top of the other, shown at the same time, and have a bit more simple symbology setup for that.

[@StarshipsSteel]

@@JohnNelsonMaps Sometimes I get tunnel-visioned on how I think things *SHOULD* work. Stacked layers would definitely be a way to accomplish this.

[@tonyjoffre2909]

I've never thought of using gray scale symbology as elevation TY

[@JohnNelsonMaps]

all manner of sorcery with this trick

[@Wiscmapper]

Great info, thanks. Q*Bert?! Ah, memories of 1980-something.

[@JohnNelsonMaps]

Q-bert was my first exposure to "@!#?@!" sort of stuff as a censored curse.

[@ashleymott8944]

I like the Hal 9000 Easter egg!!

[@JohnNelsonMaps]

Hal knows what's best for us, right?

[@burakt]

Hello, dear john, I've been trying to create this path of aircraft with a .csv Excel file from ADS-B data, but I haven't quite been able to do it yet, do you have a document or video that tells you steps by step?

[@JohnNelsonMaps]

hi Burak! unfortunately i've not worked with that format so i am unfamiliar.

[@burakt]

@@JohnNelsonMaps So well..What format did you use when you were working?

[@JohnNelsonMaps]

@@burakt i was using a shapefile, aggregated from the FlightAware service.

[@nicolaconte4839]

Hi! Is it possible to modify or customise certain parts of the hexagon tesselation? For example making some of them bold or coloured?

[@JohnNelsonMaps]

Sure you can do any of that

[@nicolaconte4839]

@@JohnNelsonMaps I'll try as soon as I get home and hopefully I will find it

[@nicolaconte4839]

@@JohnNelsonMaps Sorry ti bother you but I couldn't find a tool or a way to do what I was trying to with single hexagons . Do you have any tip ?

[@JohnNelsonMaps]

@@nicolaconte4839 it's hard to be specific without knowing what your goals are, but you can isolate some hexagons as separate layers. select the hexagons you want to isolate, right-click the layer, and choose "create layer from selected features." then you can style them independently of the rest.

[@nicolaconte4839]

@@JohnNelsonMaps thank you very much!!

[@Ahmedali_-qd9eg]

Hi john, how can I get free and good resolution hillshade file to arcgis? Hillshade in arcgis pro not enough (no good)

[@JohnNelsonMaps]

what format is your good hillshade file?

[@Ahmedali_-qd9eg]

@@JohnNelsonMaps dear john, is there website allow to download hillshade with good resolution?

[@JohnNelsonMaps]

@@Ahmedali_-qd9eg hi Ahmed! yes there are lots of options. you can download 30 meter resolution DEM images here, and make your own hillshade: dwtkns.com/srtm30m/ or you can use this DEM elevation service, if you are using ArcGIS, to make your own hillshade: www.arcgis.com/home/item.html?id=58a541efc59545e6b7137f961d7de883 or this hillshade image service that works at all scales: www.arcgis.com/home/item.html?id=c3587bd89d474dff8d306fefdc295083 and here is a playlist of videos where i use these hillshade sources: kzbin.info/aero/PLPjPOZQjCWEk_7kOjytjmEc4J7k6qVtmj

[@doribellan]

@@JohnNelsonMaps Awesome resource! Thanks!

[@colenovoa3119]

Where did you get this data?

[@JohnNelsonMaps]

this is data from FlightAware

[@samszotkowski]

I like the way you speak, kind of reminds me of 3blue1brown

[@JohnNelsonMaps]

Thanks Sam! I’ll have to check them out.

[@gistutorial593]

cool

[@JohnNelsonMaps]

thanks!

[@tonyjoffre2909]

So this is my 5th time watching this.......

[@JohnNelsonMaps]

what!

[@Alex_Jonson]

Hello. A hexagon is cool, but for a more understandable and visible visualization of linear data, it is better to show it with linear symbols (for example, a line, its thickness will reflect the amount)

[@JohnNelsonMaps]

that's also a useful visual method. there is no one right way to make a map.

[@WH-rj5wg]

Ran out of memory when I tried this.

[@JohnNelsonMaps]

Oh! Maybe create a subset of the data first?