Shapefiles Need to Die

I'll show you how to send a shapefile in an email, what all of those files are for, and much more.
Link to interviews and buy me a Mtn Dew - koji.to/@GIS.C...
Link to Esri's shapefile help documentation - desktop.arcgis....

[@Jeff_Lee66]

If ESRI really want the shapefile to die, They really need to make their file geodatabase open-source/nonproprietary.

[@GISChops]

Can’t we export to xml or json to exchange data with relationships and complex features? I’m asking because I haven’t really had to, I really only deal with Esri shops and people that request shapefiles.

[@berndtnording4856]

I couldn't agree more. File geodatabases still don't have an oledb driver to allow other programs to access them, and more importantly there's no support for "real" SQL queries beyond something as basic as "SELECT Finger FROM Hand WHERE Id=3;". I've done many natural resources type GIS projects that required lots of statistical calcs. PGDB was great for that - everybody has Access, everything in one file that can easily be sent. Of course, Access also had its limitations, but that's a moot point under Pro.

[@GISChops]

I still do one process in a pgdb.

[@JeremyZimmerman]

Team Layer Package since 9.3.1 here. Shapefiles are terrible. Always love it when somebody says they sent you a shape file and all they sent you was the DOT shp.

[@GISChops]

Right? Or they sent everything but the .prj

[@bobp4036]

Once newer versions of Excel quit saving to .dbf format, shapefiles were pretty much dead to me. I used to edit the .dbf file in excel to make changes that were a pain to do in ArcMap. It was a good thing overall since it forced me to get things done outside my comfort zone.

[@GISChops]

ArcMap did not have a great table interface. I stuck with the attribute window and calculating fields with expressions. Thanks for watching and commenting!

[@FlowFidelity]

I remembered another thing I forgot to mention in the interview. If you try to add a feature by x y coordinates ( add data -> xy coordinates) , and you accidentally click an excel file instead of a csv file, ArcPro will crash, and the only way to restart it, is by closing it with task manager :/

[@GISChops]

That is a major frustration!

[@rodrigosuarezcastano732]

Topology, sub types and domains can be done in shapefiles just not in ArcGIS, same with network analysis and most of the stuff you mentioned. Geodatabases still have a long way to go to be good enough to fully replace a shapefile imo, the commodity a shapefile presents is great for small projects, in geodatabases sometimes you have upwards of 20+ shapes in a single GDB, just going through it all and finding what you actually want is really difficult compared to having 5 or so folders with shpafiles (I know there are feature datasets but again, feels like overwork to me in a simple project to go through all that hustle), geodatabases allow for multiple characters for the field names but as such, it promotes not normalizing the data, where as in a shapefile you oughta make an efford to be efficient and concise with your field names. I think a good step in the right direction to terminate shapefiles would be to allow geoprocess to take over a file, so for example if I wanna make a join of shape A and shape B, both in a dataset, allow me to make it so that the join will fully replace the A shape in the geodatabase, or make it a temporary layer in the geodatabase. -----EXTRA NOTE----- I love your videos and I think you're a very knowledgeable person regarding the use of GIS, could you please make a video on what certifications you recommend getting to find a job in this field? And how to properly prepare for them (if possible, in a budget friendly manner) I'm currently thinking of getting the 19-001 Enterprise geodata management professional ESRI certification, but i'm not sure it's worth it nor do I know how I should prepare for it. If you happen to read all this bloat thank you dearly for all the help and tutorials, material like this is crutial to further develop the GIS community and make ourselves have an easier time dominating this art.

[@GISChops]

Wow, this has to be the longest comment yet. :) I don’t work with shapefiles outside of the ArcGIS platform, but it’s good to know there are added capabilities when working with them. I just hate the multi-file format and the loss of true curves. I’ll be honest, the title and thumbnail are to get eyeballs on my content. I actually made this video to help out a friend that did one of my interviews I have been begging people to do (I still am doing those if you’re interested) - he was tired of explaining how to package and send a shapefile and wanted a video they could watch. As to certifications - I have the opinion if it’s not a requirement for the job - why pay more and spend that time to get it when it “might” give you a slight edge?

[@jakubsisak]

Shapefile has become the cross-app, cross-platform geospatial exchange file of choice. Unless there is another format that provides the same capabilities it is unlikely the format will be killed.

[@GISChops]

You’re right about it being the go to for exchanging data, but you should have said files of choice. 😄 to be honest, the title and thumbnail are to attract eyeballs. I send shapefiles as often as everyone else.

[@mattbeal8231]

Shapefiles. Ruining your meticulously named attribute data since 1998.

[@GISChops]

That’s too funny!

[@GISChops]

You should make signs.

[@jimdear4304]

What format other than a shapfile would you recommend when the end user can’t use a geodatabase?

[@GISChops]

I guess that’s the problem. There aren’t a lot of alternatives. You could use a geopackage, or geojson, or live with the hassle of losing curve integrity and dealing with multiple files.

[@Jeff_Lee66]

Thanks for the video, one other limitation that may have not been mentioned is the 2GB file size limit. In reality and in my experience, file size becomes more of a problem for shapefiles at a much smaller footprint.

[@GISChops]

I did mention it as the first drawback. Unless I accidentally cut it out? I can’t remember. 🤔 Thanks for watching and commenting!

[@Jeff_Lee66]

@@GISChops you're right. I should have listened more closely.

[@GISChops]

Ha ha, I appreciate the second view and comment.

[@berndtnording4856]

In the Arcmap SDK curves were called CircularArcSegment, in the Pro SDK they are called EllipticalArcSegment so I'm guessing ellipticalsegments are now supported in a polyline and the line you exported does look elliptical. Perhaps the segmentation routine used to create shapefile approximated it with 2 arc segments?

[@GISChops]

That sounds possible.

[@FlowFidelity]

Thank you for making this special request video!

[@FlowFidelity]

I've found that shapefiles can be used in googleMaps ( not ideal but helpful when someone only has access to googleEarth.) I've also found that a lot of government agencies outside of the US are sending shapefiles. You mention around 4:38 a file geodatabase is much easier. Do you know if it is possible for non-ESRI users to open a file geodatabase?

[@GISChops]

You’re welcome, I hope it is helpful.

[@FlowFidelity]

You mentioned a few more of the advantages of using a file geodatabase. My use case is a pit different as I am mainly looking for just polygons of villages and the village names, but I have to say, one of my favorite things about shapefiles is I can take a look at the DBF, see what is basically the attribute table, without even having to open any ESRI product. So anyone with an open source spreadsheet reader can take a look at the shapefile and see see what is basically the attribute table before passing it on to whoever will be doing the mapping. Like I said my use case is pretty unique, but being able to see the DBF table when I am being sent multiple files has been kinda nice.

[@FlowFidelity]

So I guess my resistance to killing the shapefile is 1. You can open in other software. 2. You can access the attribute table in other software. 3. A lot of countries are still using them, if they went away entirely I would need to convert everything I get sent. Thanks again for making the video and for the interview!"

[@GISChops]

I doubt it will ever really die. It’s like the imperial measuring system here in the US.

[@argowinter2871]

Misalignment between the lines is due to inconsistent projections.

[@GISChops]

It’s been a while, but I’m pretty sure I exported to the same projection as the source data. It was weird that it lined up everywhere but the curve.

[@JeremyZimmerman]

You know I sent this to some friends today and we watched it and it really makes me wonder why we can't get more Crossover with Geo packages. I've used them a fair amount for my maps I make for Carrymap but ESRI seems to barely want to have anything to do with them. You can even store symbology and styles inside of them too. Have you ever messed around with Geo packages?

[@GISChops]

I have not. I’m usually only exchanging things with Esri shops or folks that want a shapefile.

[@TheReilly5000]

Geopackages are great and are especially useful for storing symbology and styles.

[@JeremyZimmerman]

Okay, two peoples enough for me, idea in here for a new t-shirt. NICE geoPackage!

[@GISChops]

Hahaha

[@GISChops]

I’ll have to try them out.

[@janetjacobson9322]

I have my entire geodatabase on my flash drive. I would like to add it to another flash drive, in case something happens to my initial. Can this be done fairly simply? Copy and paste to the new drive?

[@GISChops]

A file geodatabase is a folder that ends in a .gdb, so you can copy that folder over to another flash drive. You could even zip it, then copy it over.

[@janetjacobson9322]

@@GISChops Thank you!! I have been hammering You Tube to learn more, reference and just get caught up from being out for a couple years. I seriously look to you and your videos, for so many topics. Thank you! As I said before "You Rock!"

[@GISChops]

Thanks for such a kind comment!

[@bofademm]

Thanks again for the info. Some email services do not like .zip files. They get rejected. This makes it even more difficult to share shapefiles.

[@GISChops]

I guess I should have mentioned this in the video, but I get around that by changing the extension to .piz and tell the recipient to change it back to .zip.

[@FlowFidelity]

@@GISChops every now and then I do a similar thing with .exe files of course then the person really has to trust me :)

[@GISChops]

That is a lot of trust. 🤣😂

[@sokpiseth3896]

Brother could do 3d map in ArcGIS pro 3.2

[@GISChops]

I’ll have to look into doing that. Thanks for watching!

[@rick_davey3]

I understand many of the drawbacks of shapefiles, but I can't figure out how to send gis data outside of sending a shapefile. Can you make a video of that workflow?

[@GISChops]

I usually send a zipped file geodatabase. I select the features I want to share and right click the layer go to Data and choose export features. I create a new file geodatabase and export them to that file gdb. Then I use my file explorer to zip that folder.

[@rick_davey3]

@@GISChops Thanks for the tip!😀

[@GISChops]

Any time.

[@JeremyZimmerman]

Layer Package (LPK) in ArcMap, Layer Package in Pro (LPKX). Pro packages aren't viewable in Map, but since Map is going away, you don't need to worry about that very long. Layer Packages are great for archiving, and most importantly preserve the original format and Symbology/Labeling.

[@PatrickUTubeify]

The 90's called and they want their GIS file format back. So many other (better) ways to share data today...file geodatabases, KML, JSON, AGO, Excel....C😬verages

[@GISChops]

Ha ha, coverages. I bet a lot of GIS folks today have never had to deal with a coverage.

[@jackdrost9555]

This cracked me up, and yes I agree that shapefiles need to retire.

[@GISChops]

Thanks for watching and commenting. Welcome to Chops Nation!

[@mev2859]

.cpg is an encoding file that allows for displaying characters in different languages lets kill the shapefile, fgdb or geopackage ftw

[@GISChops]

Thanks for the info!

[@gafoorrussel2825]

You said 3 files required as a minimum to share. I believe the 3 file you mention (check the video from minute 2.30 to 3.10) Its 4 files is minimum, and you forgot the file. .prj store projection information

[@GISChops]

I agree that the .prj file *should* be required, but from my research it isn’t. I actually just deleted all but those three files from the shapefile I used as an example in the video, and Pro recognized it as a shapefile and added it to my map. It did give me an “Unknown Coordinate System” warning though and didn’t know where to put it. Thanks for watching, and thanks for the comment.

[@tophattommy2]

100% disagree with shapefiles needing to die/go away. I have to wonder how much was paid for that title or thought. What other applications can open geodatabases? As compared to .SHP

[@GISChops]

I’m not sure what you mean by how much was paid for the title or thought. The shapefile format has become the go-to for exchanging data, I don’t dispute that at all. But should it be with all of its limitations? I’ve been messing around with Geopackages lately, I’ll let you know what I find out. The title and thumbnail may be a little over dramatic, their purpose is to get eyeballs on my content, and it looks like it worked. I appreciate you watching and commenting.

[@JeremyZimmerman]

@@GISChops Layer Packages have been available since ArcGIS 9.3.1 and work brilliantly. You can use 7Zip to unzip them and you have a layer file, and extracted Geodatabases or Shapefiles, all without ArcGIS. Shapefiles... DO... Need to die. Just like the coverage. Geopackages are good, and can be used all over the place. It's an open standard and ESRI is begrudgingly supporting it. They sure try to hide it though.

[@JeremyZimmerman]

I don't use it, but I looked it up. QGIS, probably the most popular ESRI alternative and free GIS software out there, supports Geodatabases, and has in one form or another since at least 2012 from my cursory search.

[@GISChops]

Thanks Jeremy. I’ve been looking into GeoPackages lately.

[@tophattommy2]

@@GISChops please share your findings! Pro had been failing nearly as much as it has been running lately.

[@zzzhzhz8425]

I COULD NOT AGREE MORE

[@GISChops]

Ha ha. Thanks for watching!