Rules Based NER in Python (Named Entity Recognition for Digital Humanities 02)

[@python-programming]

Check out the Textbook for this series: ner.pythonhumanities.com/intro.html

[@arjunp5840]

Thank you for making these videos. One of the best videos series I found.

[@giacomograzia2471]

could you add this video to the playlist “SpaCy for Digital Humanities with Python”? thanks!!

[@python-programming]

GitHub Repo for the files for this series: github.com/wjbmattingly/ner_youtube . Please note, we will explore rules-based methods in more detail in video 4 when we learn more about spaCy.

[@shry92]

Thanks for the really detailed instruction, William I could follow along easily. With the known limitations of the rules based approach, could you please provide an example of where this approach might be used or even preferred over the ML approach?

[@python-programming]

You will actually see videos coming out over the next few weeks that talk about how spaCy 3.5 has expanded their rules-based pipelines to allow fuzzy searching which makes them even more powerful. A classic example for an EntityRuler would be something that has a finite list. I am on a project right now and the leader wanted to extract Continents as a distinct type of Location. I could include CONTINENT into an ML model, but that would be unnecessary since there are only 7 continents in the world and they appear in only a few variant forms in English (such as Oceania). This problem is much better solved with an EntityRuler.

[@anjumrohra9778]

Amazing video Sir. I am facing an issue while loading a json file using "json.load()". It results in an error as "JSONDecodeError: Extra data: line 2 column 1 (char 4144)". Any suggestions would be of great help. Thanks :)

[@python-programming]

Thanks! You need to specify the file as an object as an argument. So load(f).

[@anjumrohra9778]

@@python-programming Thanks for replying! I did it, but still, I am facing the same error.

[@python-programming]

@@anjumrohra9778 can you take a screenshot of your code and the JSON file? It may be as simple as a missing comma in the JSON.

[@anjumrohra9778]

@@python-programming sure I will. And where should I send the screenshot?

[@python-programming]

@@anjumrohra9778 Sorry about that. I probably shouldn't reply before I've had coffee =). You can send them to me as a DM on Twitter (wjb_mattingly).

[@zyxwvutsrqponmlkh]

Do what you want cause a pirate is free, you are a pirate! Yarr har fiddle dee dee Being a pirate is alright to be Do what you want cause a pirate is free You are a pirate! You are a pirate! Yeah! We got us a map (A nao!) To lead us to a hidden box Thats all locked up with locks (with locks!) And buried deep away Well dig up the box (The box!) We know its full of precious booty Burst open the locks And then we say hooray! Yarr har fiddle dee dee If you love to sail the sea Weigh anchor! Yarr har fiddle dee dee Being a pirate is alright to be Do what you want cause a pirate is free You are a pirate! Arr yarr ahoy and avast Dig a deep and youre digging too fast Hang the black flag at the end of the mast! You are a pirate! Yay! Were sailing away (Set sail!) Adventure waits on every shore We set sail and explore (Yaar har!) And run and jump all day (Yeah!) We float in our boat (The Boat!) Until its time to drop the anchor Then hang up our coats (Aye, aye!) Until we sail again! Yarr har fiddle dee dee If you love to sail the sea Land ho! Yarr har fiddle dee dee Being a pirate is alright to be Do what you want cause a pirate is free You are a pirate! Yarr har! Wind at your back lads wherever you go! Blue sky above and blue ocean below You are a pirate! You are a pirate!