tree-sitter explained

I do my best to explain tree-sitter and demystify everything going on it :)
Leave a like or comment, and subscribe of course! ;)
Notes: teej.tv/presen...
Presentation tool: github.com/d0c...
Neovim: Neovim btw
Links
Twitch: / teej_dv
Twitter: / teej_dv
Github: github.com/tjd...

[@esclavodeluna8000]

Why does tree sitter sound like an insult

[@Fafr]

Treeshitter

[@unsigned_var]

Don't be such a tree sitter, man

[@radvilardian740]

I stop before continuing watching. Thank God for having you. I want to know more about this treesitter. I hope we have series for treesitter.

[@teej_dv]

hoping to do longer video later, will take some time to put together

[@hazmat86]

​@@teej_dv if only you would do a long length video about tree sitter, feels like you did though, like recently, ehhhh......guess not, must be losing it....BLAZINGLY CRAZY!

[@caschque7242]

Your videos are impressive. I think the value comes from explaining one topic so one understands, how it works on an intuitive level while keeping the video short.

[@silentobserver9095]

I never comment on any KZbin videos.... but damn, this video was really great. Not only did you talk about an interesting topic but also explained it extremely well. Please keep on releasing more of these videos. We need content creators like you.

[@Tntpker]

I actually used tree-sitter in a project a while ago making a LS for a DSL. It worked pretty well and had auto-complete, hover, signature help, and syntax error highlights. However, tree-sitter's error recovery ability still remains a problem to make it a viable tool for creating LSs.

[@Richetechguy]

Great stuff, keep it up your kickstart vid has helped me out a lot on getting back into Neovim fully.

[@teej_dv]

Thanks!! glad to hear it :)

[@TankorSmash]

Haskell slander aside, this was amazing! I can't believe how much of ts you explained, along with giving clear usecases and examples right into vim. Thanks for putting this out!

[@martinocrespoalvarez4663]

I'm really jealous about the way this guy expresses ideas and concepts. Huge, huge fan!

[@slim_mike]

blown away in peices after seeing this video, I'm still recovering. This is very impresive, I want to learn scheme now

[@vojtastruhar8950]

What blows my mind is that it downloads some C files, COMPILES THEM and then runs them nice and fast and natively. Great stuff

[@mattwilly7959]

This makes a lot more sense now. Really like how you break things down in a simple way. It makes it a lot easier to understand concepts.

[@ParanoidxProd]

I would love a more in depth video on writing grammar, and more specific scanner. Been diving into writing grammars lately and so appreciate the content!

[@teej_dv]

ya, might do longer video of this later -- will have to be way longer haha

[@meatcow417]

Pulled me in with the trees in the thumbnail, stayed for the solid explanations. Thanks man!

[@jR-tm3ko]

The quality of each video you put out just gets better and better. Thank you!

[@Jscodin]

You really have a great way of explaining things in a way that anyone can understand, even beginners like myself - this was very interesting and informative, looking forward to the next explainer

[@jasonschannel]

Followed after the LSP video, and now I get this! The top quality explainers just keep coming! Thanks TJ!

[@roberto8016]

I didn't know tree-sitter uses JDSL, Tom is indeed a genius

[@intimidate123]

Great video TJ! really enjoy how you break everything down, makes it easy to understand and learn

[@LokeshKrishna1995]

Bless you for your fantastic explainers and all your work on Neovim. Huge fan!

[@codetothemoon]

Incredible. Cleared up a ton of confusion for me. Thank you!

[@funkdefied1]

Man this is dope. I never thought to try to understand LSPs and the parsers powering them. It was all just magic.

[@wadechang]

First time in my life I feel like I understand what treesitter is. Hats off!

[@rom1detroyes143]

I still don't really know what I can do with tree-sitter, but now I know what and how it does this thing in Neovim : great knowledge sharing, thanx :-)

[@megaxlrful]

This is a great explainer on Treesitter. I used Atom in the past, switched to Vim then Neovim. I knew what treesitter did inside neovim, and heard that it is the best one at its job but never really understood what is soo good about it.

[@MalakaiPrograms]

Love this video. Recently had to edit the grammar for a semi-obscure language as it was missing features and loved how simple it was. I mean it still took me 4 hours because I didn't know anything about tree-sitter but I wish I saw this video before then, it would have been so much help! Keep it up!

[@sandeepvk]

Have never understood the REAL difference between LLM, LSP and treesitter. This video was very useful

[@xcuu]

These videos are really great TJ, please keep them coming!

[@johnathonme]

Great video, thanks!!! Just the right amount of depth and standing while presenting was extra engaging!

[@gmchris4113]

This was dense. I'll be honest, a lot of it went right over my head as I'm starting to learn now, but I hope to return in time and realize it all makes sense 😅. Great content, I love the educational stuff. Keep it up.

[@maxnrm]

I'm on the journey of configuring neovim from scratch, this is second video I watched, first one was about Abstract Syntax Tree. I think I would be able to set up neovim in under 100 hours. Probably a 5-8 videos more and after that practice for 80 hours. Then finally I would be able to use neovim how I like.

[@PrestonThorpe-d1x]

TJ you're my new favorite youtuber/content creator. I really hope the full time thing works out for you, good luck. Definitely need more videos like these. Also now I hope you are able finish the interpreter in OCaml series one day :D

[@theIbraDev]

Thank you Teej for this, turns out i actually had no idea what treesitter actually did... keep it up!

[@faraonch]

Very well explained. Although, as a Mac power user it's still hard to overcome all the entry barriers. It's too easy to forget all the needed fundamentals that are needed to switch to neo vim. (as an active freelancer working on projects)

[@Jojor11]

I had the ideas but this really helped me connect them together… treesitter really is extremely powerful and I’m already getting ideas of cool things I could use it for if I learn it a bit

[@teej_dv]

So great to hear!! thanks for the nice words :)

[@JosephMcMurray1984]

You know this video is going to be good when Teej brings out the blazer.

[@jong.4864]

This guy is a machine!! Def much better than that netflix-btw-guy

[@jimshtepa5423]

who ends his streams thinking we know his name))

[@rastko614]

Really cool video. Thank you for showing us behind the curtain!

[@TroubleChute]

What would we do without trees*itter

[@teej_dv]

we would not be sitting on the trees

[@voldllc9621]

Tresitter is clearly a tokenizer and lexical analyzer, on top of you have some productions that are general enough to build branches in the syntax tree such as braces aaround a list without committing to specific languages. Am I on the right track track here?

[@gobdovan]

I thought this video is about how to do a job where you clean and water the trees when the tree owner isn't home.

[@rddavies]

You're a truly gifted teacher among your many other talents.

[@yakomisar]

TJ your presentations skill is a masterpiece

[@NStripleseven]

The bit that I was really interested in is how the tree sitter determines what the “least number of errors” is for a given invalid program. Looks like it was never covered but I’m still curious.

[@accumulator4825]

Really good explanation, thank you for the knowledge!!!

[@joaomsa]

Great video, lots of cool tools built around the ecosystem like ast-grep

[@ivandenysenko]

Thank you for such an awesome explanation!

[@RustFlow223]

Man your explanation sound great! Keep going!

[@droydi]

Godlike TJ, well done once again with the explanations, more of this!

[@huge_letters]

Oh, I have a new question - if tree sitter has no clue about context, it just parses text to ast. How does syntax highlighting working through tree sitter know when my identifier is a function or value? What I mean is when in TS I define a ``` const num = 5; function func(){} console.log(a, func) ; ``` It would highlight num blue and func yellow - how does it even know here that func is a function? From my understanding it would only know it's just some sort of identifier

[@kublaios]

I trust this man because he's wearing a PhD jacket

[@amirhoseinfarhadi2268]

I really love this new wave of content from you champ 👍🏻

[@havocthehobbit]

Great explanation .I had no idea I was building something that already existed , Iv been writing a language query system in rust ... I just have no interest in having to go back to C or train someone else in C to be able to use tree sitter building new language libraries for tree sitter to use . I'd love to know the learning curve with tree sitter when building a language parser for a new language that hasnt been done or shared with it .

[@teej_dv]

You don't write c when you use tree sitter. You write the grammar in js and you usually have language bindings already in other langs.

[@havocthehobbit]

@@teej_dvOK that sounds interesting , so basically tell it the syntax and structure or and using javascript to create a grammar , then I could use rust bindings if I wanted to read or compile that grammar into something consumable. I was under the impression I had to use C to write and compile that grammer into a library that uses C lib exports for bindings, so that I could then consume it with a Language like Rust or whatever Lang to be able to query what it parses... So it sounds like a 2 stage approach versus the 3 phase approach I was thinking . Which means I dont need to use C at all.

[@Luna-lp9op]

"isn't that outdated and illegal now" not proud to understand that joke because i'm chronically online

[@hendrykhoza4782]

Guys what is he using for presentation ?

[@ElvenSpellmaker]

Vscode seem to be discussing potentially moving to treesitter from textmate grammars, it'll be a loooong time though.

[@sarojregmi200]

Future Nile will admire you more than anyone. Great video. Ignore the spelling mistakes if any.

[@TeamDman]

Eloquently explained!

[@matthewrossee]

Some lsp clients implement syntax highlighting themselves, right? How does neovim decide on using treesitter or lsp highlighting? I would assume that treesitter takes precedence, since it's faster, but if lsp and treesitter are unrelated, then how does it work?

[@viniciusataidedealbuquerqu2837]

this looks like a logical expression query language 10:48

[@marianouvalle895]

Amazing stuff Teej, keep it up!

[@teej_dv]

Thanks :) :)

[@M0K0K0]

Bro it's late and I was like "tree sitter ? Wtf would you like someone to watch over a tree for you ?" and I feel very stupid now

[@mjhika]

LISP MENTIONED LISP MENTIONED LISP MENTIONED LISP MENTIONED LISP MENTIONED

[@DanielPaunescu]

You said tree-sitter is not an LSP, but could you use tree-sitter to implement an LSP sort of thing on top of it?

[@andrewrush7667]

Sharp sports coat, very professional!

[@WinterSchwartz-q2j]

I hate the rest of the internet. Nobody else seemed to care about explaining what everything i was installing was doing and just that i should install it. This really, really sucks while ur trying to configure neovim on your own for the first time. Please keep it up < 3

[@thiagolopes4978]

hello TJ, may I ask you something? While using VS Code, the Vim extension makes possible for me to use Visual Block mode, and then when I enter "I" to insert text, then backspace , the text follows for each line selected, and it is pretty good to use with SQL queries for example. When I try to do this with Vim or Neovim, I just can't, and I don't think it is the 'whichwrap' option, do you know what it is? Thank you.

[@michaeletzkorn]

I could be wrong because some of the information went over my head, but it sounds like treesitter is a file syntax parser and NOT a babysitter for trees.

[@astrocat97]

I hear LSP and I think Lumpy Space Princess from Adventure Time

[@SalvaToroTorus]

Nice stuff. You explain things really well.

[@THIRSTYGNOME]

What are the security implications of auto install of the different language parsers?

[@hamzarashid7579]

You forgot to say: Hopefully this video was suitable.

[@ethernet764]

3:38 oof. I didn't come here to be roasted 😭

[@homfes]

I think I understood it. So treesitter is an interpreter that serves language protocols?

[@StephenRayner]

Subbed, this was very good thank you.

[@hechuan5075]

any chance you could do an indepth LSP video, i watched the dedicated one but would like to learn more about the capabilities and handlers

[@teej_dv]

yes, i'm planning an entire video of writing a complete lsp from scratch

[@hechuan5075]

@@teej_dvthank you than you

[@Linuxdirk]

I wish it was easier to create own custom parsers and stuff, so it’s as easy to add tree-sitter syntax highlights than it was with old Vimscript.

[@PPo2]

Ayyy helix mentioned!

[@teej_dv]

very cool editor!

[@Ancipital_]

Thanks, Teej.

[@nemethda]

To the point, simply brilliant

[@alexandrostsagkaropoulos]

That is as close to art as it can be

[@iliadmitriev01]

how did you got TS query window open? I found the way to open right tree window with :InspectTree command

[@jeffbeyer462]

was wondering this too. it's :EditQuery but looks like it's a 0.10 feature

[@sparkhum]

This was awesome. Thanks for the explanations!

[@MrSpyTubes]

Great job man! Keep up the good work!

[@teej_dv]

Thanks :)

[@huge_letters]

"LSP and Treesitter are unrelated" - but do LSPs use treesitter for text parsing for example? Like LS-server receives an event with file changes, it runs them through tree-sitter to get a parsed result and then based on AST it can generate a response to the client?

[@teej_dv]

not generally no. they usually use hand-rolled custom parsers that are optimized for their language

[@mishydevil]

Teej wearing a jacket, must be a serious video

[@rasmusravnandersen5788]

Omg, what a great explanation!

[@spr3ez]

Is that a neovim presentation plugin btw?

[@alexandersemionov5790]

He's going corporate, oh no the suit

[@teej_dv]

You can than twitch chat for that

[@darkenblade986]

i am loving these videos. really great info!

[@StephenRayner]

Where is the video on LSPs?

[@PotatoGameDev]

You are a good human, TJ.

[@JordanShurmer]

you should try a different material sports coat. Tweed or Linen perhaps

[@CROXoDyLE]

Is this like a baby sitter but for trees

[@rafalg87]

So if tree-sitter can parse files so well, I guess you could build a language server on top of it, right?

[@TymekDev]

I was thinking about that too. My guess is that tree-sitter is good at syntactic level, but for LSP you need semantics. And typically that's brought to you by some internal tooling of the language itself. I would love to hear others' thoughts, though. Edit: well, it took watching first minute of the video to hear TJ's thought 😅

[@teej_dv]

you could, but generally for that kind of project you would rather write a hand-rolled parser specifically designed for solving the problems of your language.

[@Blaisem]

From this video teej said treesitter is only for the file you're in. A language server needs to span your whole project. Probably this scale is too expensive for treesitter.

[@je12emy]

Great video, thank you, TJ!

[@iliya-malecki]

couldnt it be sql or a subset of it? everybody knows sql but what the hell is that query syntax

[@remrevo3944]

First! (Was watching the stream and read the URL before publish.)

You cheat.

[@remrevo3944]

@Benefits of watching the stream and making fast screen shots!

​@remrevo3944 😂

@@remrevo3944 😂

[@teej_dv]

very smart move. big brain

[@GeorgeTsiros]

Basically, semi-structured editing.

[@felixallistar]

first time seeing lisp and was worried about the warnings lmao, but honestly doesn't look as bad as react.

[@mike_yon]

Great video except I do infact delete the file everytime I want to change something. I run echo >> file.ml (ocaml BTW) every time I need to edit something. It's a more functional way to program. Having random, nonrecursive access to a character in a file, like having access to an element in an array introduces state into the writing of my program