Make YOUR OWN Programming Language - EP 1 - Lexer

[@billclinton4913]

Imagine running an interpreted language over an Interpreted language.

[@CodePulse]

Haha yes

[@pluscrafter7117]

Many compilers do that. I think it's called bootstraping and there is a python interpreter that is programmed in python (pypy).

[@ramakanthrama8578]

@@CodePulse Lol

[@e.6z1]

@@universenerdd YOU are dumb, Pluscrafter NEVER said Python was compiled and at least Pluscrafter knows the difference between their, they're and there

[@universenerdd]

@@e.6z1 okay, so what i meant was interpreters are not compilers, if you look at the original comment it makes sence, and i do know the difference, if that was what you were implying

[@finndotbin]

Me: *blinks* Him: Now we are done let’s code the entire f-ing universe edit: dang this was a year ago when I didn't know how to code and now I feel bad for commenting this. Amazing video

[@dittomario]

@Yousef GG! sdhgasuydgsduygaysudgyasdgyaudhahdyaghyiuasdgyasgy!!!!

[@dominiktoser8531]

thats when you blink very slow

[@dominiktoser8531]

@Yousef GG! LANGUAGE

[@soupnoodles]

@@dominiktoser8531 dafaq

[@nitro-ww6sk]

He speaks quite fast, this causes some times different words to be pronoucned wrong. For example he means to say LEXER and i hear LECTURE or LETUCE.

[@shouryshinde4145]

10:35: "generates all the tokens correctly..." Me: *runs program* output: [ ]

[@hasanekmen7873]

Input : 9 * 9 Output : Input : d Output : Illagel Char : d Input : d * 9 Output :

[@gamingforever65]

I have the same problem is there any fix

[@LavenderCat175]

@@gamingforever65 Yes please encountering the same issues

[@nathanaelchia7167]

It should be at the tokens list, the [] should have a space in between

[@HT79]

It took KZbin 2 years to recommend this amazing tutorial.... Really well done mate! It's inspired me enough to dabble on my own version. Not sure about an entire language but could be really useful for making CLIs.

[@CodePulse]

Thanks! Yes definitely useful for CLIs, and also for parsing your own custom data file format.

[@Markadown]

And it took KZbin 5 years to recommend it to me, lol.

[@hamzacasdasdasd]

@@Markadown me to

[@camclarke762]

Great tutorial mate, but can we just take a moment to gaze in awe at this man's typing speed and code cleanliness like sheesh

[@chandlercoates7637]

now create another language with that basic language you're creating

[@dimitar.bogdanov]

Funnily enough, when you are able to do that, you can call your language Turing-complete (it can recreate itself, in itself), and is mostly a good-enough programming language for most basic tasks.

[@hacklordmonster8780]

Good luck with the speed of it

[@0x1EGEN]

@@dimitar.bogdanov You mean "self-hosting".

[@dimitar.bogdanov]

​@@0x1EGEN Your original reply: "You mean "self-hosting", turing complete just means it can theoretically compute anything. en.wikipedia.org/wiki/Self-hosting_(compilers)" An answer: no. A self-hosting compiler can compile itself, yes, but a language can be interpreted, and you could still make another language with it. Being Turing-complete means that it can simulate a Turing-machine. Taken from Wikipedia: "For example, an imperative language is Turing-complete if it has conditional branching (e.g., "if" and "goto" statements, or a "branch if zero" instruction; see one-instruction set computer) and the ability to change an arbitrary amount of memory (e.g., the ability to maintain an arbitrary number of data items)." That means, that as long as you have if statements and some way to loop (be it goto statements or a for loop), your language is turning complete.

[@elliotk9291]

I'm actually doing that right now

[@Drqonic]

This is amazing to be honest. I've never seen anyone else code as cleanly as you. Do you think you could make a video on making a compiler next time?

[@CodePulse]

Thanks! I do plan on doing that soon. The later episodes in this series have gotten more messy and complicated, and I'm planning on redoing this series first. I also am doing a simpler 4 part series on making a math interpreter before that. So hopefully I get around to doing a compiler soon :)

[@Drqonic]

@@CodePulse Thanks for the quick reply! I look forward to future videos :)

[@TuckerTheMandolinist]

@@CodePulse a compiler would be AMAZING!

[@sacredgeometry]

This isn't cleanly written. There is lots room for abstraction here.

[@10F2C]

You mean compiler that actually writes machine code?

[@albertlevins9191]

Dude, thanks. You don't know me, but I am a solo game designer with a solid "B" in art skills, but a "C-" in programming. I have been trying to implement a basic like script interpreter in a python program I am making. Been trying for several years to get it right. This tutorial is EXACTLY what I needed. Thanks again.

[@soumyajitkar122]

nice

[@holomode1357]

"We can multiply numbers": 1 + 2 Python default calculation: 1 + 2

[@mikethegamedev]

1 + 2 = 6

[@mikethegamedev]

SIKE its 3

[@mrwasserflasche6063]

@@mikethegamedev no it's 69,420

[@aw1lt]

@@mikethegamedev damn bro I nearly did my homework wrong because u said it was 6

[@TacticalAllRounder]

@@aw1lt ...

[@Jerraf2p0]

I can’t believe this dude wrote a full lexer without any modules. Great job

[@universenerdd]

finally, a version of BASIC that is actually basic

[@yaman09]

how help please

[@MirrorsEdgeGamer01]

Why did you make an *_as_string_* method not just use the built in Python's *___str___* method?

[@CodePulse]

I hadn't really used Python much in ages so never though of it then. I realized a while later but never bothered to change it.

[@jaysonlana9147]

@@CodePulse sorry im new to python and i recieve ths error but i dont have str in my code AttributeError: 'str' object has no attribute 'fn'

[@anshbarhate2791]

@@jaysonlana9147 If you still have that issue, it means somewhere you've typed self.fn instead of ln. Probably in def copy()

[@thelavagod]

​@@anshbarhate2791the comment was 3 years ago

[@arthur_p_dent4282]

This is great. Been working my way through Nand 2 Tetris on my own. I made it through the hardware portion all right but once it got to writing the assembler I felt completely lost. Hoping this will give me the nudge I need, so far I'm loving the content and can't wait to finish.

[@thatpenguin965]

Nice, my brother is doing that too

[@silkcut5561]

Awesome! Thanks to all channels which shows this topic. I always wondered how this works :) Thanks to all!

[@logangraham2956]

oh your lexer is so nice . it actually handles problems. my lexer just advances over them . so once/if mine ever becomes a compiler instead of a interpreter. literally everything will compile but whether or not it will run is another story entirely.

[@Markkkkkkkkkkkkkkkkk]

Nice. I'm 15 and thinking of creating one in the future. I might name it : Pascala Pascala's name is derived from Pascal + Scala

[@bmax5928]

ill take that m8

[@dimitar.bogdanov]

Don't share your project names with strangers until you've created something. Bad, bad idea

[@MrDgf97]

I’m always so envious of seeing young people with that drive. When I was 15 all I cared about was playing videogames and being cool, such useless things

[@danielshah3494]

Sounds gay af tbh

[@martilix4470]

@@dimitar.bogdanov thats the stupidest thing I’ve ever heard. The name, or even the idea don’t matter, the execution does

[@sourishsharma5745]

This channel is underrated...

[@CyberCoder]

I've got like 3 yrs of experience in C#, but only like 3 months in Python. Probably not this I should do with only 3 months of exp. but I strongly believe that my C# skills will help. Amazing tutorial ngl

[@ga_per]

WTF youtube's algorithm is already in 2080, I was thinking about creating a language yesterday and now I get this in my recommended

[@kilogram9559]

same

[@TheSpekeyMan]

Google hears ur stuff or wut ever ur saying

[@abheygupta2332]

We are making a programming language with python which is an easy language just imagine how the first programming would have been created

[@supersani21]

Well we do use C++ for Python which derives itself from assembly languages which derives itself from machine language

[@kartikeykushwaha3622]

@@supersani21 I think python's written in C.

[@GDPlainA]

@@kartikeykushwaha3622 C++ can run C code btw so its not really a big difference.

[@mcds6307]

@@GDPlainA It is. C++ is not C.

[@engelsteinberg593]

Assembly, and assembly was created with hex. Altrogut assembly is easy created.

[@sakul_the_one4821]

Really thx for this Toutorial! This was my first Toutorial that I used for Python ( If I don’t mention the Python Code in my Java File and 8 Videos of Python but I didn’t test them...) So I learned here not just how to make a programming language, I learned even Python with this Toutorial! Really thx for it!

[@cannedsoup4622]

I thought that the output of "1.2+1" would be "float: 1.2 | plus | int: 1", however I've noticed that it skips the plus sign entirely. It does that for every first character behind a number. I checked my code (I translated your code into lua and thought I messed something up) and then was suprised to see that indeed, it skips the first character that comes after any number in the make_number function. There is an easy fix to it tho! After the loop in the make_number function you need to decrement the self.pos by 1. (self.pos -= 1) After doing that it now correctfully translate the input "1.2+1" to "float: 1.2 | plus | int:1" instead of the output given above. I do not know if the same issue has happened to anyone else before, but if it did it would be nice if you could pin this message, it was not as easy to track down as it might sound from above. Nice video nontheless! It helped me out a lot!

[@UnladenSwallow227]

Are you sure you didn't call self.advance() in the while loop of make_tokens() as well, after tokens.append(self.make_number())? That would result in calling self.advance() twice, which creates the issue you ran into. I have the code in the video and I was not able to reproduce your issue at all.

[@NA-jy4zd]

The problem is just you. There is something wrong with your code. You can copy the ep1 code from github to confirm that you did everything correctly.

[@MrKoshak3]

You're just a god, I barely understand English, but I understand you more than compatriots

[@CodePulse]

That's great to hear! Thanks

[@Dorsz12303]

I always wanted to make my own programming language. And I find perfect tutorial from 2018. Thank you!

[@thinkmovies4565]

Awesome, I know have a fully working lexer and I am working on a programming language known as jublee. Which already has the basic features of addition , sub , mul and div.

[@fenderat1713]

link it and ill try it out anbd give my own feedback

[@justcallmenoah5743]

Bro you are a genius! I came for the custom OS project but I'm gonna have to watch your entire back catalogue!

[@CodePulse]

Thanks! Hope this series is as good as my operating system one.

[@justcallmenoah5743]

@@CodePulse im excited to finish this one quickly, and excited to see where the operating system goes!

[@Apoorvpandey]

The error output looks so neat! Copilot suggested me how to get output like this: calc> 12+23.3+g Illegal Character: g File , line 1 12+23.3+g ^ Error representation should be: result = f'{self.name}: {self.details}' result += f' File {self.pos_start.filename}, line {self.pos_start.ln + 1}' line = self.pos_start.filetext[:self.pos_start.col+1] result += f' {line}' result += ' ' + ' ' * (self.pos_start.col) + '^' return result

[@deckermv7530]

Absolutely great job for a talented mad man like you. Looks like this series is gonna be sick. Thanks for sharing this! Gonna learn a lot from this

[@therandomguy1701]

Finally, my next project to test my 1 month of programming experience!

[@ouioui2793]

Im creating a language called Neptune, 'cuz it is my favorite planet. I want to make it the most easy to understand programming language using a lot of simple keywords

[@CodePulse]

Good luck with it!

[@ahmedzarin2247]

@MoolsDogTwo hey me too!

[@Tashmimanam]

Make sure to make it beautiful bcuz Neptune is a beautiful planet 😁👍

[@creepersaur]

I think I got something wrong. I followed the tutorial, but what happens when I do 1 + 1 is strange. basic > 1 + 1 [, , ] NEVERMIND I forgot to put __ after __repr

[@parkeranderson23]

wow this is genuinely quality content 10/10

[@greasymushroom112]

I’m making a language called quil (yes I know there is only one “L”) this helped a lot

[@mavdotj]

What a coincidence (look at my name)

[@nathanellis7819]

THIS WAS EXCELLENT! WELL DONE MATE!!

[@HalfAsleepSam]

This man types at the speed of sound, I havent even finished typing class Token: and the video is already over like holy fuck

[@MrRyanroberson1]

7:11 really really important flaw: the while loop will keep appending same char to the string indefinitely once it starts. there's nothing to advance it. ok you caught this at 10:31, this took me a couple watches to see if you put that somewhere in the video

[@kavyaagrawal2013]

Thank you so much for this tutorial! Very well explained ;) I'll try making one in javascript! Thanks again!

[@elquelobetodo]

my teacher let me make an a compilator for my exam, i want to see his face when he try to prove the nekolenguage

[@muzafferckay2609]

You dont have to type tons of 'elif' use a dict to map symbol to its value

[@perinoveriza1658]

Im enjoy with your video...great teaching 👍👍👍❤️❤️

[@CodePulse]

Thank you :)

[@emilsayahi1]

You're fairly damn good at explaining things.

[@somethingwithtech]

i ran into a problem while coding and i literally searched for 45min befor i realised that i miss spelled a variable. Btw great Video thanks ;)

[@ScienceCodeCreations]

Good tutorial! I tried this once before when I was still pretty new at python and I didn't understand it so I gave up after the first two vids. But, now that I'm more familiar with the syntax of python, I can actually understand everything in this tutorial.

[@CodePulse]

That's awesome to hear!

[@andreweathan]

thanks for the tutorial, you have good programming skills

[@vladusa]

This video reminds me that there is always a better programmer out there.

[@jonoisedev]

Oh, man! I could watch this all day long.

[@inserteunnombreapropiado9079]

Now try to make one with assembly.

[@oldmovies3776]

How does the computer know what is basic

[@rodrigopennacchi2451]

Thank you for sharing this very clear and usefull material! it is amazing, congratilations!

[@jesnilinke5016]

If I did this in a different Programming language, do I follow these steps or should I translate it?

[@reinhold1616]

well you cant run python in an interpreter/compiler that is not for python ;)

[@IsaacHoweiner]

This is something I can work with. I love your accent

[@unh0lys0da16]

To speed things up you might want to set self.length to len(text) in __init__, else you change the complexity of advance to O(n) where n is the length of the text.

[@CodePulse]

Are you saying the Python `len` function is O(n)?

[@unh0lys0da16]

@@CodePulse aaah I've done some tests and it seems to be O(1), apparantly Python builtins (Dict, Lists etc) keep track of the size internally.

[@CodePulse]

@@unh0lys0da16 Makes sense, that's what I assumed. Different from C's strlen function which is O(n)

[@bigbabby1232]

how did you get it so that when you click new terminal it says just terminal> and not the path to the folder you are working in? Also I copied everything from github and type in the terminal 'python3 shell.py' nothing happens! Edit: I have the Python extention for VScode in version 2018.3.0

[@CodePulse]

On windows you can use the "prompt" command to change what text shows up. If you are using windows 10, it preinstalls a python3 program, but this isn't the real python, rather a link to the microsoft store to install python. This could be why nothing happens. Try just run "python" instead of "python3", otherwise you may have to update your path.

[@bigbabby1232]

@@CodePulse thanks, just changing pythin3 to python sorted it. (:

[@Cowboy8625]

Dude! Great video!!!!

[@CodePulse]

Thanks

[@Cowboy8625]

CodePulse yeah man! You write some super clean code! I’m trying to make my own language just for fun and wow this has help a lot. It’s really really fun to! Keep it up man!!

[@yeahuh4128]

I've been working on the class Error so long, now I can't pronounce the word 'error'

[@CodePulse]

Haha

[@MiMiBrokenbourgh]

FYI, it’s [ɜɹːɚ]

[@lasannishshanka4850]

Sorry CodePulse, I have some questions.. :) 1. Python is a language that is a bit slow compared to other lnguages. So can another language created using python be faster than Python ? 2. Does this tutorial series include everything in BASIC Language ? 3. What are the basics of the Programmig Language ? Many thanks to the tutorials you created ^)

[@CodePulse]

1. A language created in Python can be faster if (a) it is compiled to machine code, or (b) it is compiled to byte-code to be interpreted by some other application such as JVM (Java virtual machine), or WASM (WebAssembly to allow it to run on the web). 2. No, as the series progresses the language differs more from the basic language. 3. I'm not sure what you are asking here. Your welcome :)

[@kanuos]

Thank you so much for making this. Subscribed!

[@hartsaxena371]

This is... actually really good.

[@ihorandreienko9648]

you're amazing!! Great tutorial

[@danielmunch2219]

Nice video. But please just don't copy the same thing HowCode does...

[@CodePulse]

Thanks. My aim for this series is to improve on HowCode's tutorials, as his original series didn't use proper techniques, and he rushed his newer series in my opinion. I also plan on making the series much longer and add a lot more features that are missing from HowCode's tutorials (for example functions with arguments).

[@danielmunch2219]

Okay. That's fine. I thought you did it because you didn't know what to do. But cool.

[@CodePulse]

@Najmul hossain As in the minimax algorithm in game theory? It's not really the type of thing that I had planned for the channel, and there seems to be plenty of explanations out there already. But if enough people wanted it, I would definitely consider it.

[@beautifulmind684]

The author @you at the final episode! @@

[@jonathanvanderwal7265]

what is the commmand

[@ajithkumarnair7542]

Please help me, it is showing that module basic has no attribute run, if i try to use 'from basic import run' ,it shows cannot import run from basic

[@miniappletheapple]

a year ago i wanted to create a programming language but i don't even know what is class in python,but now i can code in rust,understanding OOP and little bit of data structure and algorithm

[@CodePulse]

That is amazing! Well done, and good luck

[@miniappletheapple]

Thanks you

[@learnforbest-4500]

thanks for your tutorial @CodePulse sir

[@skilless935]

"create your VERY OWN programming language in Python" *"I used the programming language to program the programming language"*

[@aditmagotra6914]

This does... put a smile on my face

[@megamer2340]

Actually since python is based in c... I used the programming language to program the programming language to program the programming language

[@mizu_yt]

that's actually how all programming languages are developed

[@aditmagotra6914]

@@mizu_yt Its an avengers refrence

[@mizu_yt]

@@aditmagotra6914 i know that

[@RadiantCorium]

when i try to use floats, it says "Illegal Character: '.'"

[@CodePulse]

You probably have a mistake in your code

[@hotdog9259]

Did you get it fixed? I have the same issue.

[@minyakonga8897]

A lexer is responsable turn source code into tokens, these tokens are identifiers, keywords, operators, delimiters and literals.

[@TuckerTheMandolinist]

I am very exited to start creating my own programming language! I bet this will be very helpfull!

[@CodePulse]

I hope so, good luck

[@Dustymarker9489]

When I put a space in between my numbers it errors how do I fix that, I looked back through the video and couldnt find anything I did wrong so please help.

[@Dustymarker9489]

Never midn I fixed it by adding it to the character list under Lexer and doesnt seem to have broken anything and works fine

[@roku1892]

Thanks sir! I can make my own programming language

[@just_kidding_kid]

Much much thanks for your videos! It helped me a lot while other tutorials and series did not

[@DragoniteBrian]

random quesion:what does self mean and why did you use it

[@shahzaibhassan2777]

For those worrying their language might be slow, Mojo is coming, No need to worry!

[@karanvenkatesh8004]

wait i have a doubt how do you index the line ??? like the how do you make python show the line in which your typing!!!

[@CodePulse]

Are you talking about the line number @ 15:28? If so, this isn't actually getting the line number in the terminal, it's always line 1.

[@peppidesu]

3:34 line 35 should be self.text[self.pos], not self.text[pos]

[@CodePulse]

Thanks for pointing that out!

[@noam190]

doesnt work without spaces EDIT: fixed it by removing self.advance in if self.current_char in DIGITS

[@freebiman8268]

Thank you I enjoyed this I always wanted to do something like this

[@wolfieboy09]

I know this video is 4 years old, I will just say that you don't have to do "'" + char + "'" instead, you can do f"'{char}'"

[@mohammad4388]

I don't get the part of where it states whether the no. is a integer or not

[@CodePulse]

If the number of dots/decimal points in the number is 0, then it can be an integer, otherwise it needs to be a float.

[@HyperBassYT]

I like how you need a programming language to make a programming language

[@yktv_edotty]

1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00 SKIP TO A PART OF THE VIDEO!

[@imyashkale]

Thank you. please upload more content

[@BukhariTechnologies]

Can I do the same in C#? Guide me? Thank you.

[@CodePulse]

Of course you can, everything should be quite similar

[@serverapiyoutube5490]

Please more

[@CodePulse]

I'm half way through making the next video, so I hope to have it out by Monday.

[@gign_9891]

if i knew how to make a programming language, I would call it OceanCode and code for making text with another font like hello world is anyways my favorite font is indeed consolas

[@redshed3591]

0:00 Hello everyone... this is YOUR daily dose of internet

[@GeneralN0va]

Imagine making a programming language built on a programming language, that's also built on a programming language

[@helpme2999]

That's Julia for you.

[@universenerdd]

i can just imagine teachers using this language in math class, okay students, find the answer to 435*1654, i can't figure it out, USE YOU'RE BASIC TO FIGURE IT OUT

[@it1shka]

seems difficult but i want to get into it ;) for the first time watching i just rewrite your code. When you began to improve Error class my brains just broke. I think i will watch this vid till I understand it enough good. At all, good tutorials, one of the best ive found. Unfortunately, in Russian programming segment there is no tutors how to create your own compiler/interpreter. But it seems like i found an amazing one in English sorry for broken English

[@CodePulse]

Thank's for your comment! Your English is not bad. I think I overcomplicated this series, and I intend to simply things in my future tutorials :) Good luck!

[@sa-hq8jk]

2 years ago exactly, today

[@r50142]

Hey i wanna make another programming language named Globe, how do i add custom built-in functions again?

[@CodePulse]

kzbin.info/www/bejne/Z5zCf2OlirSjoLs

[@D0T__]

Wow make a promgamming language!

[@abduksam]

Oh my godddd i will make my own programming language i will make exe of the files and done my friends will use my programming language boom its name would be py# inspired from python and c# boom well and i will be upgrading it obviously like adding classes and objects in the next 5 years coz i have just completed python 4 months ago and my friends would be like😲😲😲😲 you are 11 and you made a programming language really

[@developerethan4593]

Prepare for the OAK programming language people!

[@spencergates1786]

i get the error 'import nide' (nide is my language) at the start of the shell file.... no idea why. I even checked the github and it all was the same, im using python 3.9 can anyone help please! edit: i also tried renaming everything to what he used and same result

[@slimestaff1]

Are they in the same file folder? Yeah, I know kinda late

[@CaptainAwesomeVlog]

I had to write a C to MIPS32 compiler in university. TBH this looks much nicer.

[@jd-dev]

Super useful !! Thanks you !

[@Bl0xxy]

I personally would've used enums for the token system, I'll just try to follow along with those, hope it isn't too much of a struggle (I'm using C++, and yes I know python doesn't have them builtin but there is a builtin lib for that)

[@tellurian7999]

Basic Python C Assembly Machine Code

[@udaynj]

This is amazing!

[@ArachnidAbby]

You could use a few dictionaries for some of these