I made a SEARCH ENGINE from scratch!

[@JeffreyZang]

big chungus

[@cardboardbox_tech]

among us

[@sasodoma]

The first thing I thought when you said href will always be a valid link was "not really". And then you showed the descent into madness, love it.

[@mathman0569]

I was about to say this lol

[@cinderwolf32]

Client side hash string routing: "I'd like to introduce myself"

[@reed6514]

I suggest scraping your local news websites and making a search engine just for local news. It could actually be useful for your community.

[@HootMoot]

Good stuff and nicely scripted and edited. Once you get a mic and some traction, you'll be on your way to 100k subs. Everything but audio is extremely high quality and well done!

[@smb1397]

"my hard drive was getting too full" shows 112 gb used. walled garden problems

[@ujjwaldimri2392]

amazing video, it is well made and very informative. hard to believe you only have a 100 subs

[@vitaliykishchenko4365]

This is awesome work! Have no idea how you're doing all this at this age, you have a bright future. Research and editing all done very well. I love the TF-IDF explanation, super concise.

[@deepbrar1]

This is awesome. The video is really informative and you explained everything so perfectly. Great content, got a lot to learn from it.

[@fakejimhalpert]

GREAT video this is so cool!! keep going

[@kevinnielsendev]

Nice project! This is really impressive :D

[@casperdong]

the new GOOGLE I cannot wait

[@varram3488]

I've been in the webscriping space for years. The robots.txt page is a suggestion. It is completely legal to ignore the robots.txt file. Big tech companies want you to think it's illegal even though its not. As long as you don't commercialize human made content you will be fine :D

[@hexxt]

naughty boy

[@fujinshu]

Right now it's illegal, but I wouldn't be surprised if Congress and Trump rule that ignoring the robots.txt file is illegal, especially with some light bribery and a compromised SCOTUS.

[@cinderwolf32]

Have you ever been blacklisted for requesting routes that are denied by the file? Especially if your user agent is noticeable

[@reed6514]

Webscraping can be illegal depending what you do with the content. The u.s. copyright office has a Fair Use Index online that summarizes findings. Search should be fine in most cases, but it's good to be informed, especially if your use case is remotely iffy. Even better to talk to a lawyer.

[@varram3488]

​@@reed6514 yep exactly, and the use case in the video falls under fair use. Also, it's really funny how the entire AI industry is based around a grey area (The law is pretty outdated and vague); which we should see resolved really soon through lawsuits going on right now lmfao.

[@Gaarlicc]

Such a cool video , keep it up

[@TheRetroEngine]

Dude I'm new to your site and when the FBI turned up, I spilled my tea.

[@zinck_dome7072]

Cool project man

[@cake0539]

tfidf sounds like a DOOM cheat code

[@LegendBegins]

This was great! Nice work!

[@Mikko-Maggie-More]

fun fact: google purposefully removes results so that you'll use gemini instead

[@ClarkeMacbeth]

Just in time for 2121!

[@casperdong]

daniel zhang is the next joma tech!

[@mwguy]

Good stuff. You can improve crawler algorithm by spliting it to small worker nodes that coordinates with kafka/rabbitmq to parallelize page downloading. Also just thow away prisma and use raw sql to squize all performanice from database.

[@C4CH3S]

Or yk, don't use toy languages like JS in the backend for performance heavy tasks...

[@reed6514]

It is worth considering load on the aites you're crawling. Don't want to ddos on accident.

[@joshchen7993]

Such a informative video! How did you speed up the querying process? I’m also trying to create a search engine and calculating the tf idf of every page is taking a long time

[@MeowVR]

Very cool video

[@casperdong]

I am ur only female viewer

[@jir_UwU]

nuh uh me too

[@casperdong]

@@jir_UwU us girls gotta stick together

[@deleted_handle]

girls aren't real. stop trolling

[@LEMON_2U]

Nuh uh me too

[@Imtitled]

R30 There are no girls on the Internet

[@BiuerBoris]

Wonderful video and project! Would be interesting to have it spawn the crawler from somewhere else than Wikipedia.

[@petermarshall1634]

Amazing video how does your channel only have 200 subs

[@felixranesberger3846]

Awesome video!

[@horntoad1616]

Horntoad here

[@staniekkkkkkkkkkkkkkkkkkkkkkkk]

good shif vru

[@Matusevichfilms]

Kagi is pretty good

[@gljames24]

What is that random sound in the background of your voice? Do you have your hand on your mic? Put a gain cutoff or something.

[@garf510]

I love giggle 😀

[@mathman0569]

Should've used the URL API to check for valid URLs

[@Christian-ry3ol]

I think I just found my next project to work on. This inspired me. So fucking cool.

[@CerebrumReality]

Nice Video :0

[@casperdong]

I love you.

[@exp5261]

@@casperdong thanks

[@ShaikhRehanShakil]

everythiing is just wikipedia :((

[@ironislife9857]

Why not use a faster language that would allow you to search for things fastee

[@cinderwolf32]

The language is probably not relevant to the time it takes. It'd likely be unnoticeable whether you use Python or C++ for this (assuming you don't write terrible code). A network call / database write is multiple orders of magnitude slower than whatever the code is doing

[@reed6514]

​@@cinderwolf32caching to text files and then writing bulk queries could speed up the db stuff. The language speed hardly matters, but a couple milliseconds per page adds up when you're doing many thousands of pages.

[@solmateusbraga]

I am ur only femboy viewer

[@_jb_]

I enjoyed the animations and the effort. But your coding abilities need to improve. Keep on

[@rixanito]

The video is amazing, but i would still suggest you put more visual effects like zooming ins and transitions(especially zoom ins)

[@rixanito]

@bogxd would be a great source of inspiration

[@therealpersonion]

is this manim??

[@danielcsthings]

It's motion canvas motioncanvas.io

[@Zhane4994]

YOU LIAR, THERE IS MORE ALTERNATIVES 😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱😱

[@sandwich-plays]

skibidi

[@justwingingit8110]

20 views in 14 hours? you fell ofd

[@danielcsthings]

it's joever

[@Jetway-Yefan]

GIGGLE