Yes good point! The source data needs to attend to itself. So we need copies of the input/output data to compare. That's why the QKV variables are the same. Duplicate data for self-attention so we can multiply the vector with itself to create the attention between each token. Example of translating between English and German: 1- Encoder Self attention Q = K = V = Our source sentence(English) 2- Decoder Self attention Q = K = V = Our target sentence(German) 3- Decoder-Encoder attention Q = Our target sentence(German) K = V = Our source sentence(English)

Again neither the query, key and value vectors are identical nor their weight matrices Q, K and V. The 3 vectors do stem from the same hidden vector h pertinent to the same word, i.e. q = Q.h , k = Kh, v = V.h , but that is it, they are not identical. Is this an experiment/joke or what?

Good point! Rocket looks clean. Like a simple and elegant web app server framework. Simplicity is key. And it looks like Rocket does a great job 😄

@StephenBlum yes it does and easier to pick up. My opinion though.

`Manaco 15` is the font 😊

Good idea! Here is the checklist direct from the video: docs.google.com/document/d/1I_9e0kESyWCcoqiLfdCfjctvul7RaO0DLpLx-ynnlY0/edit?usp=sharing

Yes absolutely. Here is the checklist from the video: docs.google.com/document/d/1I_9e0kESyWCcoqiLfdCfjctvul7RaO0DLpLx-ynnlY0/edit?usp=sharing

@@StephenBlum Thank you!!!!

Yes you are right. ReactJS is very slow. Preact can help speed it up. And if you use the preact-compat library, most components should work github.com/preactjs/preact/blob/main/compat/package.json

Good point! Q dot t(K) is the token by token matrix. Each token is a vector, commonly 512 floats per token. Both Q and K are the same array of tokens

Kali Linux comes installed ( or offers easy access ) to all pentest tools. For specific pentesting, you just need to create a list of your favorite pentest tasks. A great way to get a list of ideal tools available, most of the LLMs can provide a list that matches your specific tasks 🎉

@@StephenBlum thanks for ur advise

Great suggestion. Illustrator can export directly to web-ready SVG. And you can use that SVG to make fantastic textures 🎉 just save/export the file to SVG. If there is an optimization option for web export, make sure to select that option so the file is smaller

good question! for "fast" with a million polygons, you have to do it progressively. If you are looking to draw all polygons on the screen at the same time, then it might be a non-starter. If you are only showing some of the polygons on the screen, then using most 3D render frameworks will automatically take care of this for you by only drawing the visible polygons in the viewport. 3D frameworks automatically cull. Here are some types of culling: Back-face culling: A technique that avoids drawing polygons that are facing away from the camera. A polygon's vertices have a clockwise winding if you're looking at the back face, and a counter-clockwise winding if you're looking at the front face. View-frustum culling: A visibility culling technique. Occlusion culling: A visibility culling technique.

@ so it meant to show it effectively - don’t show what is not visible at moment? I had in idea for example to have 20 layers for 20 zoom levels and cluster polygons when zoom out , and when zoom in hide what’s not visible in the area. Something like that

@ by the way, happy New Year! Very nice channel

@@sergannn thank you! happy New Year! 🎉

@@sergannn that's a great idea! that would dramatically improve performance. Multiple LOD level-of-detail layers/zooms so that when the camera is zoomed out, a simple model with fewer polygons is drawn instead of the full 1m polygon model

you're welcome! 👍😄

that's actually a really good point! as of today, most of the software we need has already been built and it works mostly well. we can make improvements and fixes. other then that, it's already been written

great to hear you say the video was helpful, your blog is really well written 🎉😄 your blog helped me out a lot thank you!

​@@StephenBlumThank you

@@siddharthmanumusic thank you! 😄 Yes absolutely you are right. C++ is relevant and heavily used in advanced AI frameworks

No problem! glad you found the video 😀

Any time 😊 thank you!

agreed, the Gemma models and Google AI models are heavily censored for safety. This can become a problem even for seemingly simple tasks, the AI will guard it's answers

`Monaco 15` is the font family. running on a Mac terminal emulator window

@@StephenBlum thank you

you're right, that's an important perspective on interest rates. with cheap money, large green (new) projects start with lots of software development. Without cheap money, companies stop making new projects

Smart question! you figured it out. Yes processing videos on servers is smart that you figured this out. AWS charges for each second the EC2 instance is turned on. The time is about 2-3 minutes per video ( about $0.50 per video ~ )

Cool Thx!

I appreciate that! I put a lot of effort into creating this video 🎉

@@StephenBlum you have really good content. If you use canva or Microsoft designer for video cover. Makes them more clickable. And call to action in your videos to subscribe. Love your content. Should be hitting 50k subs with a few tweaks. Especially on topics like pandas, ML and LLMs

you have good video tips! this is helpful thank you! 😊

one option to get 100% up and running with your at-home GPU and remote access is to use PubNub.com API to send/receive the text data. it does take some setup and coding. you can create a chat app interface that delivers the prompt to your GPU running the model, and then publishes the response back. user app -> PubNub Publish API user prompt -> your server subscribes to the events and submits the prompt to the GPU model -> your server publishes back the answer to -> user app is subscribed to the response channel and receives the answer. Copy and paste the description above into an AI tool to build the app for you 🚀

you make a strong point. the AI is just a tool. let's imagine for a second that there is an AI tool that creates perfect code. You still need a human software engineer to direct it because there's scenarios like you describe where the software needs to be maintained. The tool needs guidance

you are right, the title is wrong because it's mostly a discussions video and a mini getting-started intro with a code walkthrough. that doesn't really count as a tutorial in this case, more like it's an overview

the managers think that it saves more time than it actually does. there is a lot of hype. the GPT tools do help a bit. it's great that it saves some time we using AI tools. It allows us to focus on the more creative parts of our work

nice! yes it's pretty great to see that the government has job listings like this. you can bid on the listing and make a great outcome

Good catch you found a mistake I made 😅

@ I thought I was going crazy for a second lol

Don't get discouraged, these are all marketing stunts to appeal to CEO's and VC's. Coding and development will still be around for a long time. I'm a systems engineer and full stack software developer who uses SOTA LLM models everyday and I can tell you for a fact that LLM's are just a tool.

Ilya, it's never too late to start programming!😊 python is a great staring point. It's also a really popular choice these days as it's the primary language used for AI programming 🎉 get started today!

agreed with @nicohornswag LLMs are just a tool. LLMs are a large paintbrush. rather than typing individual keystrokes, you can use an LLM to paint larger segments of code for you

Same here , infact I am a bit older 32 and I have also wasted my 20s doing spontaneous job, and finally I found my love for coding. I have been coding since one year now and I am able to build up something. But I realised the love for coding has not developed because I was meant for it, but rather becuase it has become very easy to learn to code and with so many resources available. But since yesterday's development a big fear has developed inside me,

@@Kmps2341af nice! great you have found your love for coding 🎉 it's a lot of fun right? And it can be concerning when we hear about the new GPT o3 model, where it has become the top 200 best software engineer in the world for coding challenge benchmark

haha yes! you are right. It's pretty much exactly "buy our stocks". we are years away from AGI at this point

thank you 😊

Yes! I like that analogy. it makes the AI seem less scary. it's helping us create things by using large brushes, rather than individual keystrokes and mouse clicks

yes exactly. you got it! AI can't yet replace software developers. today the AI can help make a software developer more efficient. usually we have to lookup a lot of docs, read the docs, then construct a solution based on our understanding. the AI can do all of this for us in the IDE 😊

_"potentially if the ai is instructed correctly by someone with programming experience"_ So like one of the things I'm writing rn is a state machine for the character state. It can't do it if I just say generic terms. The kind of terms I'd have to give it to write proper code would be so specific that it's easier to just write the code. Which I do. The biggest thing those AIs can really help is picking names that are already taken. So, I added methods Start and Stop for an Action class. And the Start method is already taken by Unity. So I asked chatgpt to give me alternatives for the Strart and Stop words. It provided a reasonable list for both, I picked Begin and End.

This can not happen with LLMs. Th ey are not rule based

@@KulaGGin seems like fair tokens for this purpose. Sounds like a fun project! good idea to use a "stop-token" for this

thank you! that is great to hear 😊

AI is getting really good right? 😊 the video is really me. to check for AI generated videos, watch the Hands 🤚🖐️ and make sure that the finger count stays correct 😄 this is a common AI flaw

neovim is a good starting editor for learning vim-commands

thank you! yes, AI is moving really fast! 🚀

you make a good point! yes WebAssembly has come a long way! Though it doesn't have a great DOM story yet. It's still great for tasks that may require a bit more compute that would take JS a while to process. Great progress and it's available today! Though adoption is small because it's still a bit of a challenge to get started. If it was as easy as JS and React then we would see more adoption. Lots of opportunity for innovation 🚀

​​@@StephenBlum Yeah, sorry for being so cynical in my earlier message dude. It's a good tutorial, but I think it would be a lot more impressive technology if wasm were object oriented (including proper support for interfaces, which even typescript doesn't have) and supported manipulating the DOM. I found Yew, which supposedly uses structs in rust to fake object orientation (or does Rust have OO these days?), and I guess it uses javascript wrappers and callbacks to implement DOM manipulation. The code looks similar to React, including JSX stuff, except its programmed in Rust. You might want to look into the RSTY tech stack. It's all Rust (if you love that) and you can program both desktop frontend and webfront with it, as well as backend, and I believe the DB is made in rust as well. I found this video about it (not affiliated with the people of this video) kzbin.info/www/bejne/oqaymHiegaplqcksi=A_YQ0RckcB0QA1ZT

haha yeah! 😊😆 puns are great especially when they are causally snuck in the conversation. You found the pun! 🎉

It's pretty exciting because we can train these massive models with great accuracy. Then after training we can reduce the bit depth by 8x or more and still get great results 🙌

Ah yes! Google's Willow processor 🚀 it's pretty amazing how far Quantum processors have come.

Thanks, glad I could help! 😊 It's a powerful technique that has gained wide adoption in the industry. It allows you to run the very large models on many common household computers 🎉

You are right. when it comes to scaling it's a lot easier to use systems that have built-in scaling design models readymade. When you build using server databases, PostgreSQL and MySQL are great choices because of the built-in scaling models. For embedded DBs like in-app on a device, SQLite is your best choice there. For server-side SQLite, you can build a blazingly fast super scale model with extra planning and effort. This is absolutely inaccessible to most developers on Earth. Scaling SQLite as a server database is a significant amount of effort. it comes with amazing rewards and most developers won't want to spend enough time to make it work. This is okay! Because the pre-built options are much better ( MySQL/PostgreSQL ) 🚀🙌😊

good question! this is an article written by dnbt777. learnhowtolearn.org/how-to-build-extremely-quickly/

that's a great quote! it's really true. it's much better to write less code now and get to the end goal quickly. it takes a lot longer to optimize. even though optimization is fun sometimes! 😄

great to hear! I'll find you on LN

Nim is a great language! fast and easy to read. It's very approachable as a language ❤️

Hi Cyber Lanka your advice is excellent. this makes sense. "top x" posts might get demoted by the YT algorithm and are highly subjective. thank you! 😊