Programming with LLMs: Rant Part 2
10:41
12 сағат бұрын
LLM agents can beat human researchers
11:22
Automated design of agentic systems
9:28
Programming with LLMs: A Rambling Rant
13:30
Fine-tuning LLMs to be tutors
10:27
LLM Agents beat Human Debaters
10:20
Laypeople cannot prompt LLMs
8:55
Studying GSM8K Leaderboard
8:17
3 ай бұрын
AutoGen: Programming LLM Agents
9:17
Fine-tuning or RAG?
8:05
4 ай бұрын
Fixing RAG with GraphRAG
15:04
4 ай бұрын
Co-intelligence: book review
13:06
$10k for LLM reasoning
6:24
5 ай бұрын
LLM benchmarks
11:02
6 ай бұрын
LLMs eat entry-level SWEs
9:06
6 ай бұрын
LLMs can debug with prints
8:54
6 ай бұрын
Determinism ⇒ Fast LLMs (Groq)
10:07
GPT-4 passed the Turing Test!
6:58
Пікірлер
@mikopiko
@mikopiko 23 минут бұрын
Doesn't AMD-135M count as open source llm?
@fluffyunicorn7155
@fluffyunicorn7155 25 минут бұрын
While I agree with your general point, Stallman has the following view on open source: “The terms “free software” and “open source” stand for almost the same range of programs. However, they say deeply different things about those programs, based on different values. The free software movement campaigns for freedom for the users of computing; it is a movement for freedom and justice. By contrast, the open source idea values mainly practical advantage and does not campaign for principles. This is why we do not agree with open source, and do not use that term.”
@archiliusfowl3701
@archiliusfowl3701 39 минут бұрын
Thank you! I am tired of explaining this to people but somehow the terms have gotten mixed up with free software, open-source software, open code, open model weights, etc. etc. The list keeps going on and on.
@kevinkkirimii
@kevinkkirimii 2 сағат бұрын
Well said.
@CaptainSlowbeard
@CaptainSlowbeard 2 сағат бұрын
This might seem like a throwaway "rant" to some people, but what you've said hits at a fundamentally important issue. We're well in the midst of a co-opting crisis. For all of his personal flaws, Stallman has always had a very clear understanding of what freedom of information means, and the implications of a world in which vested corporate interests get to decide what can be shared freely, and what cannot. I think people have become far too comfortable with the assumption that the internet provides: that there have always and will always be ways to acquire things for free. Like democracy, the unrestricted sharing of ideas is incredibly fragile and only exists due to the concerted efforts of a lot of highly skilled, dedicated, and largely unsung individuals who work tireless just because they think it is the right thing to do. Open source has a very specific, ideological, and profound meaning. If we're not careful and take steps to reject the co-opting of it by powerful business entities, we will in short order end up in a position where the meaning is diluted in the same way that "owning" a book, an album, a game has slowly become "purchasing a license to access" solely on the terms of corporate machines who do not care for, nor respect, ideals such as "information wants to be free". Words and concepts mean something. All of human progress depends on this axiom. We're at a technological pivot point the likes of which our species has never seen. If we fail to take an ideological stand now, the outlook is incredibly bleak.
@AC-go1tp
@AC-go1tp 4 сағат бұрын
Thanks!
@NicolasEmbleton
@NicolasEmbleton 4 сағат бұрын
Well said. Glad this is talked about and I wish more would.
@sasha297603ha
@sasha297603ha 15 сағат бұрын
What a piece of art (the video)! When I started programming I already had auto-completion, because it boosts your performance dramatically. I'm the generation of "auto-completion" programmers and I bet there will be generation of "llm prompters". And llms feels like cheating to me, the same as VSCode felt cheating to the authors of "no vscode" article. So, I totally support your opinion. Nice comparisons, support articles and the art about lines. Lovely video!
@sasha297603ha
@sasha297603ha 2 күн бұрын
Interesting paper, thanks for covering!
@drxyd
@drxyd 2 күн бұрын
I still write most of my code from scratch, but I spend far less time refactoring and debugging. AI is best used for discussing requirements and spotting edge cases. The true benefit isn't in code generation but in gaining mental clarity-typing out the code was never the hard part. You can still get all the productivity gains without having your skills atrophy but sharpen instead.
@obszczymucha1337
@obszczymucha1337 3 күн бұрын
I agree with the notion that we should from time to time turn off the autopilot and see if we can still code. I personally see that using LLMs made me much more worse as a coder. However, I've also managed to build a lot more than I used to. So it's really a trade off. Perhaps we're witnessing a redefinition of what a software engineer is. We're coding less and less and it seems it's not stopping. We're reviewing more and more, which I personally don't like at all.
@VivekHaldar
@VivekHaldar 2 күн бұрын
> I personally see that using LLMs made me much more worse as a coder. However, I've also managed to build a lot more than I used to. Can you elaborate? Worse in that you aren't doing the coding and you feel like your skills are atrophying?
@drxyd
@drxyd 2 күн бұрын
Quantity vs quality
@khanra17
@khanra17 4 күн бұрын
My friend coming from US after a 5day trip ! BSDK accent maar raha hai 😂
@MrNewAmerican
@MrNewAmerican 5 күн бұрын
Deeply insightful
@samialghamdi9340
@samialghamdi9340 5 күн бұрын
Interesting
@doodlebug1820
@doodlebug1820 13 күн бұрын
Its amazing but…. In practical use the major problem is getting access to the papers in the first place. Elsevier and other Academic publishers will never allow public AI bot to be trained on its corpus without massive payments if at all.
@VivekHaldar
@VivekHaldar 12 күн бұрын
Paywalled academia's days are numbered...
@misterguts
@misterguts 13 күн бұрын
Hold onto your papers, fellow scholars!
@VivekHaldar
@VivekHaldar 13 күн бұрын
What a time to be alive!
@misterguts
@misterguts 13 күн бұрын
@@VivekHaldar It is indeed a great time to be alive! I am using o1-preview to help write Python scripts for signal analysis. I am watching your videos to get a clue on how to include technical documents for over-the-air digital broadcasts such as HD Radio, Digital Radio Mondiale, and ATSC TV into an LLM context. I want to increase my understanding of those protocols and isolate various techniques used.
@ChrisAthanas
@ChrisAthanas 17 күн бұрын
Nice breakdown
@ChrisAthanas
@ChrisAthanas 17 күн бұрын
Nice breakdown
@VivekHaldar
@VivekHaldar 21 күн бұрын
The first author also discussed this paper in the TWIML podcast a few days ago: kzbin.info/www/bejne/eWaoqo13jtGKbac
@andrew.derevo
@andrew.derevo 22 күн бұрын
Amazing stuff here! thanks a lot for sharing and searching this gems❤
@christopherd.winnan8701
@christopherd.winnan8701 22 күн бұрын
How does this system compare to the LLM that used the similar learning technique to max out on new Minecraft tools and skills?
@krunkle5136
@krunkle5136 22 күн бұрын
Progress isn't forever. Yes tech has been becoming more abstract... to some degree. There's also movements focused on going back on some abstraction. Also there's no optional variety; every aspect of a product and its quality creates a specific experience and connection with the audience. Anything artisan will generally be better because there's more intentionality. AI lacks intentionality, and as people are no longer able to tell what is AI and what isn't, no one will know what was intentional and they'll become cynical.
@gauravtejpal8901
@gauravtejpal8901 26 күн бұрын
If I don't know what I want, what's the llm going to do for me?
@gauravtejpal8901
@gauravtejpal8901 26 күн бұрын
You got to the heart of it. This whole, I'm-an-artist thing, is nothing more than an ego-trip. Art requires tools. Pen and paper are tools. The computer is a tool. So also are llms
@atiedebee1020
@atiedebee1020 25 күн бұрын
So is generative AI a tool????
@EMB3D
@EMB3D 24 күн бұрын
@@atiedebee1020 yes, it is, and good tool
@gauravtejpal8901
@gauravtejpal8901 24 күн бұрын
@@atiedebee1020 Not the most accessible one (in its current avatar) but yes
@arnavprakash7991
@arnavprakash7991 26 күн бұрын
LLMs have been a godsend for getting me started with languages I am not familiar with They are also really really good for framework migrations and simple but powerful scripts But for complex applications, there is still a ways to go. Wanna see what the next generation is able to do.
@pik910
@pik910 26 күн бұрын
pro tip: giving the documentation to llms improves performance, maybe even some idiomatic code. Often what you want is for it to forage through the documentation, which it is pretty good at, probably generally on a superhuman performance level. Asking it for ideas for improvements or if it sees any bugs can be nice, also, I like to discuss ideas with llms if I am unsure instead of just writing them down. It gives a baseline level of quality, it usually finds when you are doing something obviously wrong and is good at the creativity part of problem solving or just knowing lots of approaches.
@arnavprakash7991
@arnavprakash7991 26 күн бұрын
@@pik910 with something like perplexity it can literally search up the documentation. They need to start attaching more tools to LLMs. A next generation LLM with access to a browser, ide, graphing calculator, etc will be quite helpful.
@MrDgf97
@MrDgf97 26 күн бұрын
I think it goes beyond techno-primitivism. IDEs and LSPs purpose have always been deterministic, their output will always be what you expect. It doesn't "make" non-deterministic decisions for you. LLMs remove a large part of the 'thinking' step out of the equation, but it doesn't necessarily 'think' for you, since that's not how LLMs work. Problem is, it encourages people to stop thinking and "let the magic black box handle it". The code it produces, how it understand the context and the systematic interactions you want to happen, are frequently wrong, but it's presented as a solution. I'm talking about 80% from what I've tried; and trust me, I have. If you've ever worked on any repository that's expected to be maintained longer than 2 days, and tried to use LLMs to "boot productivity", you'll know what I mean. My reasons for being against LLMs are beyond a moral imperative. From a pragmatic standpoint, they're affecting my work. I've had to review so many PRs of obviously LLM-generated code that doesn't even work properly. After talking to the corresponding owners of the PRs they assure me they heavily modified the resulting code, which is even worse to think about since what's supposed to be a time saver ends up wasting everybody's time.
@pik910
@pik910 26 күн бұрын
yeah, I don't care about originality. But you can't put code you don't understand in a complex system without causing problems that waste time in the long run, llms can produce bad designs that look plausible. Throwaway scripts, how do I do X and code review is where llms can provide value. But when people have the option to be lazy, there will be laziness.
@MrDgf97
@MrDgf97 26 күн бұрын
@@pik910 Not sure where you're grabbing the originality point from, since personally I don't care for it either. It's why I think Go is such a good language; each different type of task is usually written in a very specific manner, which is good in my book. I completely agree on throwaway scripts. Recently, a coworker unfamiliar with Python needed to cobble something together in less than an hour, which he managed to do considering with the help of an LLM. Keep in mind he's an experienced developer, and Python reads like pseudo-code, so it's not like it could've been achieved in the same time by someone less experienced. This script will never be used under any other circumstance nor be maintained, so it's the perfect use case. Code reviews, however, I disagree with, since it's a slippery slope. I've lost count of how many times non-senior devs just approve a review based on a senior's approval without doing their due-diligence. People will default to trusting an LLM blindly, which is a problem considering its tendency to hallucinate.
@VivekHaldar
@VivekHaldar 24 күн бұрын
In your experience, has LLM-generated code been worse or better than that of a fresh grad first writing industrial code? What domain and language are you in?
@MrDgf97
@MrDgf97 24 күн бұрын
@@VivekHaldar My domain is currently broad. Our main product for stakeholders is written in hundreds of thousands of lines of embedded C code, but most of our time is spent on a myriad of tools for our own team and other development teams. These can be web-based tools using a common tech stack, C++ applications for running simulated environments, or C#/Rust command line utilities. Regarding how LLMs fare against fresh grads, it's not a simple thing to compare. Fresh grads with little coding experience usually commit smaller chunks of code, with their PRs becoming larger and larger as they start to familiarize themselves with our toolset and workflows. The amount of code an LLM can produce is astounding, but usually really bad and worse the bigger it gets. Since fresh grads won't make big PRs, it's hard to compare on that department. However, in very small chunks (a couple of lines for an ultra-specific purpose), yes, an LLM might produce better code than a literal fresh grad, but that doesn't hold up after a couple of months. You can rely on a person being self-sufficient and learning over time, which can't be said about LLMs.
@barrettvelker198
@barrettvelker198 26 күн бұрын
AI Rorschach test: Could I/you make art using only llms?
@VivekHaldar
@VivekHaldar 24 күн бұрын
There are artists already making art with image generation. They have an artistic vision, they use the model to realize it.
@tonyosime9380
@tonyosime9380 Ай бұрын
Thank you for sharing your wisdom, it has inspired me.
@andrew.derevo
@andrew.derevo Ай бұрын
Thank a lot! this is definitely a good paper to read on weekend 🙌
@dawid_dahl
@dawid_dahl Ай бұрын
Thanks so much, great content.
@andrew.derevo
@andrew.derevo Ай бұрын
Surprised, save me few weeks of testing 🙌❤️
@andrew.derevo
@andrew.derevo Ай бұрын
🙌 great stuff
@somanshukumar1344
@somanshukumar1344 Ай бұрын
Always wanted to see this type of content
@rtos
@rtos 2 ай бұрын
Unfortunately even the so called power users of LLM with their own KZbin channels, always seem to have a small set of stock prompts, which get repeated with every new review. If LLMs were trained on these specific questions then they're going start appearing super intelligent! Things like 'why is the sky blue', or write a 'snake game in python' are hardly a test of machine intelligence, as all that is needed is to be trained in accurate code or factual data.
@matty-oz6yd
@matty-oz6yd 2 ай бұрын
I really value what you do my dude <3
@RAHUDAS
@RAHUDAS 2 ай бұрын
Bot designer ltd crashed, I not able to access
@ashwinnair5803
@ashwinnair5803 2 ай бұрын
Why not just use RAPTOR instead?
@yiwensin5913
@yiwensin5913 2 ай бұрын
Excellent! I didn't know you before and I just stumbled upon your video while searching for material on prompting LLMs (for a local LLM project). You now have a new sub :)
@VivekHaldar
@VivekHaldar 2 ай бұрын
Welcome aboard!
@user-wr4yl7tx3w
@user-wr4yl7tx3w 2 ай бұрын
can you discuss DSPy and give your opinion on it given how it is related to prompting
@arthurdhonneur276
@arthurdhonneur276 2 ай бұрын
Nice video thank you very much !
@Starhopp3r
@Starhopp3r 2 ай бұрын
Excellent review! Thank you. I really enjoyed this book and have been recommending it to people in order to help them set expectations about “AI” without excessive optimism or pessimism. Currently reading Deep Utopia; hope to see your review soon!
@HarunZafer-f3u
@HarunZafer-f3u 2 ай бұрын
I think the main problem is how they traverse the graph by asking LLM at each step. I'm not sure if this is feasible in production.
@sasha297603ha
@sasha297603ha 2 ай бұрын
Great paper, thanks for covering!
@user-wr4yl7tx3w
@user-wr4yl7tx3w 2 ай бұрын
Excellent content. Well explained.
@PreetiGuptaAvril
@PreetiGuptaAvril 2 ай бұрын
Sir, Do you explore the vision domain as well or do you any such youtuber whom i can follow for paper understanding.
@themax2go
@themax2go 2 ай бұрын
very well "ragged"... both on the local domain (details) and global domain (overview of pros-cons) 😉😎
@wayneqwele8847
@wayneqwele8847 2 ай бұрын
Thank you for the video, that was a great paper to go through. I find RAG research techniques have so much insight to how we can develop and identify our own cognitive impediments to our own judgement. The Comprehensiveness, Diversity of perspective, Empowerment and Directness is such a good mental model to use in our own human judgement.
@fintech1378
@fintech1378 2 ай бұрын
super excellent video