Copilot Causes Code Churn? This Study Is Concerning...

[@ayaya-ayaya]

Polish dev here. I rarely felt that raw code output was a bottleneck for me. More frequently I'm thinking about writing code in a way that won't mess up the architecture. Not sure if copilot can help with that.

[@EIP674]

Understanding that is the key to become a polished dev 🙏

[@Happyduderawr]

Same deal here. Chatgpt doesn't yet tell me how to structure complex design patterns nor refactor large code bases. It would be absurdly difficult find training data to make it do that as well.

[@marcelo-ramos]

Exactly my thinking. That's also my answer whenever I hear somebody criticize developers who can't type properly at 120 wpm.

[@kuhluhOG]

yeah, I somehow have the feeling that a lot of devs who go really hard on that have the development strategy of trying to throw enough shit (aka random ideas) at the wall (aka as their requirements of whatever they are trying to do) until something will stick (aka actually works like they want it to) sure, I can work like that if I need to and be on about their speed (not exactly ofc since I do it way less), but will I be able to understand the outcome of it a few days later? probably not

[@JPilsonSumbo]

Exactly

[@Gnsbehehehh]

Lmao advent of code literally said please do not use copilot

[@matiasask3774]

First think then 🤖

[@Controllerhead]

yeah wtf dude. who cheats in AoC?! what a clown 😡

[@NeetCode]

It reminds me of that Human centipede movie except with AI generated code. Sh*t code goes in, even worse code comes out, and it just keeps getting worse. Sorry, that's a cursed comment.

[@RaZziaN1]

It seems to be that problem with AI is really similiar to problem with SEO. Now internet is flooded with content processed and generated by bots for bots, using previous bot generated content. That's why finding useful information is becoming impossible, everything is sorted for the seo purposes, not for usability by humans. Only for seo scores..

[@3_smh_3]

bruh! you went all in.

[@proosee]

lol, best metaphor ever

[@osamaaj]

My fiance made me watch it the other weekend. This comment made me relive the experience

[@williamokano]

Copilot were previously trained with human code, now it’s being trained with copilot code (user code blindly using copilot) hence getting worse.

[@javierflores09]

where did you get this information from? Or are you making the logical assumption since 40% of code is just copilot's? If so I don't think that'd be the case since any sane developer will fix whatever mess copilot may make given a crappy solution, or ask for better solutions

[@williamokano]

@@javierflores09I obviously came with this info from but a$$ 😂 It’s just sarcasm (with a small hint of probability of being accurate). Don’t take the internet too seriously man

[@rallealyt]

Autophagy of models. One of the biggest problems with current "AI". Also: "halucinations".

[@ITR]

Copilot has always been trained with human code, and most human code sucks, so most copilot code sucks.

[@Xnozea]

That's a big no for any AI developer really. I dont think GitHub is that dumb.

[@Gordin508]

04:20 The "About" page of AoC specifically demands not to use AI until the leaderboards are filled :(

[@Abion47]

I'm aware this will probably be an unpopular thing to put in the comment section of one of his videos, but IMHO Theo should absolutely be removed from the AoC leaderboards for using Copilot to speed up his solutions. The entire point of AoC is to improve your own skills as a programmer, and using an AI-powered code generation goes completely against that intention. Theo admitting to using Copilot in AoC is him admitting that reducing an arbitrary number on an arbitrary leaderboard is more important to him than actually improving as a developer (which I know obviously isn't true, but that's what it says).

[@NihongoWakannai]

"man it feels so good to be winning races now that I'm doping, I've never had a result this high before why don't more people do this?"

[@nicosoftnt]

@@Abion47 Actually I was thinking the same thing, a bit out of character coming from Theo.

[@exe2543]

@@Abion47 The fact that a leaderboard exists means that in addition to improving programming skills, solving problems quickly is also one of the points of AoC. And even with the point of improving programming skills, if AI can be used to supplement speed without loss of learning (for example, to write some quick function that you would easily be able to write on your own but would take some time), is that really such a bad thing? In competitive programming it's very common to define lots of macros to streamline possibly repetitive and time-consuming functionality, is that so different than using AI the way Theo is?

[@exe2543]

@divine_paul > you don't use ai when the competition begs you to not use ai Correct. Theo is in the wrong here with regards to AoC specifically. > and at the end, he just said he doesn't respect the competition either. cause he can make content out of the drama if they removed him. This is effectively equivalent to people who do crazy things on social media for views. While using AI still serves the end goal of improving programming skills, it does so at the expense of the spirit of the contest, ultimately harming other participants and creating a bad ecosystem.

[@TomNook.]

The Enshittification of Code It's happening to everything.

[@TwoForFlinchin1]

Thanks, Tom

[@xXxRaVeNcRoFt_LuVrr69]

Does anyone remember way back when Facebook implemented AI to post messages and the more the AI's communicated with each other, the more they optimized their communication until it was incomprehensible to human readers?

[@AlecMaly]

Just bought it for a year for side projects. It's actually pretty incredible how bad the suggestions are sometimes: code that doesn't follow type definitions in a typed language, malloc() without taking sizeof() into account, it completly fails to prevent directory traversal and file upload vulnerabilities, doesn't follow your coding style, and just overall cannot understand a codebase. That said, 5% of the time it can be really useful to hash out small functions that, for instance, remove cyclical object properties in a javascript object or throw together a quick html popup for a side project where I really don't care about code quality. For all the times it throws wrenches, it does sometimes offer tremendous value and saves time - the difficulty is recognizing the subtle bugs it introduces, it's too easy to press tab and move on only to find out later the code doesn't account for 5 edge cases.

[@tradfluteman]

That being said, the tech did improve. When it first came out, Copilot was like a drunk CS frat bro suggesting the first thing that came to mind. "That a for-loop, dude!" It was actually slowing me down. Now it's speeding me up a little.

[@Abion47]

The problem with Copilot and every other generative AI code generator is that it was trained on existing source code and you have no way of knowing how well written that code was. Not only that, as usage of Copilot becomes more prevalent, more and more public repos will contain code written by Copilot, which will in turn be used to train Copilot going forward, reinforcing or even exacerbating any bad coding habits that Copilot picked up from the original source.

[@mr.rabbit5642]

​@@Abion47 not only that. Theres also this AI-design problem which I don't quite believe in microsoft to realize being the fucking dumbasses they are. It's the problem the AI safety researcher Rob Miles pointed out a while ago - bigger AI models are desireable since they can handle more complex code and also do it /better/, but since they have to pick up on little caveats in the code they see, they may go like "aah, writing buggy and insecure code are we? Here, let me help" or ".. this is how its *usually* done. Wonderous isn't it?" Im so sad this company can so fucking easily ruin everything just bcoz they have money and ppl to resell shit they stole/bough ;-;

[@fgregerfeaxcwfeffece]

I asked GPT4all for a Butter Chicken recipe. The result did not even contain meat. So it was abundantly clear it just threw together some average list of indian dish sounding ingredients. I ma excited on seeing people use this productively for programming!

[@gaiustacitus4242]

The only thing I've found Copilot to be truly useful for is doing the grunt work of documenting my code. I still have to clean up the comments it generates, but that feature alone saves me enough time to justify the monthly expense.

[@kodekorp2064]

This is why I tell people at my job to stop relying so much on AI, only to treat AI as supplemental to your work flow, and always double check work with AI.

[@carltongannett]

My job is getting us some copilot keys but my plan is to only use it for boilerplate. Gotta understand everything in my code at least at the time of my commit

[@itskittyme]

There is nothing wrong with relying on AI, as long as you know what you are doing. Meaning, if you understand every single line of code the AI writes, then you're fine. Most of the time, this "errorenous" code from the AI is not an AI problem, but a user input problem who simple described an idea wrongfully, and the AI is taking what the user says literally. 90 % of the time the AI writes bad code, it's my own fault since I didn't word my request correctly enough. Telling people at your job to stop relying on AI is the incorrect approach. It's not the AI that's at fault, it's the people, and they can only correct themselves, if they work MORE with AI, not LESS.

[@okie9025]

Wait, some people use AI tools like Copilot to write all of their code? I personally use it as a more advanced intellisense/autocomplete, and the most complex tasks I do with Copilot are generating pure functions with less than 20 lines of code.

[@kodekorp2064]

@@itskittyme You just explained in your reasoning that exactly described my claims of saying “do not rely on AI”. When I say that, I meant it in the literal sense. AI is not the end all be all easy way to get ahead with your work. Use it as supplemental and a “tool” for your job.

[@kodekorp2064]

@@okie9025 I do have some workers in the office that believes AI code is good enough at certain scenarios, and at glances of the provided AI code, sometimes co workers literally Just copy and paste.

[@sprytnychomik]

How did we get from 'signed int considered harmful because of potential UB, so we should not use it' to 'let's just push-and-publish "AI"-generated code which (neither of them) we don't fully understand'?

[@tonyb3123]

"Writing a comment and naming a function such that co-pilot knows what to do from there" Dude, that _definitely_ falls under the cheating category of AI usage for Advent of Code. Its really annoying that top 100s on the leaderboard are getting filled this way

[@NihongoWakannai]

​@LeoMods_ he was defending himself, he was trying to say "no guys it's totally not cheating because I'm not just typing in the question and getting an answer"

[@innocentsmith6091]

@@NihongoWakannai To be fair, he also didn't sound like he'd be too torn up if he got banned from AOC for doing it.

[@NihongoWakannai]

@@innocentsmith6091 his lack of respect for the competition and willingness to make "content" from that isn't a good mark on his character.

[@fgregerfeaxcwfeffece]

@LeoMods_ *copying speed

[@yelmak]

Not allowed to use any AI tools at work. Higher ups are considering it but there's a general fear of IP leaks and our CTO is pretty sceptical of the code quality coming out of it

[@scragar]

I think the code quality is a problem. I've seen copilot generate perfectly fine code, and I've also seen it use string concat to generate queries with no regard to SQL injection fears. My big fear is some day someone will use it and introduce some horrible security flaw to our codebase because AI has no idea that what it's doing is bad, only that it's seen it before which makes it acceptable to repeat. Also seen a lot of it repeating examples from popular books from a few years ago, I'm fairly sure a bunch of people tested the code and used github to host the codebase as they worked through the examples,and now copilot is stealing the books code without crediting the books. I envision lawsuits about that in future.

[@exe2543]

@@user81069 That's pretty interesting, but wouldn't it require tons and tons of poisonous code to be published to actually start impacting these huge LLMs? I've heard about this type of possible attack on image generation before (arxiv.org/pdf/2310.13828.pdf), but didn't know it could be used on copilot.

[@retagainez]

The risks of code churn addressed in the video aside, there's also just the general issue of how AI tools are just boilerplate generators. They take your English-like text and generate the boilerplate (whether what they generate is code or more documentation is irrelevant), that's it. If you know how to do something but don't necessarily understand syntax or semantics of a library as a single layer of abstraction, LLMs are great. But as far as using LLMs for code design, or for creating layers of abstraction, this kind of stuff is simply way above the scope of what LLMs are currently doing.

[@retagainez]

Some companies are using training LLMs to act as RAG(s) to have a intelligent code lookup on their codebase using large language models. But it doesn't necessarily provide any intelligence like I mentioned. It's just now generating boilerplate relevant to your existing codebase. So, in a world where you have to discover what you know (or don't know), LLMs don't provide much in the way of help for the world of software world quite yet. At least not from a general perspective.

[@exe2543]

​@@retagainez sometimes it gets syntax wrong as well though, which can cause debugging nightmares. I find it best for writing code that I otherwise would already have known how to write in the first place, but am too lazy to write.

[@DodoLP]

Isnt in the "rules" of advent of code, that you shoudlnt use AI ???

[@chriss3404]

I have two wolves inside me, the quality focused senior dev, and the deadline focused contract worker. AI has a secret third thing.🐺

[@jsrjsr]

Then You only have 1 wolf and 1 poodle.

[@schtormm]

I think it's a bit of a garbage-in, garbage-out situation, as in Copilot will eventually just eat itself, because people use Copilot and then it becomes a bit worse every time because of getting fed a lot of copilot code

[@lrsperanza]

People will not commit all code copilot generates. People will, of course, commit some of the garbage, but not all. AI only needs a slightly better on average input than it's output, and that the developers are providing by removing some of the incorrect/bad quality code and writing correct/good quality code of their own. Also, there are code quality analysis tools that already exist and that can filter out bad quality code from the training process. Also, the training process can leave out open source projects with low engagement. The AI tools are here to stay, they are improving fast and in order to survive professionally, we gotta adapt out of what they do as they improve

[@patricknelson]

Devil’s advocate: One _could_ make the argument that, at least over time and with best practices, the code selected by developers (and committed and thus used for training) will serve as better training data. Keep in mind that the output of copilot isn’t utilized 100% of the time but rather still must be selected/committed. So, it’s plausible to suggest that while the quality could go down, _those_ actions might at least help boost some of the quality. That said, obviously one of the big issues we’ve got right now is use of bad AI generated code producing subtle bugs. So, while I’m not bullish one way or the other, I do lean a bit toward the _current_ corpus (or at least the current method of gathering it) being potentially flawed in the long run if it cannot be reinforced with verified and tested quality code.

[@danielsan901998]

Corporations only use more training with more datasets if it increase performance, why would they waste computer power and storage training it to make it worse?

[@minhuang8848]

let's be real, any dev who contributes to code churn by means of misusing copilot and friends is probably the weak point. It's not about these models failing when they shouldn't, it's about their users being too incompetent to realize their mistakes. Which, and this isn't actually at all facetious, simply is something like 9 out of 10 professional programmers. Almost any profession is ripe with completely replaceable workers, even outside all the LLM and AGI debate going on these days. Until recently, they just were still quite a bit better at their job than the few alternatives.

[@ivanlaplante]

Your code: 🥚 Your code on Copilot: 🍳

[@BenderdickCumbersnatch]

"Can I use AI to get on the global leaderboard? Please don't use AI / LLMs (like GPT) to automatically solve a day's puzzles until that day's global leaderboards are full. By "automatically", I mean using AI to do most or all of the puzzle solving, like handing the puzzle text directly to an LLM. The leaderboards are for human competitors; if you want to compare the speed of your AI solver with others, please do so elsewhere. (If you want to use AI to help you solve puzzles, I can't really stop you, but I feel like it's harder to get better at programming if you ask an AI to do the programming for you.)"

[@Abion47]

7:25 My counter-counterpoint to this counterpoint is those kinds of high-velocity companies already existed prior to Copilot and their relative amount of code churn was almost certainly already included in the baseline data as well.

[@adrianmisak07]

I work as a solo developer for a small company, so copilot helps me a lot. definitely saves a lot of time, and its very easy to check what it writes (before you accept the recommendation)

[@kirasmith1147]

This made me realize I use copilot very differently than some others. When I hit tab, I preform a proper code review on that new stuff, as if it was another developer saying "hey, these are my changes, are these OK?". These tools are best used to spitball ideas, *not* to do the work for you. Awesome for smart autocomplete of small/simple lines, though.

[@hoppy6437]

Copilot for me is like using auto complete in addition to spell check. It's good when I'm doing something obvious. Like if I'm adding a guard clause for parameter1, it correctly predicts I want one for parameter2 using the same pattern. For some things it goes completely off the rails, like if I start a constructor it might suggest 25 lines of code to put in it. I don't even read them, because if you're putting 25 lines of code in a constructor you're doing it wrong and I don't have to read it to know that. My company allows it and pays for it. It's useful, but I don't think I'd pay money for the service (give it a couple of years and I might change my mind). So my verdict: It's useful in the way that sentence completion is useful. It's good if it correctly predicts what I wanted to write anyway useless if it doesn't, and bad if someone doesn't understand what it said and uses it anyway.

[@lemurza5236]

Copilot is basically useless for anything relatively complex that is not a hackerank problem

[@EthanStandel]

As a contractor who often finds himself on short term contracts, ngl I really resent being categorized the way we are in that article. I often find the code quality and care from myself and my peers to surpass that of internal company engineers. Not necessarily _every_ time. But more often than not.

[@webbugt]

I treat copilot as a dumb bug on my shoulder. It keeps yelling suggestions. Most are a miss. But 10% when they hit, it's both a great time save and a great feeling not having to type repetative things

[@ryanleemartin7758]

This is great news. We need some reality to ground the hype

[@itskittyme]

this isn't reality, it's just bad research, AI works much better than people give it credit for. Here's the deal: AI does literally what the human tells it to do. And people have a big problem right now, with admitting their input (their prompting) was wrong, and blame the AI. That's the current state of things. If you can recognize your own mistakes, and learn to work together with the AI in tandem, you are unbeatable at the game right now.

[@losing_interest_in_everything]

@@itskittyme You might not have considered other users. Even if your prompts are right, the response could be affected by users with little coding skills. Also if you can't understand AI-generated code, your code will be hard to maintain!

[@ryanleemartin7758]

@@itskittyme I never claimed AI was bad or not an effective tool

[@itskittyme]

It's impossible to overhype AI tbh, people don't realize what's happening@@ryanleemartin7758

[@JackBond1234]

I'm surprised people "hit the tab button and accept code you're not familiar with" I review all the code Copilot generates, because I never know if it fully understood what I'm going for. I don't want to figure out how it works by testing it.

[@Brixster]

That's what all programmers do, except for, apparently, the ones who don't. 😒

[@Alan.livingston]

I manage teams out of code farms in India. All ChatGPT has done is sped up the rate at which they copy'n'paste stack overflow into stuff that fails to even run before submitting it for review. At least GPT gives the unit tests a good swing unlike the devs.

[@engineeranonymous]

One way to solve this issue is fine tuning a foundational model like llama with code written by your company so that it will help the users with code styled in your company coding habits. When you try to refactor the code you will easily find the same patterns and see what to change in a more civilized way.

[@jefferymuter4659]

I used to use it constantly. But I'm a Jr dev and realized its slowing down my comprehension speed. I still use it as a rubber duck which is fine.

[@joaquinbonifacino6964]

I don't use the tab function for exactly that issue, but I do use the chat part mostly to have conversations about "what's the best way I can tackle this problem" (since I am a junior in a startup being the only front dev in the company xd), but the few times I do ask the chat to code I only copy and paste after reading and understanding the whole thing, and if I didn't understand it, there is a reason why we have only 3 commands for the chat and one of them is "/explain" xd

[@c4tubo]

The buyer's remorse is coming quicker than I expected. These bubbles usually run for a couple of years before reality pops them. Perhaps it shows how flimsy this one is.

[@losing_interest_in_everything]

My company has fired all the developers who use AI for their work because it has literally complicated the management of our projects internally.

[@benbruda5203]

in what way did it complicate the management ? they were to fast ? was it too bad output ?

[@NihongoWakannai]

Sounds like terrible management if they just fire people instead of issuing new rules. Or maybe it was just their excuse for layoffs without calling it layoffs.

[@DarkzarichV2]

This is why I use something that doesn't write the whole function code for me, but just like autocomplete on steroids it can try to guess the next line and how to finish the current line so that I have a lot of control over what it will suggest next and in this sense: it completes my code, not generates the whole thing instead of me

[@DarkzarichV2]

@LeoMods_ you probably didn't get what I mean then or it's my fault for not telling it better: try Codeium (not a typo), compare it to Copilot you will notice that it completes a lot less but that's what is good about it, it gives you more freedom, I don't want a machine to provide a whole implementation in my place, I want it to understand my intention as I start typing without thinking further than just 1-3 lines. I don't want to make it exceptionally fast I like to code and think the whole task through I want to reduce some mechanical work tho. There is a difference between what both tools do

[@wcrb15]

I have copilot integrated with neovim and I use it at work and personally. At least once a week I disable it completely because its suggestions feel like they just get in the way. However sometimes it is helpful so I keep it around

[@cdarklock]

I feel like there's some combination of people who wrote NO code writing SOME code with Copilot, and people who wrote SOME code writing MORE code with Copilot. The alternative to this code is not BETTER code, but NO code. The code would not have been better-written without Copilot, it would simply not have been written at all. This would not just reduce the amount of code added and pasted, it would reduce the amount of code in general. LESS CODE is not necessarily an advance.

[@benbruda5203]

this is my view as well.

[@user-qr4jf4tv2x]

the anti dryness can also mean easier to pass in chatgpt when code is isolated as function

[@SliceOfFish]

At my last job I was working on new architecture. I ended up removing 300k lines of autogenerated code.

[@gonzalomunoz2767]

Copilot is not meant to write your feature for you. It literally says it in the name. You are the developer, only now you have a really, really intuitive autocomplete tool at your disposal. Use tools for the things they are good for, and always, always remember you are 100% responsible for the code you commit

[@blueprint7000]

You should make a video on how you use co - pilot and how it helps you . Also show best practises and maybe make a comparison to GPT - 4 and Bing Co - Pilot

[@purpinkn]

theo: i copy pasted the fastest , therefore i such good developer

[@mu11668B]

This is kinda not surprising to me. One of the lessons I learned when I got into programming is that, my code quality is much better when I understand exactly what each line of my code does. AI only helps me fill in a block to do a certain task. I still have to understand what its output does. I'd say these tools are awesome for building prototypes. For actual production, knowing each line of my code makes it easier for me to work with.

[@SandraWantsCoke]

yeah I can't just just copy paste and use something unless I understand what it does.

[@NihongoWakannai]

Yup. Reminds me of when I'd do game jams and push out code as fast as possible. But then if I wanted to use that prototype after the game jam it would take me a week to refactor the 48 hours of spaghetti.

[@everyhandletaken]

I would have to argue that this is generally the case with AI powered tooling, as it can also offer up better alternatives than what is currently used.. essentially a second opinion. That can then prompt learning a previously unknown or maybe just un-explored functionality. I get that some (maybe a lot) of users might just consider that “it knows best” and accept what is provided too though. That’s not a good thing.

[@Abion47]

I'd wager it's not just "a lot", it's actually "most".

[@everyhandletaken]

@@Abion47 you’re probably right too.. I guess people see it as a solution, rather than helpful hints.

[@thetylde]

Copilot is great when I do refactoring or work on boilerplate code.

[@LudicrousAvian]

I strongly agree with code generators being a problem way before AI was ever a problem. I once generated an entire MVC in Visual Studio, which added complex functionality that I never needed in the first place; I had to revert the commit because of that.

[@clementj2588]

One of the most useful things I found with using copilot for coding is form creation, I mostly need it to work, it doesn't need to be pretty, all the diferents input are kind of the same but needs to be different (position, name, validation, i18n...), for a form with potentially dozen of inputs, copilote can create all but the first one without issue. Its not a critical task so no problem if its not the most optimized thing (aka no buisness logic) but a huge hassle to make normally

[@lyrebird712]

My biggest thing is regarding my own knowledge of the code I am writing. I've been pushed by other devs around me to adopt Copilot and Chat-GPT to speed up workflow and solve problems. I am in a spot right now of learning various new stacks, which does slow me down to reference documentation, but using AI to write any of that for me isn't going to help me in the long-run.

[@alec9260]

Senior dev here, most of us just want to get paid and build cool stuff. The one offs kiss managements ass and brag.

[@MarisaClardy]

I haven't watched the video, but I will say, the more copilot generated code I work with, the more frustrated I am. It's all significantly less maintainable and the quality is just worse. I will also say, from personal experience, using CoPilot with PHP, it wrote terrible code because it knew nothing about the code base, and just hallucinated methods, meaning I was often spending more time rewriting CoPilot-written code than I would have spent just writing it myself, because the CoPilot code was entirely non-functional. I've heard from colleagues that it's gotten better, but honestly, not worth it. All my colleagues praise my work for it's maintainability, I'm not harming that for a mild speed of writing improvement.

[@abrahamsimonramirez2933]

AI doesn't up your game it only ups corporations' margin profits by delivering "fast food" faster.

[@voidmind]

I think Yes or No question on Copilot use is not painting a good picture. I have it at work, but I mostly only use it as an advance auto-complete. I very rarely write a prompt, so my usage pattern probably leaves a lot on the table regarding the true value Copilot can provide.

[@klightspeed]

Even with intellisense, I find I'm having to press Escape a lot to dismiss the suggestions, and having to clean up the random usings that it brings in. It's also annoying when it decides I need help writing an if block. I tried co-pilot for a little bit, but found it was getting in the way more than it was helping.

[@thegrumpydeveloper]

Love it for template code. It sees the pattern I’ve set for it and then autocompletes, obvious types are done then I can focus on the things that matter. Organizing and interfaces.

[@Bluesourboy]

I worry about co-pilot copying code from repos I work in. Most of it is the same copy pasta UI code but there are a few little gems that aren't ready to be shared with the world just yet.

[@Mcsqw]

The thing we have to remember is that AI doesn't understand code. It doesn't know what code is, how it will be run, what you're trying to do and why that might be a good or bad idea, how it's used etc. etc. It doesn't understand *anything*, it just has a set of vectors and a bit of fuzziness that when it sees bit of code A works out you probably want bit of code B next. For example someone made a Go board game AI that could beat the previous best AI using a ridiculous strategy, because the original AI didn't understand that it needed to protect its own pieces.

[@ColorblindMonk]

This is more or less the running theme in every industry. Unless there's some revolutionizing breakthrough in the tech, you can't implicitly trust AI not to hallucinate non-solutions to your prompts.

[@aCoreyJ]

I actually prefer the Codeium vscode integration than Copilot. Copilot chat seems to ignore your question a lot of the time focusing too much on adding context from your code base.

[@michawhite7613]

I tried Copilot at once. Then I found that it created so many bugs that I had to rewrite every single line of code it gave me. Fixing its output was such a minefield, because it all looks reasonable at first, until you realize it's missing about five edge cases.

[@haleyk10198]

Keep in mind that Copilot is still illegal on some places on the world, not that you cannot find your own workaround -- just that employers ought to ban them at workplace to avoid getting into troubles

[@zactron1997]

The concept behind co-pilot I'm totally on board for. My concerns with AI tools (and why I haven't adopted any) are the legal provenance. While Microsoft have their Copilot Copyright Commitment as a shield to protect me personally from litigation, the letter-of-the-law isn't my concern, rather the intent. I write a lot of PRs for open source projects (Bevy is my current focus), and I'm personally fine with my code being used in these tools. Not super happy knowing Microsoft will profit from my work, but it's what I expected going into open source work as a hobby at all. What I'm not ok with is knowing that GNU are other copy-left licensed code is inside that model against the wishes of its creators. People choose these licenses for a reason, and it's a moral and legal issue to ignore them. Now, if Microsoft trained a brand new model completely off of creative commons (or similarly permitted) code, I'd be far more interested in using it. I'd be especially more interested if I could train the model off the projects I work on and run the whole thing locally instead of sending every keystroke + context back to Microsoft. Until then, I'd rather get better at what makes me an engineer, which is problem solving. Being faster is always useful, but just like CPU speed is never really the limit to your web app's success, your APM in Vim isn't either.

[@guppy13]

i don't even use regular autocomplete. keeps me sharp

[@jarrodhroberson]

The problem with generated code is the generator has no concept of intent of what it just read. Large language models just generate what they think is the most likely text to come after something. They have not been trained on what that text represents from a logical standpoint, it’s just characters separated by spaces with no other context.

[@m_moehring]

A thought on the autogeneration comparison.... If I see a Rails project and start looking at components within it, I can immediately identify things that I would expect to be auto-generated when I know the framework, because of that standard that all similar components are commonly made with the cli. There's a pattern to what is typically auto-generated and what is not. With Copilot, there is no such pattern, because it can be applied to far more things and it comes down to the far-less-consistent choices of whatever devs are using Copilot as to what they auto-generate and what they don't. In a project that has Copilot-generated code in it, I'm not sure at a glance whether tests files or class files or a group of class methods are generated or not, or where I can expect to find the generated vs. non-generated code in a project. There's not enough of a standardization or even a pattern to set your expectations.

[@denshitenshi]

For someone less experienced like me I think using AI for coding is robbing myself of important experience in coding. You get better by doing and co-pilot means you do less

[@nicksophinos4611]

Garbage In, Garbage Out

[@ifonlyicouldenglish]

I'm using Continue, a similar tool that allows me to run my copilot from locally hosted LLMs! I'm only just starting, so I still need more practice using it with VSCode; however I definitely feel more capable as a developer with these AI-powered tools!

[@renatolovato]

I had never used Co-pilot until five minutes ago after I started watching this video. I was trying to find an answer to how to do something and struggling to see it in a poorly documented API, but AI gave me the answer within seconds. I loved it!

[@dhillaz]

Could it be that some of the code churn is simply devs being enabled to prototype features faster, and then rewriting them into a more robust structure once they know it works?

[@thumbupandy813]

Copilot were trained with just random code from github, not the best selected code from best programmers.

[@notarealperson9709]

give a junior copilot and you're going to have issues

[@MerkieAE]

lol and they said it would replace us!

[@miroslavpetrov3006]

So we are going to need more developers to fix the code from AI? Nice!

[@RockyTheDog756]

Copilot is definitely is a huge go-to for developers, but not for businesses, in fact the time we spend on coding is about 5%-10%. All the rest is communication with colleagues and business to align requirements/api/ui/etc Off course synthetic test where you code some personal task will show significant boost but not the real life cases. I can confirm it from my own experience as heavy Copilot user and expert software engineer in big company.

[@3ventic]

Code churn is not a bad sign on its own. We're just getting starting points shared earlier and trading that one good longer cycle for two (or more) quicker ones that can take less time combined: first the quick AI-supported solution that probably misses some nuance, then the proper review and fix.

[@mikopiko]

I refuse to give you a like until you stop with these thumbnails

[@DelandaBaudLacanian]

Unfortunately those thumbnails get more likes

[@Kane0123]

You seem upset

[@salatetis]

It looks like Theo is visiting proctologist all the time tbh

[@mikopiko]

A bit @@Kane0123

[@restcure]

@@Kane0123 Would you like help? * drafting a letter of complaint # don't show me this tip again I kinda miss Clippy ...

[@offtheball87]

They enabled it at work like ten months ago, in the middle of a project I was tackling in a new code base. It was awesome, I was impressed. I had to rewrite a bunch of code, but it was still impressive, and it made great comments. Then it stopped working, and I haven't had it since. Tbh I've not been bothered to go figure it out.

[@Toberumono]

So… what I’m hearing is that exactly what I was shouting from the rooftops would happen happened. Cool. Would have been nice if I and the massive number of other people who pointed out these extremely obvious flaws were ever bloody listened to… Edit: as somebody who gets called in when less experienced devs in the company can’t figure out a problem they’re struggling with, AI has made my job a *nightmare*. My personal favorite was a junior dev using AI to fill in security headers. (Admittedly, it was just CORS, which isn’t the worst place it could be, but they didn’t even think about it before pressing tab)

[@RandomGeometryDashStuff]

09:14 is lazarus (freepascal gui ide) codegen?

[@ShaharHarshuv]

I almost never accepts more than 1-2 lines. And I never accept logical code without reading it. Even though - reading is faster than writing.

[@StryKhymorodnyk]

Good day! Great video as always. I really wonder, how you cope to handle all you know about coding. The thing is I had to self-study as I am from Ukraine and now I know, I got Asperger's. I already have 2 higher educations, but at my 40 Asperger's doesn't allow much as I got tired from learning too fast and all I have learnt, I forget almost immediately. I shall search how to study as I think I have seen somewhere in your videos. Thank you for good content as always!

[@DamianTheFirst]

as a Polish buoy I cannot agree more with "maintaining the past". About five years ago at uni we had IT classes where we were taught... Pascal Already then I was familiar with the basics of c++ and started learning Python. Learning Pascal was like a history lesson for me but, unfortunately, it wasn't the most useless thing we were taught. Suffice to say is that I got burnt out and left before graduation.

[@96alguyf]

I like the idea of copilot, though I'm a newer dev so I wonder if I should "hold-off" until I reach a certain comfort level.

[@jsrjsr]

The general advice here, is that as junior dev you should avoid it. Then once you are "capable enough" use it as an autocomplete ( recommended use) or, at least, read and understand every line of code copilot produces before using it.

[@Brixster]

@@jsrjsrI second this. If you can't perfectly understand every line of code copilot is writing, you should absolutely back off until you do and come back when you're more experienced. Excellent advice for new developers 🙌

[@Dreamslol]

I use cursor, imo its way better than copilot because you can integrate recent docs and have more control which files are getting used for context, that helps alot

[@darrenzou2483]

I like copilot because it helps me write sql and queries really fast

[@plinatry783]

I've fixed the temptation to just accept complex functions, by reassigning the tab-autocomplete to only autocomplete the current line. Belive me, thats a gamechanger!

[@Alan.livingston]

I thought copilot was meant to be suggestions, not used verbatim. Also, where is the reviews?

[@joaodotcodes]

Copilot still helps me out a ton, especially with doing small scripts sometimes for importing stuff to a database or repeating the same logic with different variable names (something like a switch statement)

[@pyrosar3517]

> repeating the same logic with different variable names This just looks like you never needed a switch statement in the first place for that situation. You could just write reusable function that will accept that variable as a parameter, considering that the logic is literally the same and only variable names are different.

[@joaodotcodes]

@@pyrosar3517 sure, fair enough, was just using an example taken out of context and purely from memory, but the point still stands of it understanding the logic quite well and saving me some time, maybe not switch statements but… different attributes for different JSX elements with some sort of similar logic?

[@AmatuerHourCoding]

​@@pyrosar3517bro is doing a pr review without seeing the code

[@mr.rabbit5642]

If Im understanding correctly, it basically comes down to defering the construction of the component itself to the AI, but doing the wiring yourself

[@Abion47]

There are so many applications of people saying "Copilot really helped me by writing X for me so I didn't have to keep repeating it" and I just go "you literally just admitted that perceived convenience is more important to you than DRY".

[@mjsmagalhaes]

I tend to generate tiny amounts of code per prompt so I haven't seen copilot gone that bad. What I felt is since I am moving a lot faster I can focus more on the solution and not getting side tracked or changing my mind because a week later I am not so sure what the plan was.

[@davronsherbaev9133]

anyway, that's why its called Co-Pilot, because its meant to fly with human) I am a strong Copilot user, to such extend, that I sometimes think that it can hear my thoughts. I cant imagine writing console.log's by myself. On the other hand, I think many people forget that copilot is actually learning from your code style, so if you write bad/unmaintainable code, it will suggest bad/unmaintainable code. At certain point I simply explain what I want to do in the code, and copilot just does this. Sometimes not correctly, but its very easy to fix mistakes, as long as you on the same track as the code that is being suggested, if you are not, then probably its time to learn how to read code)

[@xcat4775]

itinerant is definitely a word

[@Abion47]

Itinerant: a person who alternates between working and wandering. The article is saying Copilot is basically a developer who drops into a project, submits a PR with a bunch of poorly thought-out and untested code, then moves on without bothering to stick around to see what damage their PR has done or if it was even accepted.

[@helleye311]

As a polish dev, I welcome our AI overlords. I love copilot, especially when making yet another boilerplate function. But unless it's something simple like writing a react query wrapper, I always read the code after inserting it. This does mean it sometimes takes longer to fix the AI code than it would to type it by hand, but also this brings some much needed variety to coding. Sometimes you just blaze through stuff with perfect suggestions, sometimes you write stuff yourself, sometimes you spend a few minutes debugging. But for my brain it's better than always typing manually. It's like snippets without snippets that sometimes are bad. I do think it's dangerous in the wrong hands though. I personally wouldn't give copilot to a junior dev.

[@stephenjames2951]

However non-tech leaders will push it for junior devs in order to push productivity.

[@truvc]

"I can understand the intent of a senior developer. They're trying to feel good about their job and brag." 😂 You sir are correct

[@HaseebHeaven]

As a professional software developer, I only use copilot only for my personal projects, and I recommend using people using this only to boost their productivity in personal projects, and not for the commercial use because of the privacy, and there is a lot of issue with the code generation and the code quality, so it does not maintain the code quality, and it does not have a lot of context and the knowledge of the whole court base in our commercial in some big companies there can be the largest base so right now there are not any tools which can understand the whole code base and write very efficient code, and then managed also the code and also do all the debugging themselves so the system needs to be improved for the court generation for this artificial intelligence.

[@rw-xf4cb]

I take exception to your Senior Developer statement - as a Senior - I know how good quality code which is maintainable, documented, using variable names that are appropriate, not writing in sanscript so the next developer can pick it up - there's always cost benefit to this but if the code is intended long term - year+ then spending some time to make sure its laid out well according to teams standards and code quality tooling has it meeting or beating the metrics is usually good! Nothing to brag about - same as bricklayer or welder there's doing a fast botched job where its acceptable to being something of quality.

[@AleksyGrabovski]

Also polish dev here, I've started to use copilot as soon as it became available, but quickly abandoned it. I program embedded C and it just wasn't helpful enough. Maybe it is better now, idk.

[@ausername27]

I haven't used it out of concern it'll gimp my ability to program. But I can see myself adopting it for work if I need it to stay competitive.

[@HaraldEngels]

It is the opposite. With some willingness to dig deeper into Copilot' features it becomes a great teacher. Although I am developing software since decades ,with Copilot I am learning faster than ever before.

[@antheus_s]

Honestly, Copilot is great as long as the developer that uses it is skilled enough to know how to use it and when it spits out garbage. It saves me a lot of Google time for trivial things and improves my DX when I'm refactoring code with a clear refactoring pattern.

[@sokacsavok]

The Rails take of course is bad, as always from people who haven't had much Rails experience. You don't have to use any generators to create the files. You can do it manually, copying old code (as you do with every other framework). I only use generators sparingly, mostly for generating models. Instead of writing SQL/migration code it manually, it will also automatically check my types in the command-line, so I am more certain that the generated code is correct: "rails g model todo name:string description:text user:references done_at:timestamp". Then I add some more details if needed. I would say it mostly helps when you are still learning the framework.

[@ELStalky]

All depends on usage. If you ask it to write code you could not write yourself, that's an issue. If you mainly let it help in writing what you would have written yourself anyway, great. It just exacerbates the problem of people copy & pasting shit they don't understand.

[@akparhi007]

Was bound to happen, copilot helps but only as an autocompletion system on steroids.

[@abdjahdoiahdoai]

“who’s on the hook to clean up the mess afterward?” dev be like, let’s use AI to fight AI 😂

[@proosee]

I think there are few things confused in this video, for example cli boilerplates *are not* code gen - code gen is when you have metadata and you can generate code from that metadata, example: proto files and DTOs generated from them. Code gen cannot be compared to AI generated code because the first one is deterministic - you can commit your metadata only (like proto files from my example) and every dev will have the same code after running code gen tool against those files. Although, I'm against languages who have too much boilerplate code or conventions you must follow - this is bad language design if you force dev to follow your way of doing things. I also think argument for DRY is very important, the reason why software is software is basically that you can reuse it on demand and you only write things that are new to you - it's waaaay easier to maintain few pieces that have well defined interface and are reused in multiple places than have 10/100/1000 different version of something and be forced to test it individually that many times.