When You Shouldn’t Remove Code Duplication (And How to Refactor the Right Way)

[@ArjanCodes]

💡 Learn how to design great software in 7 steps: arjan.codes/designguide.

[@OzFush]

I’m a big fan of the Rule of Three - you don’t know what the abstraction should look until you have three examples, so duplicating something once is good, then refactor instead of duplicating it again.

[@evlezzz]

One more case when DRY do not apply is accidental duplication. It happens when two independent tasks in the moment could be implemented with identical code, but has different reasons to change.

[@ApprendreSansNecessite]

That's a huge point, the principle of bounded context helps here.

[@SubwayToSally90]

At my company we started to avoid "DRY" quite a while ago, because not so experienced developers tried to take it by hearth and prematurely abstract EVERYTHING to a point where functions where not abvious if they where ther right fit for a case or not. We now rather follow "WET" -> write everything twice. Its fine to have a code duplication. The cases might be similar but not similar enough to be worth an extraction and ot might not even be worth the time to propperly abstract it (yet). Do you come by the same code a third time? Great, now is a good time to abstract it and reuse it a bit.

[@FilipePauloMax]

As a software engineer I can argue that the "calculate_any_volume" function is not a good generalisation. It's way harder to read/understand than the simple ones. In that specific example, I'd trade out a bit of duplication over fully understanding what the function is doing.

[@virtualraider]

Someone in the Golang community jokingly suggested the WET principle: Write Everything Thrice to understand the patterns before removing duplication. Not a bad idea 😁

[@ArjanCodes]

I like that! 😊

[@SubwayToSally90]

Just have seen your comment. I basically wrote the same in a different comment and we do the same (even tho we name it "write eveything twice") and only abstract the third time it comes around. Makes it easier & helps especially with unexperienced developers who "see the possiblity of a reuse" of something and directly abstract everything up 2 levels and lead to large functions with edge cases that never happens. :D It's a realy good principle. I like WET over DRY every day now.

[@dalehagglund]

Nice video, as always, Arjan. You're certainly right that it can be hard to figure out how to extract the commonality from similar-but-not-quite-identical blocks of code. One technique I sometimes use is to slowly *introduce* code to increase the similarity of the blocks until they're true duplicates at which time extracting the duplication is straightforward. (I also like the example of trying too hard to eliminate duplication.)

[@dankprole7884]

I have got myself into trouble removing duplicate code and doing abstractions way too early before i really knew what they should look like. Being a neat freak has its issues. Code always runs though!

[@tascsolutions6483]

Always revisiting your code is key, especially when starting out and learning a language. As I got better with functions, I was able to de-dupe hundreds of lines of code. I found the quickest and easiest deduping related to plotting/matplotlib.

[@yehoshualevine]

Nice! I hope you'll do another video about partial functions vs DRY principles: The `caculate_any_volume` of this video could be turned into the other volume func's as partials. In real-world code, what wold be some techniques, advantages and drawbacks to approaching DRY with partial functions? (But calculating a volume is laughably too contrived to illustrate the point!)

[@resresres1]

As someone who is not a python novice, i appreciate the more complex example.

[@FridolinRath]

Hi Arjan, once you mentioned to use more pathlib :P. I just remembered when I watched your video. Maybe there is even more refactoring possible. Greetings from Germany.

[@shubhambindlish1124]

the duplication becomes a problem around deadlines - ppl generally copy paste code and get to the deadline - focus on clean code is really really low.. one trick that we use is - after every release generally 3-4weeks - the next 3 days are spent on just reading the code and spotting these issues - feel thats a good way to spot where things are complicated / repeated..

[@radeksmola3422]

Deadlines are killing creativity and lower cognitive ability. I do not like deadlines and stress during developing process.

[@shubhambindlish1124]

@@radeksmola3422 i hear you.. but without deadlines, i dont think i would ever end up shipping a release.. there will always be something that i can improve / iterate.. :)

[@muyou0107]

@@radeksmola3422But deadlines are necessary for productivity. That’s how things are done in time.

[@DougRansom1]

So many programmers duplicate code for loops over and over again, when those loops are already available in map, reduce, and filter.

[@deoradh]

Many devs (people in general) have a hard time telling the difference between contract and coincidence. That’s driven too many PR complaints about duplicated code, even across boundaries that need to be maintained.

[@pragdave]

Just an observation. In the book, I wrote that every piece of _knowledge_ should have a single place of expression. Code is one representation of knowledge, but not the only one. It just happens to be the easiest one to spot :)

[@ronmanders]

A different take on this: You could also argue that making code too generic often results in a function/module/whatever that simply gets too many responsibilities. Then it either gets too complex, or too abstract, or both. So then it violates the single responsibility principle. So you could say that a function that calculates any shape has multiple responsibilities. And in case of conflicting principles, it becomes a balancing act between them.

[@hcubill]

Great concepts thanks Arjan

[@giorgioripani8469]

Another issue or the last example is the evaluation time or the function, since boolean ops are the worse in terms of big O timings

[@owenlu8921]

Thank you so much for this video!

[@ArjanCodes]

You’re welcome!

[@philadams9254]

But now the *validate_dimensions()* call is duplicated many times. Is it not cleaner to validate the inputs before calling the function?

[@carl2488]

What is doing the online auto complete here for Arjun?

[@MMarcuzzo]

Removing duplication and unifying things are great when all those stuff change together. Otherwise, be careful

[@_DRMR_]

And when you end up duplicating code between projects it's time to start your own library ;)

[@2broke2code]

Love your work

[@ArjanCodes]

Thank you, glad it’s helpful!

[@alexivanov4157]

Don't use DRY KISS and stay SOLID!

[@ArjanCodes]

It’s pretty clear that you truly GRASP design principles!

[@SentinelaCosmica]

remove this flag there..come on

[@tankmohit]

I use WET KISS

[@josmitube]

You can say that again!

[@flavio4923]

when you remove duplication, you can make things cleaner, but if you refactor things too much you increase coupling and decrease explainability

[@ramimashalfontenla1312]

Great video!

[@ArjanCodes]

Thanks!

[@rydmerlin]

The video starts but I don’t see the IDE giving you any hints about code duplication. Why is that?

[@andrzejostrowski5579]

Don’t use f-strings with logging! String manipulation takes a lot of resources and if you use an f-string, you often need to discard the result of that operation if your logger is not configured. Or you don’t use given log level. If do this in a loop, it really adds up.

[@AndreaDalseno]

It's a great video, as usual, but the audio has poor quality this time!

[@radeksmola3422]

I like abstraction for calculation of area, but it is hard to read by looking at function call what is happening because general name of function. And by reading it from parameters is not easy.

[@radeksmola3422]

Ha ! you are explaining it ...

[@DrDeuteron]

timestamp?

[@radeksmola3422]

@@DrDeuteron it was example of bad deduplication. I wrote here too early.

[@kiraleskirales]

You could remove the duplication in your example of calculating the volume / area without losing readability. 1) Create a function that calculates the volume of a shape given its bounding box dimensions and its ratio volume/volume of bounding box. You can also inserts checks here. 2) Define the function to calculate the area of known shapes by calling the function above, or using functools.partial and give intuitive names. This makes it also very easy to extend to new shapes.

[@Cleanblue1990]

I think the validate_dimensions function is too unclear. If it crossed my way, I wouldn't know that it raises an exception. I would have read the explicit implementation quicker. In part, this might be solved with a different name like 'raise_if_any_negative'. And that suggests the shorter implementation if any(x>0 for x in dimensions): raise Value error(...)

[@imyafamiliya2155]

Less than zero I think. Also decorator would fit better here IMO

[@jonathanpiaget5195]

When I started web development with Django, I would often shoot myself in the foot by trying to create mixins to avoid duplication in views or forms. I always regretted it later 😂

[@Henry-sv3wv]

Someone said: Abstraction is DRY, the "compression of code".

[@HanWechgelaer]

I definitely copy multiple instances of the same code in my projects (copy paste alter) which results in 'quick and dirty' solutions amd of course bugs, after that I look to my code from a 'what is it doing exactly' and then optimize it by removing duplicate code.

[@johntamplin]

DRY, like many rules, should be subject to the quote from Douglas Bader "Rules are for the obedience of fools and the guidance of wise men." Unfortunately, everyone thinks they are wise.

[@SentinelaCosmica]

I am learning multi-agent ai to develop an earlier sepsis detection system using bayes network, I have good understanding of database distributed systems and network communication, what is my odds?

[@marcotroster8247]

I think DRY is even more important for test code to avoid duplication in the test setup logic. You should show that on another episode. For production code, IMO a vertical slice architecture with proper BDD tests is fine. Some duplication buys the ability to throw the feature away without any coupling to consider. In big systems, coupling is the true painpoint anyways.

[@jasonpease7831]

I'm stealing DAMP! 😀

[@ArjanCodes]

Go right ahead 😁

[@QueirozVini]

So, DRY, unless it breaks SRP.

[@Fasyle]

The constant camera zooming is awful. I know someone is trying to draw me in or something, but it's waaaay over used.

[@patrykforyszewski4655]

DRR. Don't Repeat Repostory ;)

[@dwhall256]

I most commonly encounter code duplication between large functions. The two functions do something different, but have a common set of steps between them. And there's usually one small but difficult thing that prevents easy de-duplication. For this reason, don't let a function's size get out of hand during its creation. Once it reaches a screen height, time to factor something out. Smaller functions are easier to re-use than larger ones, preventing future duplicate code from appearing.

[@yojou3695]

if the functions do something different, they should be kept separate. And maybe the common steps you can abstract into a different thing. People are way too scared about long code, having it all in the same place, as long as it is understadable is okay.

[@shawnedwards5369]

"...not what you see in production code." Wanna bet?

[@HansBezemer]

True - "get it working and get it out" is often more important than optimizing the code. *Tip:* overestimate your maintenance jobs. It buys you some time to reevaluate your choices. And never violate the basic architecture. It'll save you a lot of time later on.

[@shawnedwards5369]

@@HansBezemer During sprint planning, and _especially_ when assigning points, I make it clear that I'll be doing testing and cleanup as part of the ticket. An extra hour of thoughtful refactoring and cleanup can (and likely will) save you days of debugging and fixing.

[@BrianStDenis-pj1tq]

The example of refactoring the calculations of area was interesting. Not only did it make the code more complicated, you inserted a for loop which makes it much slower. If you think multiplying a few things together needs to be refactored, you are the type that may over use the DRY principle. Let me say that again...

[@sebastianrossetti6167]

Are you really this dense?

[@juanpyusun]

🎉

[@sami3592]

I like the joke. 😊

[@asagiai4965]

There is a reason it is called DRY not DRC So there are exceptions to the guide. But if you find yourself repeating a lot. Either A.) There is no solution for that language. Or B.) The current code is much better than the solution.

[@efovex]

Also because that would be the Democratic Republic of the Congo

[@asagiai4965]

@@efovex haha true

[@HansBezemer]

If your "generalization" needs to many parameters - or if it requires suspect constructions like lots of references (instead of values), reevaluate your choices.

[@asagiai4965]

@@HansBezemer adequately enough parameters. And who uses too much references in a construction? I guess people who wants problems.

[@JeanMarieGalliot]

The only problem is that your examples are too complex, often involving video processing, and that kind of stuff that not everyone is familiar with. So, it makes it more difficult to grasp the design concept you want to put forward. I know it is certainly more difficult to crave simpler examples, but it will be more understandable for me, at least. I hope you will consider that as a constructive comment as I recognize and appreciate your expertise.

[@radeksmola3422]

In opposite, I appreciate more complex examples than easy one for beginners.

[@paulschockner1445]

I agree. Sometimes the details of these more complex examples does complicate the understanding of the concepts. Maybe the solution should be alternating between a simpler straightforward example and a more realistic example

[@JeanMarieGalliot]

@radeksmola3422 Being not familiar with video processing doesn't not imply being a beginner in programming...

[@radeksmola3422]

@@JeanMarieGalliot I understand, but in the example was processing some text files for subtitles.

[@manuelstausberg8923]

I also really appreciate the slightly more complex example here, as it highlights the different kinds of duplication (especially the not immediately obvious duplication). IMO you don't really have to know anything about video processing to understand that the duplication is found in the way that files / directories are handled :)

[@chrisw1462]

Complex code using libraries not everyone would use plus the scrolling jerking around... not very helpful for someone trying to learn to code.

[@efovex]

This is a video about the pitfalls of the "DRY" mantra in complicated real-life scenarios. If you're a beginner, you can watch beginner level stuff...

[@DrDeuteron]

WET: Write Everything Twice. can get WETTT: Write Everything Ten Thousand Times. Rule of Three notwithstanding.

[@asagiai4965]

There is a reason it is called DRY not DRC So there are exceptions to the guide. But if you find yourself repeating a lot. Either A.) There is no solution for that language. Or B.) The current code is much better than the solution.