When Evidence-Based UX Design Is Not Good Enough: Duolingo Redesign Backlash

[@cmbsgc]

Thank you for the interesting analysis from a UX perspective. I would like to add that the "learning science" behind the update appears to simply be the research on spaced repetetition. Very basically, spaced repetetition is the idea that material is best memorized if it is encountered repeatedly (as opposed to just once) and that material which is difficult to memorize should be encountered more often than easier material. Spaced repetition is therefore very useful for vocabulary learning (and the learning of some grammatical structures). However, language acquisition encompasses much, much more than the memorization of vocabulary. And two of the strongest predictors of successful language acquisition are Intrinsic Motivation and Learner Agency. Intrinisic Motivation refers to the motivation we feel inside ourself (enjoyment, interest, pride) when we do an activity. Learner Agency refers to the control we have over our own learning, and this in turn effects our Intrinsic Motivation. Less agency results in less motivation. The new UI completely removes all Learner Agency, and you can see the results of that in so many of the reviews ("I lost my motivation to continue etc.").

[@cmbsgc]

I will reply to my own comment to add that Duolingo also relies heavily on Extrinsic Motivation (streaks, leagues, XP, gems etc.), which is the motivation which comes from outside ourselves. If you complete an easy lesson not because of enjoyment, interest, or pride in your own language education, but because you want to maintain your streak or your place in a league, then you are Extrinsically Motivated. The streak becomes more important than the language. Extrinsic Motivation can be more powerful than Intrinsic Motivation, but only *in the short term*. A person holding a gun to your head can make you surrender your wallet, but if the gun is removed, you will feel much less motivation to give that person more money. Language Acquisition is a lengthy and demanding process, and the only thing that can fuel it properly is Intrinisic Motivation. That is the "learning science"!

[@hansulibakshi9303]

That’s a great perspective 👍

[@Svartr.HrafnSvartr]

Also to add re: Spaced Repitition - Duolingo's research study was conducted in relation to the TREE interface. Many people do not understand that this whole 'spaced repetition' thing was something that was already in place in the algorithm before the update. The CEO has also stated to his shareholders that the primary reason for the change was to introduce more IAPs. I personally give little weight to the argument that this linear path was for the benefit of language learning. It excludes many people with different learning needs while taking away agency from everyone.

[@As-yi9r]

@@cmbsgc what learning languages app do you think is better for long term language learning ?

[@heidicheung6068]

love your perspective on this. It reminds me of how important it is to inform ourselves on user psychology when diving into the territory.

[@elenabob4953]

2:06 No, the content isn't at all the same. I was at level 5 and I needed to do another 4 lessons to reach level 5 but with the update I arrived to a new module with completely different words and times. It is offputting

[@8-bitmodern437]

I just lost my streak. I was at a 240 streak in Japanese. After the update it severally stunted my motivation, as I'm a fast learner and the old tree UI let me jump around a lot. I had a crap load of crowns and was at the 3rd unit almost. I want the old ui back

[@jacktophono1]

If you're on computer you can change back just search 'how to change Duolingo's path back'

[@AlexandraElliott-tc5qc]

A better decision would have been allowing users (new or existing) to choose their way of learning, keeping old users happy and new users the ability to trial both where metrics could then be gathered around preferences for future major changes

[@Varrok-up4zf]

Definitely! Humans hate changes.

[@jaronkunkel]

Thanks for making this video! I'm Spanish Teacher and an aspiring UX designer, so this video is super relevant for me. I'm also enjoying your book, Get Into UX. Thank you for all the content you create for us!

[@rorypatrickmorrison2244]

Really love your content! The interview warmup video helped me in a job interview this morning! ✨ 🚀

[@vaexperience]

Great job!

[@teodortonev9312]

I think that people liked Duolingo because it allowed you to not be as linear and introduced a new way of learning. The problem I think is that by using learning science based methods they changed the process to a more classical process of learning. (which was the thing that differed duolingo from any other learning app.)

[@818deadboys]

Good perspective. I was looking for likeminded videos that talked about the learning aspect that's changed (mostly for the worse), but this UX perspective is important, well done

[@gezma12]

The new format absolutely sucks. I am on a 2065 day streak but I decided not to continue and I won't be renewing. I'm a bit sad I invested so much time into something that's changed so drastically for the worse. It's confusing and completely joyless now. Has anyone got any good suggestions for an alternative to Duolingo?

[@elenabob4953]

The science was wrong. If you take me as an example, I was their ideal user because I systematically follow a linear path even in the tree organization BUT I am unhappy with the change ( even considering to give up using Duolingo) because my motivation went down because the end is no longer in sight, you scroll and scroll and you get bored before you find the end. Even if the number of modules is the same psychologically the new format make it look tedious and almost impossible.

[@ilariaromeo9386]

As a Duolingo user, when the interface changed I hated it at first. I wish they at least explained why they made such a drastic change! I had to understand how to path worked on my own, and with time I got used to the new interface. Even though the new interface makes you learn new content much slower, it makes sure it sticks better to your mind. I think both methods were fine. Like this tho, people will take more time to finish a course and stick to the app for a longer time. Still, I probably preferred the old one, as I like to jump from one section to the other, rather than going step by step.

[@anthonyzheng3155]

Are there strategies they could have implemented to make the switch less dramatic? For example, creating a daily course that has 4-5 of these dots to complete. Testing the waters before jumping in basically.

[@anthonyzheng3155]

Great video!

[@vaexperience]

Most definitely 👍

[@designCred-tc1sv]

In my view, they most certainly could've AB-tested this before rolling it out. I could be missing something, but this does not look like it went through any sort of beta testing or anything of that nature

[@jeollim]

i was quite cool with the new tree as someone with 250plus days streak in dutch prior to the update. The old one did feel quite heavy and less progression satisfaction trying to complete one unit at a time last time. but i understand the frustration in being forced on a single path for learning. maybe the middle ground is where they need to be

[@Thomas-u8q]

The new interface removed *ALL* grammatical information that previously existed ('tips') for the unit lessons. This really is quite serious for languages with complex grammar like Russian, or others where quick clarification isn't so easy to come by, like Welsh or Romanian. Not to mention after the icons and names for units were removed from the 'tree' UI, it now takes far, far longer to see what each unit is about, if someone wants to revise a particular point of grammar. I haven't bothered to mention the stupid cartoons (they're still blockable with PC add ons), but I don't think a lot of people want to see a bunch of dumb animations looping on the main menu either.

[@omerqureshi5578]

yup its sort of a mess now back then we had categories ... basic 1 basic 2 family 1 family 2 etc. now its just dot dot dot

[@steffenkautz1460]

I think that the new approach is better from a learning perspective, but the implementation could use improvement. The point of criticism that seems to be the loudest is the lack of agency and overview, because it is not immediately clear that you can keep practising golden steps, and it is not immediately visible what exercise lies behind a stepping stone. So improving system visibility while maintaining the more linear level of course guidance might improve the overall experience

[@omerqureshi5578]

@@steffenkautz1460 agreed mostly they need ro work on module naming as the previous ones were better + it seems to me that they completely removed hard practice. Other than that I am getting the hang of it but still looks... can't describe it

[@steffenkautz1460]

@@omerqureshi5578 I do enjoy the personalised practice. From what I can tell it's surfacing tasks and concepts that you struggled with before. I think that was only a Duolingo super feature before

[@georgioskondylis8828]

Very interesting video! I think you might undervalue the effect of the changes regarding gamification. The new version bombards users with a free to play logic that makes it very tiring for users to continue using the app if they dont upgrade to the paid version. I used to be a user of duolingo but this development made me feel treeted almost as a victim. A victim if i use the free version because of all the stupid ads that i had to endure, and a victim as a paid version user because i was feeling that i was prematurely forced to pay for a product that hasnt yet proven its value to me.

[@camilodelbusto2172]

Hi, I like a lot your video and I'll like to know a little bit more. Do you have blogs, articles, etc where I can find more information about this change of attitude of the users based on the UX design ? Thank you :)

[@Tima.Fard.]

The reaction of the old users is completely different than the new users, It is really interesting. I just joined the Duolingo in 22, I was enjoying it 😅 The old users compare and also formed their expectations, these could be of reasons

[@emilyannfrancesmay3919]

I agree. I found one of the users on TouTube who tested the pre-release. He is in his early 20s. I found our exchanges very positive. There was no bitterness or anger sent my way. When that happens to me from a user I see that as signs of a deeper issue on their side.

[@davidbieschke9345]

Great video to send to someone who doesn't think UX is important!

[@sallymonk2645]

I just returned to duolingo after a gap of more that a year, and after about five minutes with the UX went searching for, basically, WTF about the change. Of which, clearly, there is a lot out there! I'm not usually very change-averse, but this time I was fuming "NO,NO,NO...." (little storm cloud emoji). I have figured out that my reaction is mostly just not liking change, especially going from kind of complex to very, very simple. The thing that is Kind Of making me more accepting is the idea that they changed it to work better. And I would be a lot happier with the change if they had done fairly extensive testing of the old vs. the new, showed that it was significantly more effective, and told us that. (What I have seen so far does not convince me that they did that. It may be that they just altered the UX to conform to proven educational techniques and did not bother to test the effectiveness of their old and new versions against each other). The idea that you can just do what you think is better and drop it on people and let them deal, is so incredibly arrogant.

[@ahmedrefaat4836]

Good, but a question after the courses and reading many books, how can the UX designer get projects that he can put in Portfolio

[@ancii023]

Great video, though it would have been nicer to have your point of view on what mistakes you believe happened during the Research/Design process! Was it the lack of testing with the existing customer base? I agree that their business goals were wrong and were implemented in a way that hurt their existing customer base. But what actions could they have taken in the testing stage that would have uncovered just how big of a mistake this redesign would be?

[@vaexperience]

Hey I have a followup in the books, but it could be a lot of different factors and it would be a rough guesses. stay tuned

[@ancii023]

​@@vaexperience It would be an interesting breakdown for sure, as we see it happen with so many big companies - seems like it can happen to just anybody! Keep up the great work, love to see it 🙌

[@otherworldorg]

I wish you had more personal experience with the app. There's a lot missing. I can tell you that before the change, I was pretty much using the app in the that the change emphasizes. I almost always finished a full unit before moving on. The new layout likely reflects the behavior of most users. But there's a lot that changed the way it feels like you interact. The longer pathways make it harder to look around at a glance at what you've accomplished. There were definitely issues with the old method of showing progress. But I don't think the new layout has actually resolved any of that. One of worst parts is how you are rewarded. In the old system it was possible to get between 15 and 25 points for a new lesson, and 40-50 points for practicing an old unit. Additionally, it was possibly to get double points for 15 minute periods by meeting certain requirements. The new system only awards 5 points for a practice session that runs about the same amount of time. And new lessons give you about 10 points. Same amount of time put in, significantly less competitive rewards. While the learning quality is about the same, it's going to severely affect your motivation when competing for places on the leader boards. I might have forgiven that if they hadn't forced my upgrade in the middle of the weekly competition. But when you consider the rewards-based motivations, they just put all the players on suicide mode. The competition just became a lot harder. Even other areas of reward-based motivation took a hit. You used to get a little cheer for doing 10 lessons a day. Now they push for 12 lessons. I used to get a certain reward for usually completing two lessons (give or take, their points system was always funky). Now it takes 5-10 lessons to get that same reward. New users probably won't notice this. Existing users just had the wind taken from their sails. I think I do detect some adjustments to the lessons that might help with learning.I'm trying to not quit on a whim right now. But I definitely felt like I was slapped with the change.

[@brasscross5476]

No agency felt on the player side, not even fake agency, it's like they never played a game i their life.

[@freewindfree]

All of that could have been fixed by asking customer - how they would like to learn - by following learning science or to be “in control” and choose yourself. They took away control from the user and of course it wasn’t appreciated.

[@Gagan_salwan]

The question here is if we design an app without taking care of Ux but somehow its working, Do we need to imply UX and do the redesign?

[@vaexperience]

Nothing ever just works. It can always work better

[@Gagan_salwan]

@@vaexperience Fair

[@irinavadan8604]

Can't wait for it!!!

[@steffenkautz1460]

I think this is a case for humans being creatures of habit. There are plenty of examples in the area of video games, which I would at least partially count duolingo into, where developers have made smaller or bigger changes that caused some outrage in the community. Around those times, you can see lots of negative reviews go in at all relevant aggregators, which is inherently biased because a content person is much less likely to write a review without being prompted. Either way these games tend to do a little less well for a few months and then recover or even exceed their previous plateau, because the changes they made were good overall. Some people get upset when their habits are challenged, especially when their daily Duolingo lesson(s) were a source of comfort and a constant in a turbulent life. I believe that once they get used to the change, most of them I assume will return to then get upset at the next major change that the app makes, reminiscing of the old days before learning-path Duolingo.

[@designCred-tc1sv]

I would actually hesitate to call this evidence-based UX design. In fact, I would say what they did goes against UX theories - design thinking, agile, lean development, etc. What they did here is put their design effort away from the actual users, instead strategizing based on what they think users want, which is a very bad (but forgiven) mistake businesses can make. Designing based on a theory coming from research institutions only proves that, as what comes from a laboratory is seldom what touches people on a personal level. Theories like that are meant to find out how large forces work, like supply and demand in macroeconomics or planetary orbits for physics. Businesses don't develop theoretical pieces of work. They build things that touch people day-to-day. Yet here is Duolingo breaking away from that line of reasoning. I'm actually rather shocked that Duolingo did not think about search patterns people use when they are surfing the Internet. It would've been CRYSTAL clear that search patterns are much more scattered than organized, preventing them from building a linear interface. Heck, they can just look at Khan Academy or Coursera and see how their users can just pick up any course they want and learn as much as they want, whether that's only a teeny bit of information or the whole course all at once.

[@chaturaindika]

I don't know about others , but i love the new approach

[@giuliaferrero7447]

It seems I am the only one who prefers the new version. Before I struggled to find motivation and felt to not proceed with learning. Now I perceive to be in a teaching path and I can see my progress

[@DecidiousGreens]

200th like 30th comment