Lecture 4: Heaps and Heap Sort

MIT 6.006 Introduction to Algorithms, Fall 2011
View the complete course: ocw.mit.edu/6-0...
Instructor: Srini Devadas
License: Creative Commons BY-NC-SA
More information at ocw.mit.edu/terms
More courses at ocw.mit.edu

[@danielkurniadi8805]

"The cutest little data structure ever invented, the heap." - Prof Srini Devadas @0:22

[@neffship]

shoutout to the camera guy

[@sergeykholkhunov1888]

02:00 priority queue 06:15 heap 09:34 heap as a tree 10:20 max and min heaps 16:40 heap operations 19:55 max_heapify example 26:06 max_heapify complexity 30:05 build max-heap 35:05 build_max_heap complexity 48:00 heap-sort

[@변상화-c1x]

thanks

[@ivanreii]

this guy is the fucking boss. I'd never skip a lecture with a professor like that.

[@DarkLordAli95]

even a 8:30 lecture? :P

[@randiaz95]

GreyFace. dude thats the best you dont even have to miss work to go to class

[@mubchamp]

I'm a student of Computer Science studying Data Structures this semester... I want to say that this professor is really a master at what he is doing! Thanks MIT, for this free video recording. It was really very helpful. +1!

[@kapil6562]

He is an Indian

[@marcelosilva6250]

What you doing now?

[@laraibanwar1618]

Checkout Abdul bari algorithms I hope it will be helpful

[@skittles6486]

@@laraibanwar1618 You can't compare Abdul Bari lectures and MIT OCW. Abdul Bari lectures are helpful if your exam is near and you are thinking about getting marks. But OCW is helpful if you want to get into deep.

[@aadimanchekar1032]

@@skittles6486 +inf seriously OCW provides way in depth knowledge

[@komalvenkatesh4527]

For those who are confused with the n/4..n/8, n/16... equation to describe complexity of build heap. Here's why: He's actually going bottom-up, where level 0 is the bottom most level with n/2 nodes (n being the total nodes in the tree). This is contrary to most depiction of a tree where level 0 is the root node with 1 node. So if you think about it just generally, as your work per node increases at each level, the number of nodes at each level decreases by (/2). So, in the mathematical equation you have one component that increases and another component neutralizes by going down in value. That's why the complexity is O(n). In his equation he starts from one level up from the bottom level because there is no work at the bottom most level in the tree since they have no children. So bottom level would have n/2 nodes and the level above that would have n/4 nodes and so on. So he starts the equation from n/4 - since you would start with one swap per node at that level. 2 swaps per node at the level above it. 3 swaps per node at the level above that one and so on. Although, number of swaps per node has gone up every level, the number of nodes have gone down equally to neutralize that. If you're still confused, take pen and paper and draw it out - it will be apparent. Hope that helps.

[@hassanjaber8169]

You're amazing. Thank You!

[@MahmoudGamal-io3hl]

Thank you, kind stranger!

[@MsRP2Bang]

But then in that case, won't the complexity of all tree algos or functions reduce to O(n)? Instead of O(nlogn) ? I am confused. We always consider complexity of operations like insert or search in a tree as log n and nlogn for n items

[@komalvenkatesh4527]

@@MsRP2Bang You've got that wrong. It's not O(nlogn). BST: Algorithm Average Worst case Space O(n) O(n) Search O(log n) O(n) Insert O(log n) O(n) Delete O(log n) O(n) BINARY HEAP: Algorithm Average Worst case Space O(n) O(n) Search O(n) O(n) Insert O(1) O(log n) Delete O(log n) O(log n) Peek O(1) O(1)

[@luizfelipels7]

Couldn't understand where that last term in the summation came from. (k + 1) / 2^k

[@cat_the_vlover]

Great era. I, an ordinary person, can study algorithm through the lecture offered by the top-notch universities for free.

[@TheRageCommenter]

Welcome to MIT, where even our chalkboards are better than everyone else's. Seriously, though, I've never seen a chalkboard so clean and clear.

[@peterkalivas6435]

I haven't had a single CS teacher that speaks English well enough to even remotely explain concepts. I should be paying 40K to KZbin a year instead.

[@shayantalebi2224]

This hits hard

[@gauravxdhingra]

I think that's Indian accent.

[@Naton]

You mean 'only' accent

[@gauravxdhingra]

@@Naton I don't know. I never met one

[@JB-ns6ek]

This comment is just too relatable

[@yuktabagdi8958]

This lecture was literally uploaded 8 years back and shot almost a decade back, yet it feels so timelessly new. I never thought I'd ever like this course till this lecture series. Prof Srini, you are a legitimate BAWSE. Respect++

[@TempleNirvana]

I never gave donation to my own university because the quality of the courses my university offered simply sucked. But watching only one lecture from this course, I decided to donate to MIT on regular basis.

[@kzterminator]

True. With this kind of teaching it deserves the title of a university, others I don't know what they are but they claim themselves to be 'universities'

[@kithenry]

Did u ?😅

[@yangyinjedi]

I think the fact that it is from MIT makes a lot of people think they are getting a better lecture then they would at their school...Not they case...He is teaching completely from the Cormen textbook chapter 6 exact same examples...I think it is a great supplemental lecture...I only understand it after reading the textbook and the first lecture from my prof...I think the fact that we are hearing it for the second time helps a ton and I think MIT a lot for explaining the same topic in a different light and I am now happy to know that my school isn't cheating me any with the curriculum just far less competition

[@strxpl0xmage]

+yingyangjedi I agree with this. I saw the lecture on merge sort, and he used the same examples for the insertion sort chapter that comes before.

[@davidibanez4309]

yingyangjedi thanks for mentioning the book. Sometimes the lecturers forget to tell us the book they use, and they (the books) are usually more straightforward.

[@kzterminator]

Try to take courses in my university. I was exchanged to Canada and have taken abstract algebra there, taught by a native speaker and I get a B+, not a fantastic grade but at least I get most of the material. Then back to my university, during a quantum mechanics course the prof (a mainland China professor) was explaining something related to abstract algebra, I knew the subject he was trying to explain but just can't really follow his (messy) writing and explaining(or mumbling?). I think English should be taken into account when they select professors.

[@happy-monk]

Thanks for pointing to the book

[@Fellintr]

First time a youtube video was actually helpful. No questions remain. Explained everything in a very short time. Great lecture.

[@alexandrugheorghe5610]

25:28 - it should be Exchange A[4] with A[9] since 4 and 8 keys were swapped and not 4 and 2 keys

[@JyotinderSingh]

Haven't yet read Cormen, but what I feel is this is such an amazing lecture, where the Professor gives appropriate amount of time to each and every step of the algorithm and then beautifully explains the math behind all of the stuff. I love this :)

[@antongoy9366]

I wish I would have so lectures and professors in my university. Everything is clear!

[@Squirrelschaser]

This is honestly by far the best video on Max/Min heaps on the internet. His explanation on building max heap was extremely simple and intuitive. The reason why we start at i = n/2 down to 1 is that this ensure the max_heapify assumption is always true. Ingenious really.

[@dhruvmistry6955]

What a Excellent Source of Crystal Clear Knowledge on each and every topics . Hats off to MIT profs ......

[@jackguo4260]

their professor can actually explain things, WOW

[@fighterdse3589]

Really helpful and inspiring lectures. I am so lucky to be born in this era. Online lectures are brilliant idea!

[@melvin6228]

I hope this comment is helpful for people who are taking this for the first time. I'm reviewing this course since I took a similar one on my uni 6 years ago and back then I hated it. Now, I really like it, but that's because I vaguely know what to expect. If you feel a bit shaky on understanding sums (like me :D), then I think it's a good rule of thumb to understand the following ideas. I understood these ideas beforehand (by chance) and they helped me immediately understand everything he did, regarding the sums. IDEA 1 SIGMA(n^2) is between 1/3n^3 to n^3. This means that most to any diverging sums of the form SIGMA(n^2) have O(n^3) as their answer. Explanation lower bound: a sum is basically a blocky form of integration, so you can use calculus to get the lower bound. Integrating n^2 is 1/3n^3. Integration works on continuous numbers and not discrete numbers, so the actual answer is a bit different. Here is how: since the actual answer always involves extra additive terms (of the form an^3 + bn^2 + cn) and we don't have those, the lower bound is 1/3n^3. Explanation upperbound: the upperbound is n^3, because if you take the sum of n^2 and always do n, as opposed from i to n, then you get with n = 3 (for example) 3^2 + 3^2 + 3^2 = 9 + 9 + 9 = 27 = 3^3. Source: I thought a lot about sums in the shower, because I realized they are quite key to complexity analysis and they aren't really clear cut. I also read Knuth's book about sums (chapter 2, p. 21 to 66) and he showed to me how discrete sums are basically a variant of integration. His book gave me the idea that using normal ideas about calculus are an approximation for the much more complex calculus he presented in his book -- the calculus of finite differences. The same trick works for SIGMA(n). IDEA 2 Another thing that one needs to understand is to understand series of the form 1/2 + 1/4 + 1/8 + ... 1/n, such a sum wil always be the first time plus the first time, so in this case 1/2 + 1/2. See a numberphile video on it here: kzbin.info/www/bejne/q2i9aoikjLR9hLs --- Another thing I noticed is that he has two modes of analyzing time complexity. (1) he goes line by line and (2) he does some form of summation by looking at the data structure. He did this in lecture 3 as well, but then visually. I was able to immediately get to the answer of O(n) because I visualized the operations done on the data structure, as opposed to analyzing line by line. Because of this, I have the following strategy to solve these big O questions: 1. See if I can solve the question by visualizing the data structure and what operations are done on it and sum it. 2. If I'm not able to, then count line by line, with the potential of me being wrong.

[@arnobchowdhury9641]

Thanks a lot. Though I did not understand a word about your first idea, your second idea was very helpful for me.

[@xBl00dBrothersX]

I like this professor.

[@sophiacai8157]

I prefer the other guy. It's mostly just a matter of accent though XD

[@kishantiwari3221]

Gratitude to the mit due to them i can study these lectures for free.

[@J151366]

One of the best CS open course in the world

[@alxactly]

This is explained so much better than my professor did or ever would. Thank you for these video series.

[@encryptionalgorithm]

Great and very passionate lecturer. But mind the error @25:35 "Exchange A[4] with A[8]" should be with A[9] instead as we are referring, between brackets, to indices of each node in the tree not to their values, so don't be confused.

[@jz1838]

+encryptionalgorithm I got it too, HAHA

[@encryptionalgorithm]

+Z UU Good that you spot it as well ;)

[@akshayrawat5058]

+encryptionalgorithm thats right...this also proves student are half asleep in the class :D

[@ziddy26]

+encryptionalgorithm Thanks!..And nobody in the class cared to correct him. But a good lecture tho

[@encryptionalgorithm]

+ziddy26 You are welcome! His way of teaching is so amazing that makes me feel that the students were "possessed" when listening to him, which I can understand :)

[@jayquelin]

32:47 if you watch it at 2x speed it looks like that student is casting a spell on the professor lol

[@animeshsingh1307]

😂😂😂😂

[@kal9001]

"Abra cadabra, you must give me a pillow"

[@pranaysharma2055]

lol

[@HarshPatel-iy5qe]

Harry potter though 😂 😂

[@pubgplayer1720]

MIT is secretly Hogwarts lol 😂

[@aoihikpramanik3593]

Thank you, Prof. Srini Devadas and MIT!

[@codesefod8527]

Some points which I would like to mention, if you guys are coding merge sort then make sure: 1. While building the max heap from unordered array , start from (n/2) - 1 (because index starts from 0) till 0. 2. Make sure you reduce the heap size after swapping first and last element.

[@codesefod8527]

@@gabrielkennethmarinas6244 sir that way 2 years ago, thank you for helping us out.

[@hzzyyy]

@44:05 the sum of the this series is 4, rearrange the sum as a upside down pyramid, the first layer as (1/2^0+1/2^1+...) the second layer (1/2^1+1/2^2+...), so 1/2^0 is summed once, 1/2^1 twice, etc. The we got the sum of the first layer 2, the second layer 1/2*2, etc. Finally, the sum is 2* (1/2^0+1/2^1+..) which is 4. We can do the rerrange because this series is absolutely convergent as you can verify by ratio test.

[@NarthanaEpa]

For the less geometrically inclined, it is the MacLaurin series of 1/(1-x)^2 evaluated at x = 1/2. The lectures assume it is the geometric series, ie the MacLaurin series of 1/(1-x), but it is actually the derivative of that.

[@tungthanhle6797]

In MAX_HEAPIFY operation (time 25:35), the step after calling MAX_HEAPIFY(A,4) should be like Exchange A[4] with A[9].

[@anujkhare3815]

This teacher is so good at explaining concepts .

[@tonyzhang2501]

42:20 the last term should be 1 ((lg n - 1)c) ------ Total amount of work in the for loop: N/4 (1c) + n/8 (2c) + n/16 (3c) + ... + 1 ((lg n - 1)c)

[@mohammadtorabipour5843]

exactly

[@junzhai1715]

nope. if n = 1, the last term should be 0. in your case it is negative, i.e lg1 -1 = -1. Actually he just missed one last term which is (k+2)/2^(k+1) after he defines n/4=2^k. You can have a pen to verify.

[@synthguy0428]

the most excellent explanation of heap I have ever seen.

[@Bonvivant9]

THAT is how you teach!

[@sob5520]

32:40 that kid should drop out of MIT and become an orchestra conductor

[@abaundwal]

You're not as funny as you think.

[@진희곤-n6n]

Anurag Baundwal You’re too serious than you think.

[@maheshg.n.9688]

Awesomely explained. I would have 100 % attendance with this kind of lectures

[@danieltorresdeluna4844]

Qué curiosa por ustedes explora✌😂

[@guowanqi9004]

30:58 I'm gona write pseudocode for build maxHeap, cos it's 2 lines of code. That's about the limit of a program I can understand. LOL

[@alexandervanvalin9522]

The professor's striped shirts' kung fu is stronger than my graphics card's kung fu.

[@malino24-souls]

XD

[@AbhishekAgrawal86]

Aliasing

[@davidnichol4735]

It's a kung fu that stood the tests of time...

[@JMROMERO95]

I'm so grateful MIT, SO GRATEFUL! :') Thank you so much for this!

[@lewisdiamond1]

I'm impressed by the chalks and blackboards.

[@vanduyducss]

Yes, It's so real, raw and deep. I like it too!

[@mirjanadivcic7613]

Thank you MIT and a big thank you to the professor.

[@anonymoose_yt]

In case anyone is wondering,@25:30 he should've written "Exchange A[4] with A[9]"

[@SuperSonic68]

About half way through the video I realized that this course uses the same book as my course, making this lecture series all the more helpful!! Thank you so much!!!

[@michaelruffalo5796]

Which book?

[@whyarewehere9555]

beauty of this lecture is that all the example are taken from the CLRS ......

[@satyajitbiswas4945]

I need to say one thing that in real scenario most of the language supports array indexing from index 0. If it is taken from 1 then there forms an error in the parent(i). Then i/2 should have been replaced with (i-1)/2 and the left child = 2*i + 1 and right child = 2*i + 2

[@seanmcelroy4074]

"The Pseudocode is in the notes". Me 10 years late halfway across the world checking the notes I've been taking

[@nik7426]

you can find the notes and other material on the site mentioned in the description.

[@HarvinderSandhuEsq]

Excellent lecturer, I love algorithms and data structures! :)

[@websoftwaredeveloperijtiha3093]

It's so cool that these top schools release courses like this one online free of charge. I may not get a chance to go to MIT

[@psw1]

Prof. resembles Rahul Dravid (Indian Cricketer).... Same smile and laughter too. Two greats!.

[@cheng3689]

to prove the expression is bounded. Let S = 1 + 2/2 + 3/(2^2) + ... + (k+1)/(2^k), S = 1 + (1+1)/2 + (1+2)/(2^2) + ... + (1+k)/(2^k) = 1+1/2+1/(2^2)+...+1/(2^k) + 1/2(S), when k--> positive infinite. S = 2+1/2S, S = 4 when k --> positive infinite. S is bounded by 4.

[@Anveshana837]

Thanks M.I.T and professor for providing this for free.

[@linhng7731]

my prof only explain from slides. i prefer this style of teaching - board and chalk. he explains and writes at the same time, so much easier to understand!!!

[@GTheDestroyer123]

thank you for replying an maintaining the channel. I learned alot.

[@arielgonzalez3223]

God bless this free lectures!

[@beypazariofficial]

these are really helping. definitely better than my algorithms and programming lecturer

[@ashutoshtiwari4398]

At 23:43, child node 4 and 5 have value 14 and 7 while parent node 2 have value 4. Now the important note that professor missed to tell is: Suppose, the child node would be 7 and 14 (i.e. interchange left and right node) then replace parent node with right child node because it was the largest among the two child node.

[@pinkfloydc8457]

He did mention that we take the max of child nodes and swap with parent node. So, this would have been redundant

[@MahmoudGamal-io3hl]

Heapify complexity analysis starts at 37:20

[@piyushslayer]

Only 1 or two of them are good, I prefer MIT lectures over IIT ones any day. They're to the point and don't waste time revolving around the topic and take forever to get to the original point which is the case with most IIT professors..

[@spartanelectronics]

It gives me goosebumps. ... whenever i see indians at such place

[@tabularasa0606]

Heaps are so elegant.

[@sumantasamal6761]

Thanks MIT for this lecture.

[@kapil6562]

Good to see one among many Indians there in MIT.

[@rj-nj3uk]

He is teaching foriegn students when Indian students needs him. He is good for nothing for Indians not in MIT. Thank Opencourseware to make this vid available.

[@Lethcode]

I noticed he is mixing print with cursive on the board now I can't stop looking for it. Its amazingly easy to read his writing but I think it is interesting I didn't notice it until lecture 4.

[@gedhayachandran3786]

Finally, I understood heaps and heap sort. :D

[@PashaSemf]

Heap Sort @30:07

[@MegaVuhung]

Thks MIT for this great serie of lecture! Really appriciated

[@SudhanshuKumar-lp1nr]

Heap sort starts from 30:00

[@pumpmanagementsoftware]

this professor is just superb

[@andy0401ify]

I can't believe I ended up watching the whole video....lol....so clear

[@aayushikhandelwal6669]

Watch 30:00-47:00 if you want to know how build max heap takes O(n) time instead of O(log n)

[@KageKevSama]

Yeah he teaches the same stuff but he does lectures a whole lot better than the professor that I had

[@weskerDluffy]

Con razón son los mejores, q calidad de clase

[@rayhanmahmud6579]

I think there is a small mistake in 25:36 ! Instead of A[ 8 ] it would be A[ 9 ] ! but it was a great lecture.......!

[@weforte4912]

+Rayhan Mahmud Yep it's a mistake But great work by MIT ... :)

[@VisionOfTJ]

Guy is a pro. Thanks very instructive.

[@AdityaMishra-cj7kt]

Awesome Lecture, Loved it!!

[@robertrahardja]

This is so good ... so recommended

[@nicoqueijo]

Why does MIT start indices at 1 lol?

[@billyhypertext6068]

That's so a binary tree can be constructed using the following: Left child: i * 2 Right child: (i * 2) + 1 Parent: Math.floor(i / 2) For example, from this array: // 0 1 2 3 4 5 6 7 8 9 [null, 'A', 'C', 'B', 'D', 'F', 'G', 'J', 'H', 'K'] We can derive the following tree: (A) / \ (C) (B) / \ / \ (D) (F) (G) (J) / \ (H) (K)

[@Doubleagentaron]

Or they just love that pascal

[@DebasishDas-bi4bo]

That's because they follow the book: "Introduction to Algorithms" by Cormen. The book uses indices that start from 1

[@euiyoungchung8492]

Dudes.. only the first gave the right answer. If the root index is 0, the indices of its children are 0 * 2 = 0 or 0 * 2 + 1 = 1. yeah. there occurs a corner case.

[@DebasishDas-bi4bo]

@@euiyoungchung8492 What he asked was why they use indices that start from 1 (not just this video). If you look at previous videos, Insertion sort, Merge Sort..etc, all use indices that start from 1. The reason is simple - They follow the book "Introduction to Algorithms" (as mentioned in the course's website), which follows this convention

[@lockersrandom6161]

Thank you MIT.

[@darkovidakovic3518]

There are n/2 nodes with level 0 - the leaves on the 1st level there are half the nodes than on the lower level (0), or 1/2 * n/2 = n/4

[@stevemathew8634]

He came to class with just a couple of sheets as teaching material, but carried a whole bag of cushions XD

[@MohammadrezaMemarian]

MAXHeapify is at 22:13

[@prajwalbhati2101]

Beautiful Lecture

[@sophiaman2208]

can't believe that i watched a full lecture.

[@badassopenpolling]

52+ minutes video for Heap Sort !!!! #mitocw should have released module wise.

[@shawnz9833]

starts at 6:30

[@shawnz9833]

max-heapify @ 20:50

[@KageKevSama]

What are those cushion things that he hands out to students that participate? Also, excellent lecture. MIT's professors seem to be about 10000x better than mine. Really appreciate MIT putting this up. Thank you!

[@Raylistic]

Very good lecture, prof! Thanks to MIT for posting this!

[@chengweili9516]

This prof is awesome! Great explanations!

[@japananh1]

Does anyone notice the the teacher made a mistake at 25:36? Exchange A[4] with A[9], not A[8].

[@MexterO123]

Thanks MIT, for helping me out with heaps!

[@tannerbarcelos6880]

If my DS/Algo class in school was this good, I think I’d have two internships by now lmao. Unfortunately, I was stuck with someone who you can barely understand and that didn’t have passion in the work. In the end I’m graduating but that class is so fundamental. This lecture, and the series itself is beautiful. Albeit , not there in person to retain even more and have the exams / homework in real time, learning is exponentially better here than at my school, lol

[@hyunjunkim7069]

Love this lecture, so helpful.

[@junzhai1715]

25:40 , A[4] should be exchanged with A[9] , not A[8]. 45:00 , he just missed one last term which is (k+2)/2^(k+1) after he defines n/4=2^k.

[@scottbrusnahan2083]

The lectures can be reduced to 30min just by replacing the board and chalk with a powerpoint presentation. This can save a lot of precious time for students and the professors.

[@pranavnyavanandi9710]

Exactly man.

[@purnimamishra3819]

Really a wonderful explain 😊

[@samoldfield9997]

Thank you sir for your great explanation

[@hritikbangar4341]

exceptional teaching

[@SK10300]

next day exam now i watching & leaning think you so much so helpfull

[@tr233]

Maybe its me , but i fell asleep during this lecture like 2 times, it's theoretical, need more example. Well i finally got heapify, the secret was to do some exercises , now i understand finally :)

[@sophiacai8157]

Usually, when I watch stuff like this, I concentrate 5 minutes, my mind wanders off, I refocus after some time, rewatch everything I missed, repeat...Basically I watch it concentrating in 5 minute periods XD

[@liang3163]

7:20, Sorry, I'd say "Heap is an array visualized as a nearly full binary tree", here the concept the professor used of "complete binary tree" is actually "full binary tree" by convention. Except that, wonderful course!

[@liang3163]

Because heap is already a complete binary tree, just not a full binary tree. So we should remove "nearly" for professor's words "nearly complete binary tree".