Optical Flow : Horn and Schunck

[@utkarshkathuria2931]

at 4:49 in the graph, the point on the u-axis should be -ft/fx?

[@pratikian]

Yes you are right, I made a mistake thanks for pointing it out

[@utkarshkathuria2931]

@@pratikian I am not going deep into the calculations but I just want to ask that the values of fx,fy,ft in the horn and schunk paper have 1/4(....) and what you explained is have to take avg. So how does it become 1/4.

[@pratikian]

@@utkarshkathuria2931 Good observation, so its like this in the paper if you see carefully so there are 4 important terms say Ex = 1/4( [ E(i,j+1,k) - E(i,j,k) ] + [ E(i+1,j+1,k) - E(i+1,j,k) ] + [ E(i,j+1,k+1) - E(i,j,k+1) ] + [ E(i,j+1,k+1) - E(i+1,j+1,k+1) ] ) Since there are these 4 terms the average will have divided by 4. Now normally in Image Processing we use masks to carry out such operations (you can see in slide no. 12), and the scale of the mask defines the scale of the output image if I keep a mask of 1/2*[ -1 1;-1 1] for fx I will get the exact same equation. But since fx and fy are not actually an image but a gradient of the image hence there is no constraint on the scale. Since the same mask is applied on the whole image visually there is no difference. But the difference will be in the absolute u and v values. u with my method will be the double of u with the papers method. hope that helps

[@harshpatil7684]

Thanks! What a wonderful series of videos. And what a wonderful job you have done to explain concepts from the basics. Given how optical flow is pervasive in so many modern applications I am delighted to have stumbled across your channel. Keep up the great work

[@pratikian]

Thank you so much for the kind words. 😇😇

[@sujandhali2483]

Hello pratik I guess you explained the part 21:04 is wrong if you just chain rule u will find the result

[@pratikian]

I don't understand. Could you email me what do you mean.

[@kulbhushansingh1101]

sir, a question: if we do two things 1.finding laplacian of an Image 2. finding avg of the image and subracting it with its original image will these operation give same result (got this question at 23:40)

[@arkabiswas7842]

Hi sir, I am actually very new to this subject. Could you mention how we could find the fx, fy and ft values in the constraint equation?

[@pratikian]

If you look at the slide titled masks, you will see the equations to calculate fx fy and ft. Basically we take the difference of pixels in x direction for fx, difference in y direction for fy and difference in time for ft. Hope this makes it clear

[@arkabiswas7842]

@@pratikian Thank you sir 🙏

[@savagelysavage4187]

Terrific bro it was really helpful for me

[@panktifadia4555]

You have explained very well👍👍 thanks a lots it was very helpful. Can you please make a video for luckas kanade algorithm? Also what is the difference between both of this alogrithm?

[@pratikian]

Thank you for the feedback. I have uploaded a video on lucas kanade you can see it here kzbin.info/www/bejne/sIqycptur5Z7jqc I have told some of the difference between the two in the video

[@ankitranjan8363]

@24:20 pls explain

[@pratikian]

So there I have told how can we compactly write the laplacian operation Say I have a image 1 2 3 4 5 6 7 8 9 Now say I want to apply the laplacian mask for the centre pixel, in this case 5 So here When you put the given laplacian mask on the centre pixel 5 the output of the filter will be as follows Output = 1×1/12 + 2×1/6 + 3×1/12 + 4×1/6 + (-1)×5 + 6×1/6 + 7×1/12 + 8×1/6 + 9×1/12 Now we can just right the above eqn as Output = Uavg - U Here U is the middle pixel because at the output will be stored at this position Uavg = 1×1/12 + 2×1/6 + 3×1/12 + 4×1/6 + 6×1/6 + 7×1/12 + 8×1/6 + 9×1/12

[@mxcube2812]

​@@pratikian Hi sir, Thanks for you detail and wonderful explaination. I have a question about Uavg in iteration process. @24:00 If I want to get the finanly u on your case of 5, Is that means I have to firstly get all the pixels' u, then to calculate refined u of 5 with Uavg by iteration process?

[@pratikian]

@@mxcube2812 yes that is correct.

[@sandeepsj9572]

great explanations. Thanks a lot

[@carlosvazquez1835]

Why would we need to change the laplacian mask used by Horn and Schunck in their paper.

[@pratikian]

Sorry, I did not get the question, I have used the same laplacian mask as given in the paper

[@carlosvazquez1835]

​@@pratikian Are there situations where we may need to use a different Laplacian mask? Here you where using the mask obtained from paper and I wanted to ask if its possible to use a different mask? and why would we need to change the mask?

[@pratikian]

So the mask tells us about the weights that we give to the neighbours the weighted averge of these neighbours is stored in Uavg and Vavg . Now if you see the weights the 4 neighbours are having more weight (1/6) and the digonal neighbours are having relatively less weight (1/12) this tells us that we are considerding the smoothness for the 4 neighbours to be more dominant than the diagonal neighbours, if in some application you want to consider equal weights for all the neigbours ( say when you know that the images square boxes moving around you can prefer an equal weightage to all ) but it wont affect much. You could also change the size of the laplacian mask, here it is 3x3 you could consider a 5x5 or 11x11 mask which would mean that you are strengthening the smoothness over a larger region so in this case if there are larger objects it would give better results with a trade off at the edges of the object

[@carlosvazquez1835]

@@pratikianOh ok this is why they have larger masks for Gaussian filter and smaller mask for sobel or canny edge dectection ?

[@pratikian]

@@carlosvazquez1835 so acctually sobel and prewitt are quite traditional operators i guess images at that time used to be small hence they are 3x3 but you can define a larger sobel mas as well. Gaussians are mainly used for blurring so the amount of blur depends on the size of the gaussian mask. And canny edge detector is a little complex it uses what is called as difference of gaussians

[@kulbhushansingh1101]

respect++

[@zptang]

Thanks so much!