Statistics 101: Nonlinear Regression, The Very Basics

In this Statistics 101 video, we learn about the fundamentals of nonlinear regression. To support the channel and signup for your FREE trial to The Great Courses Plus visit here: ow.ly/xVD030fiZ8S
My playlist table of contents, Video Companion Guide PDF documents, and file downloads can be found on my website: www.bcfoltz.com
Happy learning!
The Great Courses Plus is currently available to watch through a web browser to almost anyone in the world and optimized for the US, UK, and Australian markets. The Great Courses Plus is currently working to both optimize the product globally and accept credit card payments globally.
#statistics #regression #machinelearning

[@sat8716]

Totally misleading. You should correct this video. Many of us are getting wrong information. This is not NONLIN EAR REGRESSION.

[@BrandonFoltz]

To the extent the video is "misleading" it is due to the various meanings ascribed to "nonlinear" in statistics texts and other materials. The video is literally based on problems from books sections titled _Modeling Nonlinear Relationships_ . Nonlinear can mean "not a straight line" OR linear combination of parameters OR non-linear parameters depending on what you are reading. This video uses the first two implementations of the word as it appears in into stats books.

[@sat8716]

@@BrandonFoltz Then mention that non linearity in regressors. In general Nonlinear regression means non linearity in parameters.

[@marcelopinheiro1219]

@@BrandonFoltz Nonlinear Regression is a Nonlinear Regression. Nonlinear Regression is not a Polinomial Regression. This is totally misleading.

[@BrandonFoltz]

@@marcelopinheiro1219 I have already explained this above. If that doesn't suffice then nothing else more I can add.

[@ElinaGoroshkova]

Thanks god we have youtube and I can know how looks like the best teacher. Really thank you for all your videos!

[@parrw0rdable]

Hey Brandon. Just a Big Thumbs up for the great teaching. I am following your channel rigorously and wait for every new video.Your work and knowledge is just awesome. Please keep up this good work and keep teaching us. I think I can't thank you enough ever for your great videos. Cheers to you.

[@BrandonFoltz]

+Rahul Jain thanks so much! A labor of love. Thank you for making the world a better place by committing to learning!

[@alevyildiz239]

I can't thank you enough for these videos. Extremely helpful and you make it easier to understand. You are an awesome teacher!

[@miroslavtratniik7110]

Hi Brandon. I just recently came across your commendable video presentation. Many things, which caused duality in me, became clear to me through your presentations. If there is a proper appreciation for your selflessness and generosity towards the less fortunate, among people in today's world, I would award it to you - let's call it the Oscar of Scientific Selflessness. Honestly and from the heart.

[@BrandonFoltz]

Your time is the most valuable thing in existence. Some of which you spent with me. There is nothing more I could ever ask for. Just keep learning. And pay it forward when you can. Thank you 🙏

Dude, your videos make so easy to understand stuff that takes pages and pages in books...

[@lyndaromana4784]

Your videos make stats seem like a breeze!! Thanks for all the help 😍

[@xRennieBunny]

This saved my bachelor's thesis, thank you so much T^T I really love how patient and thorough you are with explaining the details

[@beemdude2]

Superb stuff man ! very clear and crisp, cheers

[@smithjonh7261]

great channel, thank you

[@jannfietearp4439]

can't believe I understand more in 20 minutes than the 2 days + tons of google pages

[@ylazerson]

Fantastic video!

[@acy9901234]

Thank you for another Great video!!!

[@nawilliam2754]

Another Great Explanation !!

[@Diegoblismartmedpengar]

Wow thanks! this was exactly what I needed (Y)

[@user-lt5ne1ff1w]

Also a student from Taiwan. You taught way much better than my professor in college!!!

[@raadhashim9221]

You are amazing. Thank you so much

[@ashishchaurasia3912]

Hi Brandon. Thanks a ton for such insightful videos. They are just superb. Many concepts have got clarified. Have a query . Do we need to always test every linear regression for non linear regression to know whether non linear fits better. Do we always need to evaluate both ? Pls if you can help me with this. Again thanks in advance.

[@mkilptrick]

He's back! Yes!

[@camiloverts]

Thanks!!

[@samf680]

really good video... a nice review for me also get more insight

[@williamespinoza1540]

Is this correct? I thought it was the parameters that determined if a model was considered linear or nonlinear. For example: Linear: Y = b0 + b1x Y = b0 + b1x + b2x^2 Nonlinear: Y = b0 *x^b1 I believe linear regression includes the quadratic regression you have in this example. Even though the x term is squared, the model is still linear with respect to its parameters.

[@vaibhavchittora1579]

Yeah I Agree. If the model linear it is suppose to be linear with respect to parameters only. If model is not linear with respect to parameters then only we can consider it non linear model otherwise it would be linear model.

[@prasenjitbose123]

Yeah, the video is misleading!

[@TheBjjninja]

Yes you are 100% correct. Additive form means the model is still linear. We are actually just improving a linear model by controlling for quadratic relationship. His wording is a bit off but certainly still a useful video to improve the fit of linear regression model.

[@TheBjjninja]

This is actually considered the study of curvilinear relationship in a linear model but it is still a linear model.

[@xiaohaoyi]

Very good video! It's so much easier to understand you than the lecturer :D

[@manzoorahmad-mu3xv]

Your videos are very helpful, could you please help us understanding models such as, Vector auto regressive model, white noise model, error correction model etc?

[@simone9610]

Brandon i studied statistics and i took a pretty good grade. But the problem at my or most of universities is that you probably learn how to pass the exam but you don't know how to apply them or most important what do these numbers, results mean to you.... I m following econometrics and I found out that many concepts and statistics topics i almost just know the name ! But thank to you, i m on the road haha ! Keep making videos and let us know how can we suppor u.

[@robinhoman8594]

YOU ARE THE BEST. I LOVE WATCHING YOUR VIDEOS! ty for all of your work.

[@Dustyone23]

Being honest, I have never had someone explain these statistical models as well as you do. Have you considered making videos for categorical data analysis techniques?

[@wenyange2607]

Thank you, Brandon. It really helps a lot to understand the essence of the regression model. I have 2 questions, can you help me? 1st question is that x1 and x1 squared are highly correlated in nature and surely have a variance inflation factor higher than 10. I wondered if this would be a problem and searched on the web, but could not find the answer. 2nd question is that quadratic model (-0.00185x^{2}+1.4094x+63.85) will reach its peak on about x=380 and y=332. After x > 380, y, the cars sold will decrease. Usually, this is counter-intuitive, how can we fix the model so it remains that y will always increase when x increases.

[@lucycai3356]

From what I read in Wiki for linear regression, the quadratic example is a linear model, because it is a linear combination of parameters. Who can tell me where the problem is ? Thanks!

[@andreaskunz4701]

I also got a similar question: To me this also looked like if the coefficients of the linear model were simply transformed. The variables are quadratic but the function is still linear?! Is this correct? Where is the line drawn between non-linear function and transformed parameters?

[@Ajay-xd7zq]

Thanks Brandon, Your Stats videos are very helpful. Thanks a ton for all your help. However i had a question regarding non-linear regression approach here... - In a non linear model, the parameters b0, b1, b2 etc.. they should be non-linear right, and not x0, x1, x2 etc,,, - A non linear model is , combination of non-linear parameters b0, b1, b2 etc.. and NOT the non-linear combination of x0, x1, x2 right? Something like y = e^b0 + Sin(b1)x1 + e^b2 x2

[@rahulsingh7508]

Hi Brandon! In one of the regression problems that I am solving, the independent variable has non-ordinal categorical values (or integral values) and the dependent variable has contiguous values. The correlation between the variables is 0.73. After creating a scatter plot, I observed the dependent variable values increase as independent variable values increase. But for a particular IV value, there are several DV values. In such a case, what kind of a regression model should I build?

[@st093076]

Thank you a lot~~~~ I am a student from Taiwan(台灣 It's my second semester(the last semester) to learn Statistics, and I am preparing my final exam now! Thank you for your good videos~~~ I will recommend them to all my friends who need to learn Statistics~~ 嗨，Brandon~ 我是一個來自台灣的大二學生，這是我最後一學期修統計學，我現在正在準備期末考哈哈 非常非常感謝您精彩又清楚明瞭的影片 我明年一定會推薦給所有要修統計學的學弟妹!!

[@shwetasingh2389]

Thanks alot

[@user-rw9hc2ox8y]

Amazing video, thanks! One question: Can you also have: y=a * x^2 + c or is it always: y=a * x^2 + b * x + c ?

[@panagiotisgoulas8539]

Brandon I don't understand something. Besides the 0th week which makes sense how can you get a general method that the intercept will pose a problem? In a way it makes sense since I see most job/total be like 1:2 or 2:1 so the low predicted values wouldn't even make sense. But is there a general method that you came up with? Also do you by any chance have any video uploaded on how to deal with outliers (statistically and or software)? Thanks so much

[@AN-kb4kh]

Great video, best explanation on nonlinear regression I've seen! Will you be doing a video on nonlinear regression with more than one predictor variable?

[@davidboozer2410]

Great video! I learned a lot from it. One thing to think about, however, is how well will your model fit future data? For example, with a quadratic model, what goes up must go down... so, according to the model, the more experienced salespeople will start to see declining sales, with each week being more worse than the last. Perhaps that makes sense in this context, but modeling is more than just reducing residuals. Of course this example was for educational and introductory concepts, but I feel it's still worth mentioning. I'll be watching your future videos. Thanks!

[@kevinchetti2603]

You could remodel every month ... You dont just deploy the model once and for all with an initial train / test ...

[@dboozer4]

@@kevinchetti2603 Excellent point!

[@mustafa6455]

thank you very much for your effort in explaining, I watched all of your videos on ANOVAs, I have a question though, whats (the fit model for an appropriate light output ??? )I'm studying full factorial design and came cross this subject.

[@mustafa6455]

any thoughts?

[@LeonardSHong]

Hi there, How did you add the equation created from the regression model into the Charts? I'm trying to do that on Excel but it is really difficult. Cheers.

[@marcop7891]

First of all you are doing a great job and your videos get to the point easily and precisely! But in this case, as some have pointed out, the regression is still linear in the parameters...for it to be nonlinear shouldn’t it be something like y'=b0+bx_1+b^2x_1+…+b^nx_1?

[@dboozer4]

You are on the right track, but b² can still be interpreted as a constant once the fitting is done. Nonlinear regression would be something like Y=a sin(bx +c) +d, where the parameters are not a linear combination of various functions of x.

[@ccuuttww]

I suggest feature selection before consider nonlinear regression if u find a feature domaint the model go a head But i find most of the large data set model usually fit the linear one U can try something like cos log but this is not that significiant to the model

[@tonycardinal413]

Thank you so much for posting this. Ques: If there is a high covariance between the parameters (coefficients) what does this tell us? Is this bad? If so why? thanx!

[@salrite]

Is there a video on Residuals, what is Residual and what it signifies?

[@ys6285]

Isn't polynomial regression a type of linear regression? It assumes a non-linear model, but I heard it is a linear model in terms of there parameters (parameter vector Beta or weight vector w)

[@dboozer4]

Yes, because the parameters are a linear combination of x, even though x is being operated on. In this problem, we know all the x and y values, we just don't know a,b,c,... once you sub in the x and y values, you have a system of linear equations that you can then solve with linear algebra.

[@furqanadeel1182]

Awesome

[@vannanuon6077]

Thanks, Brandon for your clear explanation and really help to understand the concept of this model. May I ask you as the following? I do a small research and using nonlinear regression to see the relationship between fish catch (kg) and year (from 2007 to 2018). The output of the regression has an upward trend with R^2=0.72 and P>0.05. So, can I interpret that my fish catch increase over the survey period even if my P>0.05? Thanks in advance and I hope you can help me since I spend a few days to find this answer and could not get the answer.

[@youngcobra2011]

You should try to fit a different model to get a better P-value. I do the same kind of work modeling weight-length relationship of fish.

[@PinkFloydTheDarkSide]

I strongly doubt if it is correct. At 10:32, you showed an X value squared and called it a non-linear scenario. However, it is not the X values that define the linear or non-linear case, it is the value of parameters b0, b1, b2 that defines the non-linear case.

[@luvju85]

Exactly...its still linear in parameters hence linear..@ Brandon - please correct the video

[@JackIsNotInTheBox]

Yeah, linear regression is "linear" with respect to the parameters/coefficients, not the independent variables. This is a good tutorial for polynomial regression, a special case of multiple linear regression, but it should not be titled non-linear regression. I suggest the title be changed to "Non-Linear Data" instead of "Non-Linear Regression".

[@samf680]

Hello Brandon, thanks for informative videos. Do you have any video regarding the scores (like: R-square, MSE, MAE, Residual etc) and what is the meaning of each of them? and how we can analyze the result of our regression model properly? thanks

[@BrandonFoltz]

In my simple linear regression and multiple regression playlists we go over all those many, many times. :)

[@Maymona93]

That was 👌 one question, p value for weeks on the job coefficient is 5 not 0.0005 therefore not significant????

[@jennscottnicholson8300]

Hello,please can you explain me the difference between the slides at 17:52 versus 18:19? On the first slide, the predicted values are the same as the model values. But on the second slide, the predicted values are different from the model values? Why? And what are the equations of the individual curves? Many thanks.

[@panagiotisgoulas8539]

I have same issue plus I don't understand how he generated the polynomial cars sold in both charts

[@dboozer4]

Do you have a video or guidance on how you generated the ANOVA tables in this video?

[@cw1428]

Sir, so quadratic regression as well as polynomial regression are considered as nonlinear regression, although they're in the form of y = a + bx?

[@Dr_Finbar]

When would choose a quadratic model like this, instead of transforming one, or both variables? or vice versa thanks :)

[@awfan221]

So if we are working on a software, do you recommend that we just run all the relevant models and see which one gives the best r squared? Or do you recommend that we plot out the residuals on the software after attempting linear regression?

[@BrandonFoltz]

I would say just look at the data first visually. If it is obviously curvilinear then skip simple linear. From there it is a balance between a flexible model and one that overfits. Try different models, look at how the R-square changes, and the residuals. :)

[@awfan221]

@@BrandonFoltz Alright, I'll do the eye test first, thank you. I use STATA so it should be easy to compare by scrolling down and looking at the outputs of each model

[@ranjanibhat4417]

Sir is constants necessary?? Can i remove the constants because my p value is greater than 0.05

[@vkak1]

You cannot get rid of constants. If you are getting pvalues over 0.05, maybe adjust your threshold to 0.1 it accept the bull hypothesis.

[@kamiloweluckypanczoweblend1957]

YO ARE SUPER KOX

[@MuhammedShiharMZaid]

Quadratic model looks cuter than the linear model.

[@Furiac.]

I understand most of everything, but it is never mentioned how you actually plot the line and find the regression...

[@FalakVats]

This is not non-linear regression, its linear or non-linear depends on parameters not on the variables. Its a Linear regression too

[@wilsonlwtan3975]

Only if colleague professors were as good ..

[@petersaucier8214]

VERY SAD!!! This video is a malpractice of statistical teaching. Contrary to the video, exponential functions are linear models. A model is nonlinear when its parameters are nonlinear.

[@pedroalonso45]

Exactly, this is a video on variables transformation, not non-linear models.

[@FalakVats]

True