Python 🐍 Nonlinear Regression Curve Fit

The Scipy curve_fit function determines four unknown coefficients to minimize the difference between predicted and measured heart rate. Pandas is used to import and view the data. A Matplotlib plot is used to see how the initial guess values and the final optimized parameter values compare for generating predictions. An R-squared value is calculated to show how well the predictions and measurements align.
Course Material: apmonitor.com/che263/index.ph...
Github Repository (see HW04.ipynb): github.com/APMonitor/learn_py...
Schedule: apmonitor.com/che263/index.ph...
Solution Videos: apmonitor.com/che263/index.ph...

[@alibastami8626]

Just the fact that you provide this invaluable content for free, when others are selling low quality courses for a lot of money, proves how a great human being you are. You are a true Master.

[@apm]

Thanks Ali!

[@arcesarino]

Perfect! Thank you so much for the great video.

[@SuperCamProductions]

This was exactly what I needed, thank you!

[@arushijain4457]

Super! So easily explained, thank you.

[@OnlyfansCustomerSupport]

thanks for the help this was a long long journey it took me like 3-5 hours to translate my german function name in its actual english equivalent since the direct translation is miles miles away :D and finaly i got a model that should help me solve my enzyme activity measures

[@DFS370]

thank u X 10^10 . U r a great professional and teacher !

[@gomezest]

This is really good content! Thank you very much for sharing your knowledge. Im still blown away by the quality and how you explained the material

[@apm]

Glad it was helpful!

[@asifraj321]

Very helpful and valuable too. Thank you.

[@eaglepaul87]

Oh really useful!! Thanks a lot

[@datastako156]

thank you for sharing!

[@Thahid]

Excellent video

[@bastianian2939]

cool video!

[@badinhbk]

Thank you very much. Your channel is very helpful for me :)

[@rrc]

Thanks!

[@lukas-santopuglisi668]

thanks a lot ! :)

[@alvaro_gavilan_rojas]

Thank you!

[@agustingambaretto2260]

Thanks for the demo, very useful for a beginner like me! managed to do something similar following the steps. I wanted to ask something. My formula in stead of having a single variable ( time), y have 3 input variables, i have to compute p1[i], p2[i], p3[i] to be able to calculate the y value. I manage to adapt my code to that and plot it, but i got stucked in the curve_fit. In the curve_fit uses only 1 xdata if i am correct. What variation can i use? I hope you know, thanks !

[@apm]

Check out this tutorial apmonitor.com/pds/notebooks/06_regression.html

[@kareemsakr41]

thank you

[@NawtieBoy96]

6:26 whats the formula really that you have used in that bpm fn ? i didnt understand that part. usually what I know is that therea re various non-linear methods that one can choose for non-linear reg types such as polynomial fits , lograthmic fits and so on . correct me if I am wrong pls

[@apm]

Check out 3:58 for the correlation that is used in the bpm function. There are many different types of nonlinear functions that can be used or even machine learning where something like an LSTM can predict time-series data. Here is more information on machine learning approaches: apmonitor.com/pds There is a regression module that may help in particular: apmonitor.com/pds/notebooks/06_regression.html

[@margalanis]

Hey, great content and pretty clear explanations! I have a question though. Is it easy to add some constraints to your variables (e.g. 0

[@apm]

No, you can add simple upper and lower bound constraints to variables in Scipy curve fit but not more complicated bounds that involve two or more variables. There is also scipy.optimize.minimize and python gekko that support these types of constraints: apmonitor.com/che263/index.php/Main/PythonOptimization

[@margalanis]

@@apm thanks a lot for the feedback!

[@obinnaizima9387]

Hi, this was very helpful. However, if you had a dataset that comprises of multiple exponentially decaying curves, how would this work out? I can relatively do this for a single exponential decay curve but unsure on how to deal with multiple exp. decay curves.

[@apm]

You can add additional variables and equations to the model. It is no problem to have any number of variables and equations. There are additional tutorials on regression here: github.com/APMonitor/data_science/blob/master/06.%20Regression.ipynb

[@stevethach3340]

Hi! Great video! A couple of questions: 1) What if you had multiple people's heart rate? Is it possible to create a curve that would fit all of them? 2) What if we do not have the BPM equation? Thank you!

[@apm]

Yes, just add the other people to the objective function. You'll likely need a nested loop inside the objective function. The objective function is a single number so all the sum of squared errors are lumped into one value.

[@guerreirodaluzgmailcom]

Hi Mr. ,What criteria does it take to choose the values for the initial conjecture of the parameters? I will be grateful for the answer

[@apm]

They are guess values. If it doesn't converge with guess values of 1 or 0 then I sometimes create a plot and adjust the values myself until the predicted gets closer to the data. Vanishing gradients or local minima can be a problem with poor initial guess values.

[@certifiedcriticmedia1532]

Nice

[@shreeniketjoshi]

Hi, that was awesome and very helpful! Just one question, when you entered c, cov where was the cov used? Only c was printed out, right? If I enter just c in place of c,cov would there be any difference? If you can attach a link for the same, that would be great. Thanks a bunch

[@apm]

Yes, you can look at the error or R^2 value when you adjust only one of the parameters at a time. Here is another example: apmonitor.com/me575/index.php/Main/NonlinearRegression You can turn the status to off (0) for any of the parameters that you want to hold constant.

[@apm]

Yes, you need to have both output arguments otherwise it includes c,cov as a tuple.

[@shreeniketjoshi]

@@apm Oh thanks! So basically the curve fit function outputs c, cov and we need to receive it. We may choose not to use it. Thank you so much

[@skkumarish]

I'm a newbie to python. can you please explain me why use choose the particular equation for your curve fitting?

[@apm]

The choice of equation is just an example to show nonlinear regression. Sometimes you have to try different equation forms to get one that fits well.

[@tonycardinal413]

Thank you so much for posting this. Ques: If there is a high covariance between the parameters why is this bad? thanx!

[@apm]

The parameters are single values so there is no covariance between them. If you have a high degree of covariance between data (x, y), it means that the two variables have a strong linear relationship. If one goes up, the other also goes up (or down).

[@KarthikeyanMmmm]

hello sir , is there any similar package in python as curve fit , in which we can able to impose some constraints ( just like during regression at a particular point slope should be a specific value for continuity ) during regression, OR is there any posiblity in curve_fit to impose some constraints

[@apm]

Try Python gekko or scipy.optimize.minimize for two constrained options in regression: apmonitor.com/me575/index.php/Main/NonlinearRegression

[@KarthikeyanMmmm]

@APMonitor.com sir , I have X and Y data, in which first ranga of data fits in certain equation and next range of data fits in some other equation, I used scipy.curve fit first set of data to non linear equation, for second set of data to be fitted in second equation , I need the regressor to follow the constraints, 1. at the break point / trasnsitiin point of frist equation to second, the value of y(x) two equation should be same, 2 . and also slope of equation one and two at that transition point should also have to equal , so I have to use this constraint during fitting of second range of data to the second equation , what I understood minimize () is not a regressor ,, a equation solver, and in GEKKO I don't know how to impose those constrains, please give a help or suggession, Hope you aware of maple, in that 'LSSolve ' was there to docl this regression with constraints, I need a similar function or package in phython to do curve fitting with user defined contraints

[@apm]

@@KarthikeyanMmmm you can switch equations with one equation with 0=b*(eqn1)+(1-b)*(eqn2) with b=1 when eqn1 is active and b=0 when eqn2 is active. You can set the derivative equal at the switch point with (s*(eqn1.dt()-eqn2.dt())==0) where s is the switch point that is 1 when there is the switch. Please see apmonitor.com/do for additional examples. If you have questions as you get started, please post a complete and minimal example to StackOverflow with tag [gekko].

[@KR-uy6or]

So for non linear regression we have to assume/guess which curve equation will fit the data and unlike linear regression in PYTHON which predicts itself??

[@apm]

Yes, that is correct. Machine learning has fewer assumptions on model form. apmonitor.github.io/data_science

[@KR-uy6or]

@@apm thanks for the help🙂

[@erenyeager4452]

very good tutorial, however my doubt is what if you have a matrix of parameters? how do you make the objective function. eg: Ax = b. I need to find A. how do you frame the problem?

[@apm]

You may want to use Gekko for that problem: apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization Check out the matrix operations with Gekko: github.com/BYU-PRISM/GEKKO/blob/master/examples/test_arrays.py

[@KarthikeyanMmmm]

is there iany way to solve equations in a python list [eqn1,eqn2,eqn3.........] with adition constainsts in form of equations e.g. a+bX=90, a*X^2+b*X=100, like in regression we have a model, if we substitute data points in it we will get many equations of model parameters along with it we add additional constaraints please help me

[@apm]

Yes, please see apmonitor.com/me575/index.php/Main/NonlinearRegression as an example of this type of problem where you have an objective and constraint with data regression.

[@giocanox97]

Thanks for the very clear video. However, it doesn't seem to work for my specific function: after defining variables x and y using x.values as in the video at 10:56 from a datafile, I try doing curve fit and it tells me that "only size-1 arrays can be converted to python scalars", which I am unable to fix. All the other steps are the same as in the video This is the function I am trying to fit: y = c_1 * x^{c_2}

[@apm]

It is x**c2 instead of x^c2. Does that help? Source code is here: apmonitor.com/che263/index.php/Main/PythonDataRegression

[@giocanox97]

@@apm thanks a lot! I was using math package but without it it works fine

[@senolkurt7864]

Thanks for the great tutorial. Since in real life we have only the data, how can we find the best non-linear equation that fits to our data?

[@apm]

If you don't know the non-linear equation, try using regression with Machine Learning: apmonitor.com/pds/index.php/Main/RegressionOverview The lazypredict package in Python helps evaluate the regression on multiple models. Otherwise, try different functional forms that you design.

[@benedictodhiambo4107]

How did you determine the values of the constants

[@apm]

They are initially just guesses. With the solver, they are selected by the optimizer to minimize the difference between the predicted and measured values.

[@nutakkipradeep2708]

if i know the predicted curve z_pred = x*y after some matrix transformations and i know the actual value z_actual then if i find MSE(z_actual - z_pred) then how can i reduce this error? Anyone can answer

[@apm]

If x or y are adjustable parameters then you typically let the optimizer modify those values to minimize your MSE or SSE.

[@user-kai516]

Hi, I am trying to fit my data with SIR model. Is there any suggest for fitting the data with ODE equations with two coefficients (reaction rate constant)? Thanks in advance.

[@apm]

Here are related examples: apmonitor.com/do/index.php/Main/EstimatorObjective and apmonitor.com/do/index.php/Main/COVID-19Response

[@user-kai516]

@@apm Thanks a lot! I'll give it a try ：)

[@vchamilka]

How to estimate parameters of a differential equation by fitting to experimental data?

[@apm]

Here are some examples: apmonitor.com/do/index.php/Main/DynamicEstimation

[@oxydol3456]

Does this curve fitting function sometimes return wrong answer?

[@apm]

If the solver reports a successful solution then it is at least a local solution. Try different initial guesses or try a different solver such as Python Gekko: apmonitor.com/me575/index.php/Main/NonlinearRegression

[@besttom8823]

How to actually find that correlation function is there a way?

[@apm]

You can either select a form yourself based on knowledge of the system (linear, nonlinear, exponential, logarithmic, etc) or else use concepts from machine learning to determine the correlation. For ML content, see apmonitor.com/pds (Regression section).

[@user-hk2uj7jl4g]

Does principle is gauss-newton or grading descent ? And I want to learn step by step , no use modules THX😭

[@apm]

It uses a gradient descent. There is step-by-step code here: apmonitor.com/me575/index.php/Main/QuasiNewton There is also an optimization textbook that details the methods: apmonitor.com/me575/index.php/Main/BookChapters

[@KR-uy6or]

@@apm so when using ML libraries , python automatically uses one of the approximation principle to find out the global maximum/ minimum ? I mean we don't need to create a function for example to do gradient descent approximation....

[@sobersabin]

hello sir, I did not understand , how you get initial guess.If you have time can you please explain me.I am master student in France.

[@apm]

In most cases you can just use an initial guess of 0 or 1 for all values. In this case, the problem is very nonlinear so it requires a better initial guess. A good way to come up with the initial guess is to try values until the predicted and measured values are close. An optimizer can then find the best parameters if you start with a good initial guess. Sometimes you also need to add bounds to the parameters to get it to converge. A better solver can also help. Here is a tutorial on Python Gekko: apmonitor.com/che263/index.php/Main/PythonDataRegression

[@hibadu251]

Is this Levenberg-Marquardt algorithm??

[@apm]

No, it is an SQP method with a Nonlinear Programming solver. More details on solver methods are at apmonitor.com/me575

[@MaddingAlex]

How to find t-statistics, F-statistics and p-values for this regression? Using Python

[@apm]

Here's some related information apmonitor.com/che263/index.php/Main/PythonRegressionStatistics

[@basichack6974]

Hello please. I have done everything in your video. It works every well. But how can I predict the heart rate for the time out of the range? I mean what is now the prediction function I have to call??? Please help me for this task.

[@apm]

You can use the coefficients to build an equation for the prediction.

[@apm]

Try creating a new function for the prediction. More help is here: apmonitor.github.io/begin_python (see module on functions).

[@basichack6974]

Oh yeah I got it. That is so easy. Thanks a lot❤️❤️❤️

[@zartadavid2900]

how can I know the chi square?

[@apm]

Here is a good summary www.google.com/amp/s/data-flair.training/blogs/python-linear-regression-chi-square-test/amp/

[@cccloud3256]

Sorry, but I can't access the data. Can anybody help?

[@apm]

Here is the source code with the data: apmonitor.com/che263/index.php/Main/PythonDataRegression

[@ruslanruslan338]

Thank you very much for your video! By the way, is anybody know how to solve the same issue, but when you have more than one variable. For example Y = a * x1^b + c*x2^d. We have dataset with x1,x2, Y. And how we can optimize a,b,c,d?

[@apm]

Here is something similar: apmonitor.com/me575/index.php/Main/NonlinearRegression

[@ruslanruslan338]

@@apm Thank you very much!!! Yes, it is what I'm looking for.

[@HealthyFoodBae_]

Is this non parametric regression?

[@apm]

No, it has parameters that we adjust to fit the predicted values to the measurements.

[@JJGhostHunters]

Hello...Can anyone provide any resources for how to perform "nonparametric quantile regression" in Python?

[@apm]

Does this help? pypi.org/project/vqr/

[@JJGhostHunters]

@@apm This seems very promising howerver the documentation is not clear how to get started. Perhaps I am making things overly complicated, however I have yet to see any examples in Python or R that will generate the curves shown in Wikipedia. This seems "simple" but apparently it isnt. en.wikipedia.org/wiki/Quantile_regression