hello, where i can find your dataset?? can you share?
@rezanadimi33124 ай бұрын
Dr. Esmalifalak, thank you so much for your teaching in a short time. I wonder that there is any relationship between number of steps and lags in LSTM? What will happens if number of lags variables are far less than number of steps? Technically, it is possible, because the lstm looks for finding coefficients for a function makes between inputs and outputs. Is there any rule to limit number of time steps based on lag variables? for example, if the lag variables are 8, then number of time step (future step for prediction) must be less than 8. Thank you so much for your consideration in advance.
@MUHAMMADIMRAN-ii2yf10 ай бұрын
How to download
@AIwithDrMo9 ай бұрын
please contact the product customer service directly. Thanks
@peterpham4410 Жыл бұрын
Hello dr. Mo
@peterpham4410 Жыл бұрын
I want to hire you as my CDEGS tutor asap. Please reponse
@peterpham4410 Жыл бұрын
I have very important questions regarding the CDEGS. Please reply if you are existed.
@peterpham4410 Жыл бұрын
Dr. Moe, I have been looking for you regarding CDEGS If you received this message please reply.
@gulsaherdogan8441 Жыл бұрын
Thank you Dr.Esmalifalak, I have a question regarding the sliding window approach that you used for time series data. Due to the sequence of output, there will be multiple predictions for a single timestep, resulting in overlapping predictions. I am curious about how you handled this while evaluation model ? And plotting the predictions result ? Thanks a lot !!
@AIwithDrMo Жыл бұрын
Thanks Gülşah for your comment. You can handle evaluation and plotting by averaging predictions (the most common way), selecting the most recent prediction, or modifying your evaluation metrics to account for overlaps. When plotting, you can either average predictions or use transparency to visualize overlaps. If you have enough time, it is always recommended to try different methods and see which one works better for your application.
@joshuasuasnabar6058 Жыл бұрын
thanks you profesor, just a question. Is possible deal with categorical variables? Is important the type of enconding to use (one hot or label enconding)? Thanks you in advance
@AIwithDrMo Жыл бұрын
Joshua, Thanks for your comment. Yes it is possible! You can use Extended Isolation Forest (EIF). Please take a look at this page for more info and a python example: capable-timimus-00a.notion.site/Isolation-Forest-in-Categorical-Values-b5534c14548b4ba881199477939044c2
@SP-db6sh Жыл бұрын
Step stone of DS projects ... Plz make video on it to work with this step with customisable pipelines for different usecases .
@michael_bryant Жыл бұрын
Thanks, really helpful video
@kamakshishinde9984 Жыл бұрын
Thank you for the video. I found it very informative Can you please show how to run .py files for example where do we need to give filepath name and filter city name and can you also please show how the results looks like that are generated from .py file Thank you!
@tenten7379 Жыл бұрын
I have a question, this is an unsupervised model, right? is there a way to make the model predict a user input?
@AIwithDrMo Жыл бұрын
This is unsupervised anomaly detection method. It can be applied to user input data to detect anomalies or unusual patterns in user behavior over time. The basic idea is to use the algorithm to learn the normal patterns of user behavior based on the historical data, and then to use the model to identify any deviations from these patterns.
@MikeSaintAntoine2 жыл бұрын
Great video!
@tawnyarhorer50172 жыл бұрын
😘 ᎮᏒᎧᎷᎧᏕᎷ
@پوریاحبیبی-د1ف2 жыл бұрын
Thanks for the nice topic. I am wondering if we can do this considering the effect of seasonality? Like, lagging the sales values multiple times and creating new features and then training and testing the anomaly detector?
@AIwithDrMo2 жыл бұрын
you can do that for sure then train/test your model with similar approach.
@peymanrazmi59092 жыл бұрын
Excellent Dr.Mohammad. Are these types of algorithms (KNN) considered as weak algorithms in ensemble learning? Please make the similar video and post for other algorithms.
@peymanrazmi59092 жыл бұрын
Thanks Dr. Esmalifalak. Your explanation is very useful. How does the accuracy of the program change by changing the step size and log? Will the changes be noticeable? Also, I would appreciate it if you could post a similar video about multi-variable.
@AIwithDrMo2 жыл бұрын
Thanks Peyman. For the accuracy it is usually better to grid search different hyper-parameters such as number of lags. Trying different lags and testing the predictions (by walk forward method for example) would generally reveal the skill of different combinations of hyperparameters. I will have a video on the testing of time-series so stay tuned!
@neginpirannanekaran12362 жыл бұрын
Thank you so much. This was really helpful👌
@seyedmortezamirhoseinineja9442 жыл бұрын
Thanks Dr Mo.
@seyedmortezamirhoseinineja9442 жыл бұрын
The greatest ml videos in KZbin
@AIwithDrMo2 жыл бұрын
Thanks Seyed!
@peterpham4410 Жыл бұрын
@@AIwithDrMo Dr Mo I am looking for you regarding the CDEGS. Please reply
@AIwithDrMo2 жыл бұрын
Timecodes 0:00 - Intro 0:19 - Problem Definition 2:14 - Importing Data 4:46 - Changing data types - to_datetime 5:48 - Changing data types - LabelEncoder 8:28 - Reindexing - set_index 9:47 - Converting time series to conventional ML problem by shifting dataframe 18:55 - Model training 23:28 - Model evaluation 28:00 - Creating python files for MVP 29:32 - train.py 36:51 - predict.py
@tareqal-masri17822 жыл бұрын
Hi Dr. Esmalifalak, I'm a huge fan of all your videos, they've helped me with getting through university and get a career, can you please upload more videos, what data visualization tool do you use?
@alwaaffa2 жыл бұрын
You can help me with a master’s thesis for my software part (coding) in Python?
@AIwithDrMo2 жыл бұрын
Please fill out the following form for any specific questions, forms.gle/Jz4pkrNSGUqGhPug9
@alwaaffa2 жыл бұрын
@@AIwithDrMo I can connect with you by email?
@alwaaffa2 жыл бұрын
You can help me with a master’s thesis for my software part (coding) in Python?
@AIwithDrMo2 жыл бұрын
Please fill out the following form for any specific questions, forms.gle/Jz4pkrNSGUqGhPug9
It is hard to find such good explanations on Isolation Forest. Keep up the good work!
@ugurileri99363 жыл бұрын
very helpful video; I want to ask one question about time series part; you have entered n_neighbours=5 why 5? What about if it is 2 or 3 or 4? If I use time series anomaly detection part for 4 - 5 sensors column data; what should I choose for n_neighbours parameter? again 5?
@AIwithDrMo2 жыл бұрын
Thanks Ugur. n_neighbours depends on your application and we usually try different ones to see if the outputs makes sense for this specific project or not.
@soumikbasu15563 жыл бұрын
A very well-structured but simple way of explanation. Can we also have a look at measuring the efficacy of the model?
@AIwithDrMo Жыл бұрын
Thanks for the comment. Isolation Forest is an effective anomaly detection method that can handle high-dimensional data and has several advantages over other methods. Its efficacy depends on the specific characteristics of the data and hyperparameters used. For example, the performance of the algorithm can be affected by the choice of subsampling ratio, the number of trees in the forest, and the choice of distance metric used to evaluate the splits.
@mehdichellak43733 жыл бұрын
thank you sir.
@wakilkhan88753 жыл бұрын
Please make another video on, Anomaly detection One-class SVM for Novelty detection
@MrSanghan19903 жыл бұрын
Thx, I will apply it~~
@shahrzadamini57463 жыл бұрын
Hi, good job, I have a question, how we can resample according to the year?
@AIwithDrMo3 жыл бұрын
I usually use 12 months resampling like "resample('12M')"
@coeurblanc49993 жыл бұрын
good video. suggest to turn up the volume. good content nonetheless. thanks
@shahrzadamini1403 жыл бұрын
Hi, thanks I found it really helpful, but I have a question about the Contamination parameter, how we can choose a suitable value for this parameter?
@AIwithDrMo3 жыл бұрын
glad you liked it. Contamination should be tested for your application. You can start with small numbers ( like 2%) and look at the results. If algorithm catches things that are normal to you, you may decrease the threshold otherwise keep increasing it ... You will find something reasonable for the data set you are working with.
@shahrzadamini1403 жыл бұрын
@@AIwithDrMo Thanks a lot for your explanation.
@staceywang78353 жыл бұрын
I really love your video. could i ask if there is part 2 of 2 for this section? thank you very much!
@rubenr.24703 жыл бұрын
thanks for this video! its not easy to find high quality content like this! keep it up!
@alhanoufalsuwailem39923 жыл бұрын
Thanks for the clarification ! after applying iforest , how can I evaluate the cluster's result ? do you have specific method used for evaluation this type of unsupervised learning? I'd really appreciate that.
@AIwithDrMo2 жыл бұрын
I usually prefer to have a small labeled dataset (from client etc.) and validate my results with those labels.
@tiger06t3 жыл бұрын
Hi! Thanks for the great tutorial. But I have a question, is it possible that isolation forest output different result? I have used isolation forest on my dataset, but the output results are a bit different than previous results everytime (I haven't changed any parameter in the model and the dataset I used is the same).
@AIwithDrMo3 жыл бұрын
Thanks Johnson. Isolation forest randomly splits the datasets so there is no guarantee to have exactly the same results each time but, if you do it enough times and average out the results, it should converge to one solution (with reasonable data sets of course).
@tiger06t3 жыл бұрын
@@AIwithDrMo Thank you! Dr. Mohammad
@aashi97813 жыл бұрын
Hello Dr. Mohammad, Is the algorithm effective with the real time streaming data? I have sensor data of around more than 100 sensors, should I need to find the important variables before feeding into the model or should I pass all the variables and let the algorithm decide by itself? Multicollinearity exist in the data .
@AIwithDrMo3 жыл бұрын
Hi Aradhna, Isolation forest is one of the fast algorithms in anomaly detection and people use it with large datasets like financial datasets. For sensor data you don't have to process very high frequency data. You may need to find the right sampling rate (for example temperature usually is not changing sooner that 10-20 sec so sampling every second is not necessary ). If your window is 1 minute, you should not have noticeable problem in a regular application. I usually start will all of the data and the drop/minimize if I have to...
@zaynyao38634 жыл бұрын
You solved a big problem for me,thank you
@AIwithDrMo4 жыл бұрын
I am glad that helped you.
@hamzasmidi34454 жыл бұрын
Thank you Mohammad
@AIwithDrMo4 жыл бұрын
I am glad that you liked it.
@rezamonadi42824 жыл бұрын
Great explanation...
@AIwithDrMo4 жыл бұрын
Thanks Reza. I'm glad you liked it.
@VladimirOlteanu4 жыл бұрын
Hello! Just a question. Is this an algorithm a classic isolation forest or an extended isolation forest (I saw you named the object with the predictions eif)? Is there any way to implement an extended isolation forest? Basically the difference between EIF and IF is that the EIF takes random intercept and slope and does the split based on the trend line. Thank you for the video!
@AIwithDrMo4 жыл бұрын
Hi Vladimir This is classic isolation forest and as you mentioned, EIF can also be used similarly.