Removing outliers in R with tools from dplyr and ggplot2 (CC232)

[@haraldurkarlsson1147]

Pat, This two-part exercise was probably my most favorite of those I have observed from you. I am currently in Northern Indiana so not suprisingly my data looks similar to yours. The data set I used went back to the early 1890s but there was a break in data collection from about 1910-1920 (World War I?). I did not see the extreme values in precip and snow that you observe. The most extreme precip and snow values were around 200 mm and 500 mm , respectively. When looking at the ten highest snow values I found events that line up with the big snow fall in Chicago in late January 1967. The station I am using may be experiencing higher precip (snow and rain) due to the lake effect. I am not sure how far or how large the lake effect is in Michigan in your neck of the woods. Finally, I converted the snow to rain using 1:12 ratio suggested by NOAA for this region. I thought it might be interesting to compare precip and "converted from snow precip". There are some 'hint' of a trend but caution is needed since the conversion of snow to precip is highly dependent on temperature.

[@Riffomonas]

Very cool! Great job digging into your data 👍

[@cyrillejar1914]

Very intresting and useful, as always. What do you think about tests like dixon's test or Grubb's test to identify outliers ?

[@Riffomonas]

I think it’s best to understand why an observer is an outlier before removing it

[@lalitpl4]

I stumbled upon one of your videos few months back. My life is much easier since then. Your videos helped immensely in my work. Thank you!

[@Riffomonas]

Wonderful! I’m so glad you found the channel 🤓

[@pratish143]

Thanks!

[@Riffomonas]

My pleasure! Thanks for tuning in 🤓

[@Riffomonas]

I just realized that you sent me $10 - thank you so much. That is very generous and much appreciated!

[@sarvagyavatsal9569]

very useful

[@Riffomonas]

Wonderful! I’m glad you enjoyed it 🤓

[@brianwood9610]

Wouldn't it be worth imputing a median value to those outliers?

[@Riffomonas]

Hmmm. Interesting suggestion. Considering how patchy the precipitation/snow fall data are I’m not sure. Maybe for the temperature data which would be more continuous

[@petongtitus]

how about replacing outliers with NA instead of removing outliers? could you show me how?

[@sven9r]

First 😝

[@Riffomonas]

🤓well done

[@yaqinguo8971]

Thanks!

[@Riffomonas]

No problem!