Bootstrap Hypothesis Testing in R with Example | R Video Tutorial 4.4 | MarinStatsLecutres

Bootstrap Hypothesis Testing in R with Examples: Learn how to conduct a hypothesis test by building a bootstrap approach (Re-sampling) with R statistical software without a package, step by step. 👉🏼Related: Bootstrap Hypothesis Testing in Statistics Video: bit.ly/2USN1Se 📝 Find R practice dataset (chickdata) and R Script here: (statslectures.com/r-scripts-d...)
👍🏼Best Statistics & R Programming Language Tutorials: ( goo.gl/4vDQzT )
►► Like to support us? You can Donate (bit.ly/2CWxnP2), Share our Videos, Leave us a Comment and Give us a Like ! Either way We Thank You!
►In this R video tutorial, we will learn how to use R to perform a hypothesis test using a bootstrap approach.
► Bootstrapping in statistics is a resampling based approach useful for estimating the sampling distribution and standard error of an estimate.
► Bootstrapping in statistics and in research provides an alternative approach to approaches based on large sample theory (you may recall that many approaches rely on having a large n in order to carry out the method). It becomes particularly useful when dealing with more complicated estimates, where their sampling distribution and/or standard error may not be easily calculated
► We will focus on comparing the means (and medians) of two different groups, although we present the approach in a more general way, so that you can test a hypothesis about any other estimate/statistic calculated from your data.
► An R package does exist for bootstrap hypothesis testing (package name: boot), although the package is limited in the sorts of estimates/statistics it can conduct a bootstrap approach for. Our goal is to show you how to build the bootstrap approach yourself, so that you can make changes to the sorts of statistics/estimates you conduct tests for. You can practice building the test yourself, and then compare the results to what you get using the "boot" package in R. Note that if you do this, numeric values will differ slightly because you and the package will end up with a different set of bootstrap samples, and so there will be a slight numeric difference in results.
■Table of Content:
0:00:33 import the data into R
0:00:39 exploring the dataset used for performing a bootstrap hypothesis testing in R
0:01:12 How to visually compare the two groups in our dataset in R? Creating side by side box plots in R
0:01:30 Introducing the first test statistic for Bootstrap in R: the absolute value of the difference in the mean weight for the two diets (a two-sided two-sample t-tests)
0:01:43 Introducing the second test statistic for Bootstrap in R: absolute value of the difference in median weights for the two diets
0:02:09 Steps to calculating the two test statistics: 1) calculate the mean for each of the two different feed types with R programming language
0:02:40 calculate the test statistic 1 using the' with' R command (function) as well as a 'tapply', in R
0:03:07 calculate test statistic 2, using the with function and tapply function in R
Bootstrapping in R Step by Step:
0:04:27 setting a seed in R
0:04:32 why should you set a seed for bootstrapping in R
0:05:04 setting the number of observations, the number of bootstrap resamples and the variable in R
0:06:07 How to ask R programming language to resample with replacement from our variable
0:06:46 checking the bootstrap matrix produced in R
0:07:03 calculating the test statistic 1 and test statistic 2 for each of the n bootstrap resamples using a loop in R statistical software
0:09:31 reminder of the definition or the calculation of the p-value a
0:09:51 interpreting the generated test statistics for our bootstrap hypothesis testing
0:10:43 how to us R programming language to check the generated test statistic
► ► Watch More:
►Bootstrapping Statistics & Bootstrapping in R bit.ly/2GL6AYS
► Intro to Statistics Course: bit.ly/2SQOxDH
►Getting Started with R (Series 1): bit.ly/2PkTneg
►Graphs and Descriptive Statistics in R (Series 2): bit.ly/2PkTneg
►Probability distributions in R (Series 3): bit.ly/2AT3wpI
►Bivariate analysis in R (Series 4): bit.ly/2SXvcRi
►Linear Regression in R (Series 5): bit.ly/1iytAtm
Follow Us
Subscribe: goo.gl/4vDQzT
website: statslectures.com
Facebook: goo.gl/qYQavS
Twitter: goo.gl/393AQG
Instagram: goo.gl/fdPiDn
Our Team:
Content Creator: Mike Marin (B.Sc., MSc.) Senior Instructor at UBC.
Producer and Creative Manager: Ladan Hamadani (B.Sc., BA., MPH)
These videos are created by #marinstatslectures to support some Statistics and R Programming courses at the University of British Columbia (UBC) (#IntroductoryStatistics and #RVideoTutorials ), although we make all videos available to the everyone everywhere for free.
Thanks for watching! Have fun and remember that statistics is almost as beautiful as a unicorn!

[@marinstatlectures]

In this R video tutorial, we learn to use R to perform a hypothesis test using a bootstrap approach. An R package does exist for bootstrap hypothesis testing (“boot”), but the package is limited. Here we will show how to build the bootstrap approach; this will allow us to make changes to the sorts of statistics/estimates we want to conduct the test for. The R script accompanying this video has all the R codes used in this tutorial plus extra R codes for students to explore on their own ( statslectures.com/r-scripts-datasets ). If you like to support us, you can Donate (bit.ly/2CWxnP2), Share our Videos, Leave us a Comment and Give us a Like ! Either way We Thank You!

[@AndreaLyuu]

Hello Mike! Thanks for all the great content! Please help me with this question: If our hypothesis test were to be a one sided (H0:mean-c>=mean-m, H1: mean-c

[@auroralorenzi2957]

Hi Marin. Thank you for your videos! It's not an easy topic to teach, but your videos are very clear.

[@carlosalexandrecosta4234]

Just an amazing content. I'm studying Data Science and i can tell you, your videos are helping a lot! Thank you!

[@iheleanbeefpatty]

Lovely video, thank you! everything clear and I love that you showed us what is behind those functions, so one can experiment with other libraries having good benchmark.

[@niceday2015]

Thank you very much, Your teaching has expanded my perception of the world

[@kahuilim2122]

Thanks for taking the time to put these amazing videos together.

[@marinstatlectures]

You’re welcome

[@JJ-wx3nd]

Thank you so much! Especially after I found the boot() in R documentation is not enough to know.

[@kaibecker1411]

10:23 Minor mistake: 48.0 and 68.2 (#16) are larger than your test statistic

[@alinevm4915]

such a didactical video, thanks a lot!

[@parvenraj98]

You are the best !!!!!

[@xdienn]

Great video, thank you so much! I'm still wondering though, how to proceed if you have a 2x2 factorial design? Do you then calculate 4 test statistics, one for each group? And for interaction effects?

[@alinepontes7360]

Thanks for the video! It was the only way for doing a hypo test for a complex dataset. The boot package was not enough. BTW, is there a recommended citation for this boot method (e.g. a book)?

[@farahyounes2813]

thank you for your explanation, for the case of a time series how we can apply the method of bootstrap to compare two spectral densities

[@b.ambrozio]

Thanks for the video! Question: About your last statement: "Any time doing a hypothesis test, we should also include a confidence interval to give an ideal of how big the difference would be". Does it mean I should run my t-test against the CI instead? (e.g. calculate the CI of all my 1000 arrays, and do the t-test agains the means from the CI means? In other words, should I use the means calculated from the range between the first and last quantile in my t-test? )

[@gruppenzwangimweb20]

nice video! btw... this BootstrapSamples

[@mathieufen2239]

Very cool video! Thanks! I wonder if this approach could be used on paired data...

[@marinstatlectures]

Definitely, the concept of bootstrapping can be used for just about any structure of data. I explained it simply here, but the concept transfers very widely

[@cb4808]

5:00 what do these numbers in the brackets mean? set.seed(112358)

[@zjardynliera-hood5609]

its just an arbitrary number to set the seed to that rng state. Basically if I set that seed on my machine, I would get the same results he is getting, and if I reran the code, I would get the same results each time.

[@forestalgarcia1506]

If I have more than two "diets", how to calculate the absolute difference of means?

[@MsWilliam63]

Hi Mike, great videos. Really clear and helpful. I have two questions. What is the best way to report these results in text? Is it best just to report the bootstrapped difference in means and SD and p value (e.g., observered stat = X, mean ± SD, p-value=X)? Is it possible to combine a t-test or Mann-Whitney U with the bootstrapped data in order to get a t-stat as well as the p-value for your difference in means/medians?

[@marinstatlectures]

it's hard to say the best way to report, as that really depends on context...what is the discipline, what was the focus of the paper, etc. regarding approaches, you can certainly combine this sort of approach with a parametric one like the t-test. for example, you may wish to use a Bootstrap only to estimate the SE of your estimate, and then substitute this estimate into a standard approach like a t-test ex: Confidence interval: Estimate +/- t * BootstrapSE this of course requires the assumptions of the standard t-test/confidence interval approach to be met.

[@TanMan1]

at 5:35 in the video - if you want to bootstrap for multiple variables, how would you adjust the code?

[@Rainstorm121]

Thanks very much Sir. If I have random distribution of scores for each variables as follows: A=7, B=13, C=23, D=19, E=15, F=30. If I want to do hypothesis testing to find out which of the variables has statistical significance of score, what is the best advise in using bootstrapping in this situation? Given that Ho: expected probability for each of the variables is equal to 0.12, and Hi: is not equal to 0.12.

[@nevertheless4504]

HI sir. you video really helps me a lot. However, I just have question. wen we want to do the one side test. We just need to delete the abs right, and when do the comparing, mean(Boot.test.stat1 >=test.stat1). we only use < or. >. instead of >=. Am I right

[@duncanrager7180]

Hi Mike, thanks for the helpful video. In this case, the first test statistic is the same as used in a two-sided, two-sample T-test. As an alternative, to use the same test statistic as a one-sided, two-sample T-test, would that be the difference of the mean weight for the two diets (not the absolute difference)?

[@marinstatlectures]

yes, that's right

[@AndreaLyuu]

Hello Mike! Thanks for all the great content! Please help me with this question: If our hypothesis test were to be a one sided (H0:mean-c>=mean-m, H1: mean-c

[@sohanaryal]

How can we randomly assign first 11 to one type of feed and remaining to other type of feed in bootstrap matrix?

[@MrSoumyaBanerjee]

Thank you for the video, but by resampling from d$weight, won't it cause the casein and meatmeal observations to get jumbled up in most cases? Wouldn't it be better to create 2 separate dataframes for each set of weight values, and resample separately from these 2?

[@marinstatlectures]

Good question. Because in a hypothesis test we begin by assuming there is no difference in weight of the two groups, we want them all mixed together..to see how often we’d observe a difference as large or larger than we saw, if there really is no difference in the groups. When building a confidence interval however, we want to keep the groups observation ms separate, to preserve any group differences observed. Hope that clarifies it

[@SinanMavruk]

@@marinstatlectures Thank you for the video and explanations. You made the point very clear for me. But in this case, the only difference between using permutation or bootstrapping is in the replacement in resampling? So, how to decide which test is better in application?

[@yinanxue8653]

Thank you for the video. I got a question. How do you know that in the bootstrap data, the first 12 rows are weights of casein and the last 13 rows are weights of meatmeal?

[@yinanxue8653]

Nvm. I figured it out. Is it because H0 is difference between mean equals 0, so if H0 is true, we can pool the data?

[@yekhtiari]

I have the same question.

[@marinstatlectures]

for bootstrapping, we resample with replacement...so really, it just matters that we have 12 observation for the one group, and 13 for the other. we could just as easily label the first 13 rows for meat meal and then the next 12 for casein. all that is really necessary is that we assign 12 of them to casein and 13 to meatball...but the order we assign them in doesnt matter...hope that clarifies it...

[@gillianjean2237]

@@marinstatlectures Hi Mike, brilliant video but I'm still confused about this part. I have 45016 observations, 22049 in group A and 22967 in group B. How do I make sure that I'm sampling correctly from both groups?

[@rhosigma2199]

@@marinstatlectures Thank you for the video. I also have the same question. BootstrapSamples = matrix(sample(variable, size =n*B,replace =TRUE ,nrow =n,ncol=B) Does this function assure that the BootstrapSamples have 13rows of the meat meal and 12rows of the casein?

[@feloria1862]

When you resample at 6:20 does the resample keep the feed type variables separate? Or do weight values from either feed type resample to anywhere in the 23 observations?

[@feloria1862]

I figured it out the feeds are resampled to any position because the null hypothesis is that the two populations are the same, so swapping them to any position even if the feed type doesnt match is fine.

[@ogundepoezekieladebayo9428]

Can we then interpret the results of Bootstrapping with the way we interpret the result of independent t-test?

[@marinstatlectures]

For the most part you can. The beauty of the bootstrap though is that you can also work with more interesting/relevant statistics, aside from just mean/median, which the classical approaches use. You can work with just about any statistic/estimate you can imagine

[@Loggies89]

You lost me at the i=1 and i=2 bit. Is there a step you aren't showing where these are created? I'm getting an error saying object 'i' not found, so i assume i have to create it at some point before entering it into the boot test statistic.

[@munafahmed725]

In mean.default(BootstrapSamples[1:12, i]) : argument is not numeric or logical: returning NA How to solve this error?

[@juanfranciscoecheverry3031]

I have the same problem. thanks a lot!

[@veducatube5701]

Sir i jeed to bootstrap spatial point data... Meaning I have 10 values with lat long and a z . I need to bootstrap pairs of xy in a defined region (shapefile) can u help???? regards from India

[@marinstatlectures]

It difficult to answer without knowing exactly what your data looks like, but it sounds like you will want to res ample entire rows of your data

[@veducatube5701]

@@marinstatlectures Thank you for replying sir. Im giving you a dummy data : lat long water table ( depth in m) 29 79 23 28.45 78.30 21 27 77.45 25 30.30 79.02 26 31 77 22 25.45 80.30 32 Assume that all these original points of latitudes and longitudes with water table values fall in a district (boundary line of this district is a map file format called .shp or ESRI shapefile ). Sir, I want to bootstrap these three columns so that I may have more geographic points for water table in my district. That is possible only when latitudes and longitudes must not fall outside the district boundary or shapefile, meaning the lat long column values must remain contend within shapefile latitudes and longitudes. Sir its very crucial for me. Please guide or share some codes with me.. THank YOU

[@chathuraedirisuriya6535]

Bootstrap Hypothesis Testing R script link direct to a wrong file. Please correct it.

[@marinstatlectures]

thanks for letting us know, It should point to the correct file now.

[@mbellett74]

if the column "feed" is not ordered with( respect to meat meal and casein), how to order it before to run the boot.test.stat? many thanks

[@marinstatlectures]

You don’t necessarily need to order it, but you can do that with the sort() command. You can also use the tidyverse arrange() command as well

[@mbellett74]

@@marinstatlectures many thanks Mike, I did it using arrange(). Maybe I have to do it because in the boot.test command I have to define two groups of lines to confront: abs(median(bootstrapsamples1[1:938, i]) - median(bootstrapsamples1[939:1511, i] , ...and in my data-set the two groups of events to confront are mixed, again many thanks!!

[@santiagomendozapaz2135]

@@marinstatlectures maybe I am misunderstanding the figure, but, if our matrix from which we are going to resample contains 12 values for type 1 and 11 values for type 2 and we apply the resampling directly to the 23 values, the resulting resampled matrix is going to contain randomly 23 values from both types, therefore, why are you obtaining the mean between [1:12] and [13:23] as in the resulting matrix we are not sure if type 1 is contained in [1:12] or type 2 in [13:23]?

[@sunayana98]

Hi Marin, when I'm trying to find the test stats of bootstrap samples, R is telling me 'i' is not found. What do I do?

[@marinstatlectures]

it's difficult to tell without knowing the code you've entered, etc. but it sounds like this part of the code is not in a loop that is running from i=1,2,...,B the "i" is referencing the iteration number in the loop...and R cannot see what i is, so it sounds like you either having initiated a loop, or that command is outside of the loop

[@sunayana98]

@@marinstatlectures I've typed the command exactly like how you've typed it i.e., in the square brackets. However, it says 'i is not found'. Is there an alternate command?

[@pilobond]

@@sunayana98 I had the same problem but then I realized I type "for (i in i:B)" instead of "for (i in 1:B))" by mistake. Once this was corrected it ran fine. I wonder if you have the same problem.

[@damianspencer]

Do you do consultations? Please contact me.

[@marinstatlectures]

It depends on the work. I have no way of contacting you. You can get in touch with me if you like, my contact info is in the about section of our channel