Thanks for watching! More experimenting with both suggests that it really depends on the context. Any difference is really minimal

Definitely, I've seen people create papers as R packages - check out this pre-print peerj.com/preprints/3192v2/

@@Riffomonas thanks for sharing, I also tried myself to embeded my analytic code into a package.... but at some point I felt that generated data are too heavy for a R package. Also, I never considered (but I would like) using "test that" for an analytical workflow. In addition beyond "report generation" it would be nice to include github action within the workflow.

Something like Test Driven Development for data analysis is always rolling around in the back of my head :) It's hard because it requires using functions to test and most data analyses don't use home made functions, they use functions from other packages, which are hopefully already tested

Thanks! A few more weeks 🤓

I highlight the lines and then use the shortcut to comment them. On my mac it's shift-command-c

It's about reproducibility and transparency. Modifying data in excel is very much not reproducible or transparent. It's next to impossible to document changes in excel files. Also, excel is $$$ whereas R is free. With R, we can document all of the changes in the script and rerun the script multiple times without having to worry about breaking things. Also, if I make a mistake, I can easily correct it by reprocessing the raw data with the corrected code.

My pleasure - thanks for watching!🤓

wonderful - thanks for watching!

This type of thing is much easier using the {patchwork} package. Thanks for watching!

My pleasure! thanks for watching🤓

You can get it through the links in the blog post linked in the show notes. github.com/riffomonas/climate_viz/tree/a428f64b7db493145bf84ec6f38f8e89da258675/data

You can certainly grow the list, which really isn't a problem if you don't think it will be long. Alternatively, I've also seen people initialize a list that's larger than you think it will be and then prune it after you know how big it actually should be

I think that only works on packages already on CRAN

@@Riffomonas Okay, thank you

my pleasure! thanks for watching 🤓

hah - thanks!

lol - thanks for watching 🤓

absolutely - thanks for watching!

my pleasure! thanks for watching :)

Thanks for your very kind comment!

wonderful! sometimes it's fun to go into the weeds a bit :)

So glad to hear it - thanks for watching!

Thanks - I'll have to check that out

I have found that the major packages are available in one of the conda collections. It's not a horrible process to contribute a conda package if you need to. This SO link is somewhat helpful stackoverflow.com/questions/52061664/install-r-package-from-github-using-conda

Thanks for watching - I'm glad you enjoyed it!

Hah! Thanks so much for watching. I'm glad you're finding my videos helpful 🤓

Thanks for the head up - it's there now

Wonderful - my pleasure!

Sorry! If there's anything specific let me know what was confusing

lol - well, you're #1 today 😂

Sorry, i'm not familiar with the BiodiversityR package. You might try installing tcltk and then try BiodiversityR again

my pleasure! thanks for watching🤓

Hey, if your local community uses Julia, go for it. Same goes for Fortran, Haskell, whatever. There's no debate. The only rule is that people need to learn to program. It's best to learn what your local community uses. No interest in engaging in any type of language wars here 🤓

Thanks Philippus!

Thanks! I think I got the @returns after editing the video :) It really doesn't matter if you use @return or @returns, but @returns is the new way of doing things.

I’ve used it for this very purpose. It’s very good and saves so much time

I'll make a deal with you and all of my other viewers... I promise that I will never intentionally use ChatGPT or its ilk to generate code, documentation, anything on my channel :)

I believe you, seeing how good you recall complex regex ;) ​@@Riffomonas

Thanks - I've used it in other episodes, but I find the {microbenchmark} is easier to use for some applications

Thanks for watching!

Thanks for the feedback - I'm finding that if I use as.data.table or setDT, I get similar results as plain data.table and inner_join

​@@Riffomonas I made synthetic datasets since I don't have your fasta data. In my test, dtA was the fastest, followed by using setDT and as.data.table. data.table() was similar to dplyr with inner_join. Here is my code: each_num <- 1e4 animal_legs <- map_dfr(data.frame(animal = c("cow", "fish", "chicken", "dog", "sheep"), n_legs = c(4, 0, 2, 4, 4)), rep, each = each_num) %>% mutate(n = 1:nrow(.)) animal_sounds <- map_dfr(data.frame(animal = c("cow", "chicken", "cat", "sheep", "dog"), sounds = c("mooo", "cluck", "meow", "baaa", "bark")), rep, each = each_num) %>% mutate(n = 1:nrow(.)) # make a copy for setDT because it changes things in place, # which would affect everything else relying on animal_legs and animal_sounds # in microbenchmark. animal_legs_2 <- copy(animal_legs) animal_sounds_2 <- copy(animal_sounds) #data.table for dtA test animal_legs_dt <- data.table::data.table(animal_legs, key = "n") animal_sounds_dt <- data.table::data.table(animal_sounds, key = "n") microbenchmark::microbenchmark( base = base::merge(animal_legs, animal_sounds, by = "n", all = FALSE), ij = dplyr::inner_join(animal_legs, animal_sounds, by = "n"), dt = { animal_legs_dt_test <- data.table::data.table(animal_legs, key = "n") animal_sounds_dt_test <- data.table::data.table(animal_sounds, key = "n") animal_legs_dt_test[animal_sounds_dt_test, nomatch = NULL, on = .(n)] #inner join }, dtA = animal_legs_dt[animal_sounds_dt, nomatch = NULL, on = .(n)], as_dt = { animal_legs_dt_test <- data.table::as.data.table(animal_legs, key = "n") animal_sounds_dt_test <- data.table::as.data.table(animal_sounds, key = "n") animal_legs_dt_test[animal_sounds_dt_test, nomatch = NULL, on = .(n)] }, set_dt = setDT(animal_legs_2)[setDT(animal_sounds_2), nomatch = NULL, on = .(n)] ) Here's the results table on my computer: Unit: milliseconds expr min lq mean median uq max neval cld base 29.754131 31.562985 33.248288 32.509195 34.936214 38.573338 100 a ij 3.847486 4.260528 5.063156 4.359981 4.581972 10.221760 100 b dt 3.840432 4.029632 6.483946 4.263007 4.815313 145.038057 100 b dtA 2.474058 2.599248 4.473535 2.822705 3.102000 137.916202 100 b as_dt 3.255442 3.403854 4.397897 3.637299 4.121683 11.064186 100 b set_dt 2.881791 2.995842 3.809361 3.250692 3.767623 7.726401 100 b Base R was actually the fastest when I did this test with the original animal_* datasets, but it clearly doesn't scale very well.

@@Riffomonas Apologies if this is showing up for a second time, but I replied earlier and now it seems to be gone. I made some synthetic data because I don't have your fasta data. I consistently find that dtA is fastest, followed by setDT() and as.data.table(). dplyr and data.table() are comparable. Here is my code: each_num <- 1e4 animal_legs <- map_dfr(data.frame(animal = c("cow", "fish", "chicken", "dog", "sheep"), n_legs = c(4, 0, 2, 4, 4)), rep, each = each_num) %>% mutate(n = 1:nrow(.)) animal_sounds <- map_dfr(data.frame(animal = c("cow", "chicken", "cat", "sheep", "dog"), sounds = c("mooo", "cluck", "meow", "baaa", "bark")), rep, each = each_num) %>% mutate(n = 1:nrow(.)) # make a copy for setDT because it changes things in place, # which would affect everything else relying on animal_legs and animal_sounds # in microbenchmark. animal_legs_2 <- copy(animal_legs) animal_sounds_2 <- copy(animal_sounds) #data.table for dtA test animal_legs_dt <- data.table::data.table(animal_legs, key = "n") animal_sounds_dt <- data.table::data.table(animal_sounds, key = "n") microbenchmark::microbenchmark( base = base::merge(animal_legs, animal_sounds, by = "n", all = FALSE), ij = dplyr::inner_join(animal_legs, animal_sounds, by = "n"), dt = { animal_legs_dt_test <- data.table::data.table(animal_legs, key = "n") animal_sounds_dt_test <- data.table::data.table(animal_sounds, key = "n") animal_legs_dt_test[animal_sounds_dt_test, nomatch = NULL, on = .(n)] #inner join }, dtA = animal_legs_dt[animal_sounds_dt, nomatch = NULL, on = .(n)], as_dt = { animal_legs_dt_test <- data.table::as.data.table(animal_legs, key = "n") animal_sounds_dt_test <- data.table::as.data.table(animal_sounds, key = "n") animal_legs_dt_test[animal_sounds_dt_test, nomatch = NULL, on = .(n)] }, set_dt = setDT(animal_legs_2)[setDT(animal_sounds_2), nomatch = NULL, on = .(n)] )

​@@Riffomonas I've tried replying a few times, but youtube seems to be autoremoving the comment. Maybe something to do with the code snippet.... Anyway, I made large synthetic datasets because I don't have the fasta data, and ran everything again. dtA is consistently fastest, followed by setDT and as.data.table(). Here is my code below. each_num <- 1e4 animal_legs <- map_dfr(data.frame(animal = c("cow", "fish", "chicken", "dog", "sheep"), n_legs = c(4, 0, 2, 4, 4)), rep, each = each_num) %>% mutate(n = 1:nrow(.)) animal_sounds <- map_dfr(data.frame(animal = c("cow", "chicken", "cat", "sheep", "dog"), sounds = c("mooo", "cluck", "meow", "baaa", "bark")), rep, each = each_num) %>% mutate(n = 1:nrow(.)) # make a copy for setDT because it changes things in place, # which would affect everything else relying on animal_legs and animal_sounds # in microbenchmark. animal_legs_2 <- copy(animal_legs) animal_sounds_2 <- copy(animal_sounds) # data.table for dtA test animal_legs_dt <- data.table::data.table(animal_legs, key = "n") animal_sounds_dt <- data.table::data.table(animal_sounds, key = "n") microbenchmark::microbenchmark( base = base::merge(animal_legs, animal_sounds, by = "n", all = FALSE), ij = dplyr::inner_join(animal_legs, animal_sounds, by = "n"), dt = { animal_legs_dt_test <- data.table::data.table(animal_legs, key = "n") animal_sounds_dt_test <- data.table::data.table(animal_sounds, key = "n") animal_legs_dt_test[animal_sounds_dt_test, nomatch = NULL, on = .(n)] # inner join }, dtA = animal_legs_dt[animal_sounds_dt, nomatch = NULL, on = .(n)], as_dt = { animal_legs_dt_test <- data.table::as.data.table(animal_legs, key = "n") animal_sounds_dt_test <- data.table::as.data.table(animal_sounds, key = "n") animal_legs_dt_test[animal_sounds_dt_test, nomatch = NULL, on = .(n)] }, set_dt = setDT(animal_legs_2)[setDT(animal_sounds_2), nomatch = NULL, on = .(n)] )

​@@Riffomonas I've tried replying numerous times, but my comment gets removed each time. I think it doesn't like the code snippet I'm trying to share... Anyway, I made synthetic datasets ~50,000 rows long, where each row is a unique group so that it is comparable to your fasta data. dtA is consistently fastest, followed by setDT and as.data.table. One thing I had to control for was using a copy of the dataframe for setDT (e.g., df_copy <- copy(df)) because setDT works in place. If you use the same df reference for all items in microbenchmark, you run the risk of setDT changing your dataframe in place to a data.table. Then any subsequent runs with data.table() or as.data.table() take the same amount of time because df is already a data.table. Maybe youtube will let me share the code as a github link... : github.com/mrguyperson/joins_example/blob/main/R/joins.R

Thanks! Glad you enjoyed it 🤓

Great - thanks for the feedback. For now, the input to the phylotypr functions will be data.frames, but they should work fine if people provide tibbles or data.tables. The output will be base R structures like lists and character strings.

> class(iris) [1] "data.frame" > x <- tibble::tibble(Species = iris$Species[1]) > class(x) [1] "tbl_df" "tbl" "data.frame" > iris |> dplyr::inner_join(x) |> class() [1] "data.frame" > iris |> dplyr::inner_join(x) |> tibble::as_tibble() |> class() [1] "tbl_df" "tbl" "data.frame"

Thanks - I hadn't seen data.table::merge. That would simplify things considerably

yeah bash can sometimes do different things with single vs double quotes. a backslash can be useful for escaping quotes if you need to have quotes in quotes

@@Riffomonas this was without the URL in any quotes, like your script.

Hmmm, I'm not sure - why are you giving metaMDS shared instead of a distance matrix? Could that be the difference between what you and I are doing? github.com/riffomonas/distances/blob/main/code/nmds.R

@@Riffomonas But at 12:27, you were using nmds<-metaMDS(shared, autotransform = FALSE), using shared as input?

The rest of the video goes on to say that the defaults were not ideal and that rarefaction of the data was necessary

@@Riffomonas I see, thank you Pat!

Thanks for watching check out these two videos: kzbin.info/www/bejne/fWa0lJ6nppp4qNk and kzbin.info/www/bejne/nmiyo4mjiZx9l5o

My pleasure - thanks for watching!

lol - thanks! I'm glad you found this helpful

I'll see about circling back to data viz types of videos. I've made a bunch of them and am not sure what I might have missed 😂

You might be interested in using mikropml from my group. I made a series using it about 2 years ago. Here's a link to the playlist: kzbin.info/aero/PLmNrK_nkqBpKpzb9-vI4V7SdXC-jXEcmg