Test Multiple Variables at Once to Optimize Anything

Check out my sponsor Brilliant, free for 30 days (and get 20% off an annual premium subscription!) by using this link: brilliant.org/nighthawk
In this video I explore a multivariate experimental method called orthogonal (Taguchi) arrays.
Thanks to Guillotined Chemistry and Xoltri for their contributions that helped make this video happen. You'll find lots of useful info on their channels, especially in reference to making senko hanabi:
/ @guillotinedchemistry
/ @xoltri
Here is my own senko hanabi playlist: • Senko Hanabi
Thank you so much to those of you who support this channel on Patreon! Your support really helps give me confidence to spend my time researching projects that are of value for more than just video views. Shoutout to my top patrons: Eugene Pakhomov, Peter Gordon, Evan Hughes, Teague Lasser, Matthias S., Michel Pastor, PabloXIII, Parker Jones, Simone Chiesi, Steve C, Yanko Yankulov, Walter Montalvo, Carl Katzenberger, Damián Arrillaga, Dan L, Edward Unthank, Gusbear, Jon Hartmann, Kejie YU, Kirk Werklund, Lisa L, Mark Roth, PabloXIII, Santiago Perez, Steve C, Thibaud Peverelli, Tristan Tonks, WilSkarlet, Yanko Yankulov
/ nighthawkprojects
Thanks everyone for watching!

[@guntereisenherz9102]

As a programmer i also want to highlight the method of trying the same thing again, without changing anything, expecting different results.

[@janszreniawski]

The definition of insanity...

[@TrySpy]

That's all fun and games until it works o.0

[@andrewferguson6901]

Gpupdate and reboot a 3rd time, maybe that'll fix it

[@TallMilk]

You forgot mindlessly scrolling for hours through your previously perfect code looking for an error

[@bensawyer3778]

​@@TallMilk Add to that all the times, when looking over that "perfect" code, you find yourself saying "wtf was I thinking when I wrote this part"

[@nlabanok]

I spent 20 years working materials and process development in the semiconductor industry and these design of experiment (DOE) methods are foundational to quickly identifying meaningful improvements to manufacturing processes.Great work articulating and demonstrating these methods.

[@koustubhavachat]

Please recommend course so we can learn more

[@stevestarcke]

I learned from this book. Practical Experiment Designs for Engineers and Scientists by William J. Diamond

[@garrysekelli6776]

This is a very generic reply. 1 you place yourself as an expert in the field. 2 you don't discuss any relevant point. 3 you don't present any conclusions. Typical bot comment.

[@dimm__]

@garrysekelli6776 are you a fun guy? yknow, at parties?

[@garrysekelli6776]

@@dimm__ ask urmoomson.

[@isavedtheuniverse]

I'm gonna go out on a limb here and guess that this is the only video in the entire universe that covers boiling eggs and making Senko Hanabi in the same video.

[@VilleWitt]

What about boiling eggs using Senko Hanabi? Mega Senko Hanabi? Mini Eggs?

[@knight_lautrec_of_carim]

On earth, maybe. But there's a few planets that do this regularly.

[@NeonDripKitty]

what if we make the lamp black with burning eggs hmm

[@bastiaan7777777]

Is there a tiny hole drilled in the shell before boiling...?

[@Xfactor7430]

IKR.. no hole no worky worky. Eggz explode when no hole.👍👍

[@mhuebner1000]

As an engineer who spent many years in R&D, I just wanted to say thanks. Introducing system optimization and DOE concepts can help people understand that there are so many tools available to them. And, as you illustrated, their use is not confined just to labs and manufacturing. Your clear examples and approachable teaching style make your content so impactful to the community. Thanks again.

[@ronwoodward716]

I have used these methods in industry. Your presentation and explanation of the methods are really wonderful. I can see this inspiring many young scientists to improve their methods and develop better ways of doing nearly anything.

[@Nighthawkinlight]

Thanks for the encouragement!

[@IdealisticDog]

Exactly the feeling I'm getting watching it. Drive to improve the way I measure effectiveness in both work and personal life.

[@mumiemonstret]

And not only young, I can attest to. My husband who has been working with wastewater treatment processes for 20 years asked me today for the link to that video about test optimisation that I mentioned a month ago.

[@DucktorThallium]

My father (an engineer) tried to explain this to me (a chemist) a couple of years ago. Unfortunately, he didn't understand how it worked and couldn't explain the process. This video was extremely informative and makes me want to study up on Design of Experiments, since I have heard that it is an extremely useful field for scientists. Thanks for sharing this with us!

[@Chad-Giga.]

Chemistry sucks

[@jeremybub2]

Another commenter has mentioned Bayesian Optimization, I just wanted to expand a bit on what the difference is, and why you might prefer a more general optimization framework over Taguchi arrays. First, it's important to mention that Taguchi arrays are only accounting for pairwise interactions: for any two variables we choose, all possible combinations of those two variables will appear in the array. In the wikipedia article on Orthogonal Arrays, they generalize this idea to choosing t variables, where t=2 is the Taguchi array, and t=n is the full factorial analysis. But we could also generate arrays for intermediate variables of t. These arrays would be helpful if we expect up to t variables to interact together to create an effect, which is often the case. But any time we are choosing t

[@Metal_Master_YT]

I'm just commenting because I've made similarly long comments before that took a lot of hard work and I hate it when nobody bothers to read it xD

[@TheHectorOg]

Yeah, gonna look at this.

[@jacksonpercy8044]

@@Metal_Master_YT I can definitely relate, lol. It's fascinating to learn how deep the field of Design of Experiment (DoE) goes.

[@Metal_Master_YT]

@@jacksonpercy8044 I just want my experiments to work the first dang time...

[@eric.is.online]

I learned about DoE during my PhD, why it's not part of the curriculum at high school is beyond me. So glad to see this approach being shared in the community cause it's a real game changer.

[@edwardnardella6854]

It should absolutely be taught in high school, so many people perform experiments at some point in their life.

[@coler154]

@@edwardnardella6854 high school is only for making obedient workers

[@askquestionstrythings]

High school graduation requirements are sadly still focused on 20th century factory floor workers minimum requirements. Sure they throw some "college readiness" stuff in the mix but those are often fluff courses where students just need to show up. I totally agree that statistics, DOE, basic calculus, intro physics and intro chemistry should be mandatory requirements for k-12 education. Those courses literally change how people think. Sadly with math like statistics and calculus the pedagogy model currently used makes teachers afraid to teach those topics before college to most students. There is too much believe that every student has to learn math through nearly the same method that humanity discovered each math area. A long and laborious process with little reward.

[@TheDuckofDoom.]

It's hard enough for most students to grasp the much more basic concept of isolating a single variable. Though I agree this should be mentioned in the basic intro to science class, and then later fully covered in a highschool AP class and any college 101 science class.

[@carsonhunt4642]

@@askquestionstrythings Most ppl don’t want to admit that teaching kids all this stuff is useless. Past the basics, most ppl aren’t smart enough. And the ones that are clearly see how their talent goes to waste in and after college. So in the end it’s most pointless endeavor. Welcome to endstage capitalism where all we do is manage the decline of things.

[@derrick4804]

I could use this to improve my air cannons in the future. I've always wondered how much air pressure affected velocity, and to what extent. But along with that there's also tank volume, projectile mass, barrel length, valve response time, flow rate, and how tightly the projectile sits in the barrel. Now I can test all of these variables without doing hundreds of tests!

[@GimmeOo-mox]

I have an idea for you if you'd like. People who do gold sniping use a simple bottle and tube called a "Snuffer bottle". If you could reverse the mechanism of a squirt gun so that it pulled in water and material instead of pushing it, you could replace the traditional snuffer bottle. Call it a "Snuffer Gun". If you do this and get rich, please don't forget about meeeeeeee😅.

[@whynotdean8966]

@@GimmeOo-mox It already exists, it's called a hand dredge :)

[@GimmeOo-mox]

@@whynotdean8966 I didn't know... But now I know. Lol.

[@GimmeOo-mox]

@@whynotdean8966 Actually, I knew about the hand dredge but the concept I'm thinking of is much smaller and would replace snuffer bottles.

[@ldcent8482]

If I'm remembering fluid mech, you can find the relationships between some of those values with dimensional analysis, and then run an experiment to solve for the coefficients of your setup. That would give you performance curves that you can use to optimize the design.

[@RebelCowboysRVs]

As an egg farmer that tested egg pealing extensively, you are correct about egg age being the biggest factor. The factor you missed is cracking the eggs before cooling. My theory is flash cooling only loosens the shell if it is broken an can move as it contracts. Some say its because water can get in. But either way, cracking before cooling seems to help

[@jasonnickel384]

Eggs peel perfectly EVERY time if you cook them in an InstantPot (or any other pressure cooker).

[@RebelCowboysRVs]

@@jasonnickel384 Dont think I have ever pressure cooked eggs before. I hace both a stove top pressure cooker and an instapot. But just use them for big meals.

[@gg-gn3re]

@@jasonnickel384 the best way is to steam your eggs, just use any vegetable steamer or a colander in a pot of water to steam them. Then refrigerate them assuming you make many at a time to eat on some other days down the week.. Once they are cooled the shell basically falls off. If you crack immediately after cooking it's still easier than any boiling method except maybe the vinegar. also it generally uses less energy to steam far less water than to boil a lot more water

[@jamesyoungquist6923]

Unfortunate that the biggest factor is the age of the egg, and that wasn't included in the experimental design...

[@RovingPunster]

Whether or not you crack before or after the post-cook ice bath is a factor I hadnt considered - I will definitely try that, thx. I do have something to add to all this, which is (1) the role of the membrane (which lies between the white and the shell), and (2) the cracking extent & technique used, and how it somewhat relates to (1). As for me, im a retired engineer who's just used to thinking analytically. My household is only 2 adults, but I love eggs, so we often use as many as a family of 4+. (1) Membrane: I believe this correlates with the degree of freshness factor, because as an egg ages the subtle permeability of the shell allows for some moisture loss over time, which in turn both increases the telltale size of the bubble at the big end, and reduces the degree of surface vaccum (i dont know the correct term) between the shell and the outside of the membrane. I believe this is the main mechanism behind why fresh eggs are noticeably harder to peel than less fresh examples. 2) Cracking Extent & Technique: First, consider how the face of a freshly cut slice of raw waxy potato tends to seize onto the smooth flat side of a chef knife (surface vaccum effects). With that in mind, i suspect the egg membrane of a hard boiled egg can have a similar effect on how easy/hard an egg is to peel to some degree. The next time you peel a bunch of eggs, pay attention in any given moment to whether you're peeling a piece of shell away from the membrane (which is harder), or if you're peeling shell and membrane together away from the white (which i find easier). My point is that ive found the shell comes off easier if you make sure you're UNDER the membrane every time ... which brings me to my larger point of cracking extent and technique. When dealing with surface vaccum adherence effects, the 2 biggest factors are the surface area (read: size of the current shell fragment being peeled) and the surface tension of the moisture between the cooked white and the interior side of the membrane ... if you think about it, cracking the shell more thoroughly not only reduces surface contact area with any given piece of shell, it also creates more points of compromise in the membrane, which allow it cling less to yhe white and also to tear apart more easily. A last subfactor to consider might be adding something (food grade obviously) to either the cooking water or ice bath that reduces surface tension of the moisture between cooked white and membrane. Soap, surfactants, emulsifiers, ph, temperature and even electric current can all affect that - not all seem appropriate for use with eggs, but clearly is an angle worth considering. Regardless, I think that may explain how a little vinegar helps. So, bottom line: For easier peeling, I believe it helpful to fragment the shell more thoroughly, but without compromising the egg white (if pushed too hard). It might be fun to play around with ultrasonics, but since i dont have that handy, i content myself with just cracking the shell gently but as thoroughly as possible. I start by laying down a single sheet of paper towel, for traction and absorption. Then I usually crack the big end, tilt about 20 degrees, then rotate it several times with gentle pressure to extend the cracking in a ring around the pole. Then I quickly repeat at the other end. Last, i turn the egg on its side, crack the equator, then gently rest my palm on it and roll the egg back and forth at least a half dozen times, gently rocking my hand to one side then the other until the shell is thoroughly cracked EVERYWHERE, from pole to pole. Then I roll a few more times until I feel the shell and membrane noticeably loosen. Then I peel. With practice, and eggs of optimal degree of freshness, im usually able to peel several eggs per minute. One caveat: if the eggs are a bit older than optimum, the white begin to weaken and are prone to tearing away with the membrane, so there is a definite bell curve to optimal freshness for peeling ... you dont want too fresh or too old, but rather something in the middle. I suspect the sweet spot is fresh enough for the boiled white to still be firm (and less prone to tearing), and old enough for moisture loss to create some space between the membrane and shell, but not so old that the white weakens and becomes prone to tearing. Sorry to ramble, but I think about stuff like this all the time.

[@ostahlarune]

Senko, microspheres, and process improvement all in one video is such a heavy hitter, this is peak KZbin content

[@morganrowlette678]

YES

[@arshadmohammed1072]

I feel so guilty for never commenting about design of experiments (DOE) before. I use these daily in research, Taguchi array is the one I use to introduce persons to DOE. There are so many different methods, especially for non-linear (non-orthogonal) analysis. Another cool one is the Mixture DOE, which is mainly for recipes.

[@Nighthawkinlight]

Looking into this topic I realized there is much more to learn, but Taguchi arrays at least seem like they should be common knowledge because of how much you can learn from them without going beyond very simple math.

[@cuberille]

@@Nighthawkinlight Kudos for even attempting to explain DoE in such a limited time. Not sure how far down the rabbit hole you've gone, but you can take your results and plot them in a 3d graph in excel to better visualize where the sweet spot is.

[@marcolivio3128]

In case you are interessed I suggest to take a look at specific statistical software that would help you in gathering more info from your data. In particular Minitab is the preferred option in many industries including the one in which i work (i.e. Pharmaceutical development)

[@davebennett5069]

@@Nighthawkinlight "should be common knowledge" - i feel the same way about probability and statistics. it's so much more useful than trigonometry / calculus to most of the population that i think it's criminal to have public schools push students into trig / calc while sometimes not even offering prob / stat. it's the same kind of thinking that would lead one to a more intuitive understanding of the benefits of a taguchi array, IMO

[@steaming_mangos]

@@marcolivio3128 do you know of any free alternatives?

[@MaxWithTheSax]

How did I not know about that. It's so useful. Thanks for enlightening me.

[@Nighthawkinlight]

That's what I said when first hearing about this!

[@EctoMorpheus]

I have literally worked as a scientist and had never heard of this. In computer science we mostly just stuck with changing one thing at a time and slapping on more computing power :| Of course we also did random grid searches and then computed statistical tests on the results to still try to figure out something about the influence of each variable, but it wasn't nearly as elegant as specifically designing an experiment beforehand... Thanks a lot for this video! Looks like a super useful skill

[@brianmrzyglod2073]

As a process engineer, I've completed plenty of DOEs ... we use a statistical software call JMP. Among other things, JMP will design and analyze DOEs, telling you which of your factors are significant. It even has prediction sliders, letting you virtually adjust the levels for each factor.

[@gorak9000]

Haha, you work for the big I company too, huh?? Honestly, I f'ing hate JMP through - the most convoluted scripting language to actually get anything working. Once me and a co-worker spent an entire afternoon trying to figure out how to script something that took 2 seconds to do by hand, but was nearly impossible to get running in a JMP Script... and yes, we looked at the "journal" - it didn't help... apparently everything DOESN'T get logged to the journal like they claim... eventually we figured it out by googling and combining 3 different solutions from the JMP forums, but my god was it an ugly solution. Something that you could do in Python with Pandas easily without JMP'ing through so many hoops...

[@alantripp6175]

Same here great fun loved this part of the job.

[@askquestionstrythings]

We are saddled with using minitab for our DOE analysis but in school it was all done in R. I wonder how JMP compares.

[@DeShark88]

​@@askquestionstrythingsminitab holds your hands a lot more. JMP seems a bit more powerful, but the results are harder to interpret without a solid statistical foundation.

[@DeShark88]

​@@gorak9000haha, sounds familiar. But it's not just Intel that use JMP; it's used fairly widely in the industry. And yes, JSL sucks.

[@lambda_calc]

This is one of the best KZbin channels. The clear method of communication, worthwhile projects, and deep insights like this make me feel like I get more from these videos than most others. Keep it up, I love what you're doing.

[@avelkm]

As a MSc in applied physics (20 yrs ago) with strong experimental part of my education I'm speechless that this method was never taught in uni. My mind is seriously blown. I will use this method in next week experiments to get maximum color separation in epoxy pour (it's heavy temperature dependent, at least 3 variables to play with). Thanks!

[@Nighthawkinlight]

Let me know how it works for you!

[@MrAlFuture]

This was fantastic! It's inspired me to have a play with some genetic algorithms using this type of approach to programmatically evaluate the successive generations of code changes

[@storminmormin14]

Holy crap that’s genius.

[@riuphane]

Talk about a method for optimization and bug catching!!!

[@an_asp]

I work with evolutionary algorithms for a living, and tuning the parameters is definitely one of the most difficult parts about using them. There are so many interacting parameters that can dramatically affect their behavior. I've tried things like adaptive parameter control, "self-adaptive" parameter control (which is a fascinating technique of letting solutions in the EA evolve their own parameters as it runs: if e.g. a high mutation rate was helpful in producing children with a higher fitness, then that high mutation rate gets passed down to the children), and even using a second EA to tune the parameters of the first EA in some kind of meta-evolution. But for some reason, despite studying this type of experiment design in school, I've never actually tried it out in practice for parameter tuning. I think I never realized how much potential time savings there was when you have so many variables and levels, since the examples in class were always tiny for demonstration purposes. I guess I should give it a try too.

[@YogeshKumar-js7tx]

Good

[@TheHDreality]

I was thinking about using this as a mutations strategy as he was explaining it, though I think you're in luck! If you've ever heard of Covariance-Matrix-Adapatation, it's doing something closely related, after you generate a set a candidates, you select the top n candidates and create a covariance matrix of all their features, and then generate the next generation to be a spread of values with the same covariance distribution. It doesn't have the same kind of rigour but it has excellent coverage and convergence. If you wanted to adapt an evolutionary strategy from this, you could generate a Taguchi Array for low, medium, and high values of your n dimensions, and then use the covariance matrix trick to define your values for low medium and high. Is there anywhere I can follow your progress in case you pursue this? Because I'm extremely interested in novel genetic algorithms, and CMA is already super interesting for being state of the art while having very few hyper-parameters to deal with.

[@--sql]

According to J. Kenji Lopez-Alt who ran a ton of experiments on egg peeling, the number 1 variable that makes eggs easier to peel is the temperature of the water in the pot when you start boiling your eggs. If you place the eggs in a cold pot of water and slowly bring it up to temperature, they will tend to be much more difficult to peel. Always boil the water first, then place the eggs into the boiling water.

[@claudiu7909]

Good to know. Also, if you don't peel the eggs after boiling them and leave them for a later time (next day), from my experience they will peel easier if you keep them in water. My speculation is that the water keeps the membrane right under the shell hydrated so it doesn't stick to the egg white.

[@riuphane]

Glad to see another Kenji follower. His experiments and results have been the basis for many of my own recipes and tests, especially when it comes to sous vide! I immediately thought about his egg testing too!

[@microwave221]

This explains to me why I've struggled with it so much, and I'm excited to make deviled eggs with an hour of agonizingly removing shell superglued to the whites

[@grabasandwich]

One time when I put them in boiling water they cracked and some of the white oozed out. Do not recommend.

[@grabasandwich]

@@busimagen thx I will give it a whirl. I do have one of those small machines to make hard-boiled eggs that does the same but it also starts from cold.

[@Jinakaks]

Multivariate analysis was one of my favorite things about my STEM classes. It's a shame it's been missing from hobby science!

[@gnusamgnu]

As a hobby, I play to do fermentation sodas. I stuck to the same recipe since each batch takes days or weeks to be ready. With this method I might be able to improve my results. Same when you grow plants from seeds!

[@manatoa1]

I thought the same about beer brewing. There are so many possible variables and long cycle times. Or bread making, for that matter. This is exciting!

[@cerberez]

I'm an engineer and i have never used this method! Now I'm excited to use this for my experiments!

[@askquestionstrythings]

I had a special class on DOE which covered the statistics and 2k factorial design for one of my technical electives. It's definitely something more engineers should study.

[@KG4JYS]

Same here! It's hard to believe it will work as well as the video shows. I will find out for myself.

[@OfficiallySnek]

I have never heard of this method for experimental design. Thank you for sharing this will significantly speed up data gathering and even reduce the amount of experiments needed to be done saving resources

[@hedgeearthridge6807]

I'm an industrial quality analyst and I am OBSESSED with Genichi Taguchi, especially his Loss Function. It's joked in QC that you have to have a doctorate in mathematics in order to fully understand his ideas, even though he wasn't a doctor, just an ingenious engineer. He's a founding father of Quality Control, as well as Kaoru Ishikawa (inventor of the Fishbone Diagram), Dr. W. Edwards Deming (the "father of quality control"), Dr. Walter Shewhart (the inventor of the Control Chart and a mentor to Deming), etc. Thanks for helping me understand Taguchi's experimentation method! I was never going to get it by just reading. the math is just insane

[@MarkStoddard]

One I came to love when homebrewing is the "triangle test". Take 2 of one sample, and 1 of another. If a tester can't determine which is the odd one out, there's little to no difference, and if they can, ask what the difference is. When it's just A vs B, a person may be biased to think they can detect a difference.

[@askquestionstrythings]

2k factorial and design of experiments is amazing. Sadly people (especially in industry) are not willing to make the effort to utilize these methods especially when the tests are time consuming or expensive. You did a great job explaining the benefits.

[@MyHandleIsGood]

That's an extremely useful tool that really needs to be more well-known. Hopefully you use your new-found knowledge to optimise some of your older projects too. I've always steamed my eggs and had no issues peeling them, but adding vinegar to the water when boiling them does make a lot of sense to do.

[@marcfruchtman9473]

Steamed... hmm, ok I will give that a try as well... Thanks for the tip.

[@manonfire285]

This is a lovely and uncomplicated way to do DOE, especially when your variables are discrete (or, as you did for this video, discretized). If you're interested in optimizing under continuous changes, which is probably especially relevant for chemical recipes, you may look into Bayesian Optimization. The method can be tuned depending on your variables but would already mostly work straight out of the box using a sensible default configuration. Let me know if you want to discuss this!

[@steaming_mangos]

I am interested in learning about this, do you have any resources you would recommend? Everything I'm finding online revolves around data science but I am interested in learning it in the context of chemistry.

[@gordonspond8223]

This type of experimentation is very common in industry and has been around over 80 years. I am always amazed this isn't used more in scientific research.

[@temp_name_change_later]

I could see this working really well for recipe testing. A lot of cooking KZbin channels talk about how important it is to only change one variable at a time while recipe testing, but I imagine that this could speed things up by a lot.

[@sagopalm279]

it always surprises me when each video you post is better than the last

[@Oroborus710]

This is why we love NightHawkinLight 😁

[@mkeyx82]

You should put your surprise into an orthogonal table and see what you get.

[@dankelpuff8381]

I know absolutely nothing about the method you used to improve your sparklers..however, the second I saw your score sheet at @20:00 alarm bells start going off inside my head. You are scoring your sparklers on non uniform parameters which which in itself is not a mistake but because your values are so different you are effectively washing out all contribution from effects which have a numerically smaller measurement. This is a very important factor to consider and correct for when using something like PCA on a larger dataset. Image for example if we want to guess a persons likely hood to get a heart disease based on their eye color, height, weight and age. If we measure height in millimeters rather than centimers we will have HUGE values which will overshadow the weight in Kg and the age in years! Even if we only measure in centimeters or meters we still have vastly different scales. Furthermore we cant assign numbers to eye color as 1=green and 2=blue because then blue eyes are more important than green just because we chose so! This means in order to have a well rounded sparkler you must standardize your scoreboard no matter how you measure success. You do this by subtracting the mean value of a score column from itself and then divide by the standard deviation. Do this for each column of results and now every column will have the same equal weight in your scoreboard. This would allow "climbing ember" scores to have an equal impact on your evaluation compared to the "bursting sparks" score no matter how big or small the numbers you assign are.

[@Nighthawkinlight]

Yes you're exactly right. I weighted one sparking stage far heavier than the others. I did this on purpose but did not find a place in the script to talk about it. In this case I wanted to record the data from all stages so I could discover what contributed to each, but overall my priority for considering success was based on the first and most difficult to achieve stage.

[@dankelpuff8381]

@@Nighthawkinlight Fair enough :D

[@naviinprabhu2468]

I don't know whose gonna read but I'll leave it out here. Systems design engineer here. I do DoEs day in and day out, Taguchi method is an awesome tool to narrow down the design search space. When there are a handful of inputs Xs for a measured output Y you can control and intuitively understand, this is all you need to optimize . When there are a large number of Xs, there's something engineers do which is called parameter sensitivity/pareto analysis. There are many fancy ways to do this pareto chart but I try to keep the intuitive part alive. I min-max normalize the inputs and output between 0 to 1 and do a multiple linear regression. The coefficients of Xs gives its impact on Y and its sign gives the direction. This helps eliminate less influential inputs to further reduce the dimensionality of search space. Also, a cool way to visualize them is using a parallel-coordinates chart 😁

[@buschtoens]

This gave me a whole new perspective on how to run software A/B split tests to optimize customer conversion rate. Thank you!

[@Nevir202]

Oh no! the last thing we need is advertisers getting better at brainwashing! LOL

[@sangomasmith]

Sweet, I did my masters on using Taguchi arrays to improve a biotech system. I've long thought that this process doesn't get enough love...

[@jtadevich]

Aha! Then you are the person I should ask. At 12:08 he shows us an example of an array of 4 variables, and 3 settings. It dropped the experiment count from 81 to 9. I noticed the result is the square root of the original count. Are there simple equations that will tell the reduction in experiments needed by using Taguchi arrays? Note: if the explains it in the last of the vid, nevermind the question. I have to go before the end.

[@sangomasmith]

@@jtadevich So this was a while ago, but from memory there's actually an empirical look-up table that you use to find the right array size for your experiment. There accordingly isn't a strict mathematical relationship between number of variables and number of experiments. Taguchi methodology is supposed to be used in industry to improve processes, so it's kind of rough-and-ready rather than being mathematically or statistically rigorous. It's biggest advantage is simplicity - allowing you to rapidly identify important variables and zoom in on the optimal region for that system.

[@jtadevich]

@@sangomasmith thanks. That is awesome. I have studied science for a long time, and realizing how much time this method can save is astounding to me.

[@lewismassie]

This is incredible. It's like seeing a huge chunk of the world come into focus. A major part of how I think about problem solving has been rewritten in watching this. Thank you so much

[@eliorbilow8797]

This is possibly one of the best videos I've seen on this platform. It reignited my passion to tune and refine a project (3D printer) that had me demoralized for ages because this method showed that I can combine multiple tests to reduce time and filament used for testing.

[@CraftAero]

As soon as I saw the title I chuckled, "Taguchi Method". You did a good job of introducing the topic to lay-people, with a VERY simple and relatable experiment set. (I will certainly use some vinegar from here on). On any more sensitive ($$$) projects there are SO many more support elements behind defining the DoE and then interpreting / validating the results. PS: in the 80's we didn't have software for this... charts / calcs were done by hand.

[@GuillotinedChemistry]

Uh-oh... Brilliant better watch out... I think Eggland's Best is angling for a sponsor role! 😂 Seriously though, thanks for the shout-out and introducing me to Taguchi... It has absolutely changed the way I look at experimental design.

[@edwardpaulsen1074]

I slowly discovered that multivariable table over the course of several jobs in different industries... once I got that figured out, I also discovered another side effect of working in so many different industries via my automation and robotics skills... I was able to cross platform experiences and bring fascinating new insights into seemingly unrelated fields. While focusing into a niche position can help you gain a mastery of a given craft, there is also a strong value in the "Jack of All trades, Master of none" approach as well. By not getting stuck in a proverbial rut in areas that have "always been done that way", I was able to leapfrog things into new areas that provided substantial gains rather than the incremental ones at larger time and expense. Baseline idea is to definitely pursue whatever specialization interests you most, but also do not sacrifice reasonably passing knowledge in other fields... who knows what could be that "Eureka" moment gleaned from something seemingly wildly different then your current course or field.

[@Nighthawkinlight]

100% true.

[@IdealisticDog]

RE: "How did you not know about this?" -- Sure, plenty of people know about this methodology, but we live in a time where "common" information still requires a catalyst to gain awareness of.. I work in data analytics, and most of the education around this is so dry and sterile that it failed to gain purchase in my mind for applying more widely. It's easy to see ignorance of techniques as an indicator of incompetence or laziness, but I think that perspective lacks appreciation for just *how much* information is out there. Thanks for the video, and for following your passion in sharing and expressing your excitement.

[@renaissanceengineering-lee4210]

this was a WAY more engaged presentation of DOEs than my manufacturing classes, six sigma classes, and general online reading. well done!!

[@ares395]

This blows my mind tbh. I knew there must be a better method but I was never smart enough to figure it out. Might help 3D printing nerds with perfecting their quality without wasting 100s of hours and a ton of filament among other things. Also, I might finally get into cooking due to this haha. I don't mind eyeballing things 'to taste' but having a recipe on paper that is modified to your liking would be cool. Always love your videos because they are super easy to follow, I still use the button trick to hold pieces of fabric together.

[@greese007]

Experimental design should be part of every technical education, along with statistical analysis. After I took a job doing R&D in polymer manufacturing, I soon realized that my education had not prepared me for the job. I taught myself with books and offsite seminars, and this soon became one of my most-used skills. I now believe that college professors should do a turn in industrial settings before teaching engineers and scientists what they will need. KZbin videos like this provide a valyable service.

[@dinoflame9696]

Length of cooking time is the most significant parameter. All eggs peel easily because you hard-boil them for 11 minutes. The inside is semi-liquid, the outside is mineralized. When you heat the egg, the shell expands and the longer this goes on, the more the shell fragments consolidate into larger pieces which are then easier to peel off. When also cooled off after cooking for a long time, the outer shell is already consolidated (and thus remains intact as it contracts with no structural weaknesses causing it to fragment), whereas the liquid inner part contracts as a liquid uniformly. This effectively "peels" the white/yolk from the inside of the shell. At least that's my hypothesis as a geologist and amateur egg-boiler.

[@ItsjustmeElisa]

I just realized that your channel is the one I've been continuously watching on and off basically since shortly after youtube started...and FAR longer than any other channel by like probably 8 years.😅 Really speaks to how your approach to topics even as the videos and topics shifted from me going like "ohhh shiny fireworks" to "hm this isn't something I ever thought or cared about but I'm thoroughly enjoying this!" over the years just captured my imagination for like 15 or so years at this point if I had to guess. Even though I have been subscribed for a fairly short time that's because I liked the idea of most creators and topics to just percolate back into the subconscious until they organically reemerged after months or years at which point I could go on a watch spree hehe. But as youtube, upload schedules and monetization changed so did my viewing behavior...also I'm getting older and there are so many quality channels now I just can't keep track anymore 😁 Anyways it was just a fun realization to have that this is the only channel for me that I've been watching for this long and I wanted to say thank you and show this tiny bit of appreciation, for whatever it's worth, and hope you have a great day!

[@Nighthawkinlight]

That's nice to hear, thanks for sticking around!

[@gorak9000]

KZbin's been around a LOT longer than 8 years...

[@ItsjustmeElisa]

@@gorak9000 I said I had been watching his channel for 8 years LONGER than any other channel I'm currently watching, around 15 or 16 years in total. 😉

[@mailleweaver]

This does look extremely useful. Especially when testing things like ratios where it's literally impossible to change just one thing at a time. When one goes up another goes down.

[@Primalmoon]

While I see everyone else is piling on how useful this is, do realize that this technique comes with downsides. 1) Because this isn't widely known, it is much harder to explain your experiments because you always have to explain this method first in case someone isn't a regular viewer of your channel. 2) Because the results are derived by comparing to the other experiments, you have to do all of the experiments the table lays out and then do the analysis. Assuming you can do all of them at once that's great. But if you can't because of limited equipment / shelf space / whatever / initial money and have to setup multiple batches and go through multiple rounds of this, by changing a single variable at a time you can possibly learn things before all of the others are done. Maybe you figure out that you can abandon a particular line of research or that your experimental design is flawed and needs to be reworked.

[@nicklockard]

Can confirm. I taught factories to use multivariate DOE studies (Design Of Experiment). It does require good software like JMP or Minitab statistical analysis software, but quite often, it's the ONLY way to develop better, cheaper, and faster processes. Classic science where you only vary one parameter can only solve about 5% of real world problems!

[@matthewluttrell9413]

I've only seen this in a text book, your examples were top notch and explained vary well! You're doing a great job with these videos!

[@lanceanthony198]

Nothing like a new nighthawkinlight vid to make my day better

[@davidfetter]

It's delightful to see design of experiments get such a clear and concise treatment. There are a LOT of holes to fall down, and you managed to miss all of them. Well done!

[@jstwinkles]

It blows my mind that basic methods like this were never taught in my undergrad engineering courses. I can't wait to get a chance to apply this to my debugging processes at work.

[@andrewevenson2657]

I'm a laborer in a steel foundry, and I applied to a quality control position cause I am capable of so much more than just labor. This orthogonal array method will be awesome for me to bring up during the interview!

[@descai10]

did you get the job?

[@neithanm]

What breaks me often is thinking I've settled a test that I use to compare to others and when nothing works I go back, re-do that test and realize something went wrong and had been using a broken yard stick. Especially with chemistry or cooking and other "flimsy" subjects where it's very hard (for me) to make 2 "exact" runs of a particular set of variables.

[@Mr6Sinner]

This is potentially my favorite video of yours so far.

[@espenrosenquist6683]

The advice of dropping them in cold water is, according to chef Pepin, much more complex and helps remove the sulphuric compounds (rotten egg smell) from the boundary between the yolk and the white, avoiding an unpalatable texture, colour and taste.

[@JanTonovski]

Very cool how the method of testing can make such a difference. Btw, I hope you didn't have to eat all those eggs alone 😂

[@Nighthawkinlight]

I did eat a LOT of eggs.

[@whifflingtove]

@@Nighthawkinlight Deviled eggs, egg salad, egg sandwiches, scotch eggs...

[@3nertia]

@@whifflingtove Egg gumbo, egg tar tar, fish eggs ... Sorry, couldn't resist the Forest Gump reference xD

[@RuvianCSPlays]

@@3nertia Scrambled eggs, fried eggs, poached eggs, eggs benedict..

[@fakestory1753]

His parrot take care of eating all of those eggs, don't worry about that.

[@thegingineer0]

Only a few minutes in but this couldn't have come at a better time as I try to perfect my conductive painting process.

[@beautifulsmall]

Fascinating and instructive. Those final bright light shots are stunning.

[@DaveEtchells]

Wow, that is amazing! I love looking at data and seeing significant information pop out of it. I’d been completely unaware of this method before; this is just *so* useful and important! Thanks!!!

[@noob19087]

I've been doing this in the kitchen for years. Because whenever I run an experiment in my kitchen, I'll have to eat the results, so I'd much rather make the same thing 4 times than 8. I always considered it a bit of a dirty trick, though.

[@Leo99929]

If your experiment is reactive enough I've found that robustly doing a design of experiment can make learning take longer. You're better off just playing with it and seeing what you learn without wasting time planning. Then you form some hypothesise to test and eliminate some red herrings/distractions. This is especially true of when it's design by comity. Just make sure you make SOME notes to remember what you did and what happened. You can go back and do robust structured experimentation when you have a feel for what matters and what doesn't.

[@jaydeep-p]

Your method can lead to your biases weighing in very heavily and may exclude some very effective combination that your biases dont conform to.

[@Leo99929]

@@jaydeep-p You're totally correct. Although I've seen PhD researchers just ignore results that conflict with their beliefs about how things should work. Scientific rigor can't eliminate bias. It's important not to fall into confirmation bias too. Test to see if you can find a way it doesn't work, in addition to ways it does. Those are arguably potentially just as if not more important than just what does.

[@AnonymousAnarchist2]

​@@jaydeep-pI follow a hybrid method, Start with the base that your moving from. play around and record what youve done and if it made an measurable change then do all of the variables together, and reccord what changes you find together. Then sus out what is causing your changes and how much. You can find some very compelling processes that way that would be totally ignored or missed, as well as save time by noting what absolutly changes nothing in two diffrent steps, one changes nothing alone, the other changes nothing combined.

[@DanielSaidov]

Well it only took him 10 years to realize he had formed a bias early on. And that's the risk you run with going fast and cutting corners. I say this mainly to myself

[@Jackham13]

DOE, design of experiment. Use this all the time. A great low level textbook for it is: An Introduction to Design of Experiments by Larry B. Barrentine. The hardest part is usually quantifying the results.

[@adamrichardson2227]

As a self taught software engineer for the last 10 years, this is an amazing new tool that I think will prove be exceptionally useful. I am going to share this video with my team.

[@lollotk882]

that was really useful to know, great video. Also, your passion for science is really inspiring and the video format just perfect

[@airsoft3563]

I remember how i was in my early teens watching Nighthawkinlight and the king of random diy projects after school, amazing how im now in my mid 20 and still enjoy your videos!

[@alexandrevaliquette3883]

Congrat: you have now step up to the next Dunning-Kruger level. Even you, with your very open mind, clever, know it all, curious, etc There is always room to get surprised by our lack of knowledge!

[@lucdrouin2625]

Finally, thanks! This is such valuable information! Please continue your excellent work with all the due diligence you continue to invest in your world class instruction.

[@meyegui]

This is absolutely amazing. Never heard of orthogonal/Taguchi arrays before, and your demo blew my mind.

[@vibrolax]

I was introduced to the design of multivariate experiments in the early 1990's as an engineer producing optical metrology instruments used for semiconductor manufacturing. It changed my life as an engineer. Instead of endless meetings to prioritize which treatment conditions to vary, we could test them all in a Taguchi design and let analysis of variance figure out what really mattered. Higher resolution designs could be used to de-confound interactions between variables.

[@marcfruchtman9473]

Thanks for the useful info. I had paused the video just before he mentioned Taguchi and read this comment, then went back to see him literally using those exact words. (Good reinforcement).

[@ShmingsThings]

1) Experimental design is the backbone of scientific inquiry and I would eagerly watch more from you on this topic. 2) Now I think I need to dive into the math behind making these tables, because that's a personal interest of mine.

[@ImHighOnPropain]

Fantastic video! You make complicated things so straightforward.

[@TheEngineeringToolboxChannel]

Awesome video! I am an engineer by trade and work in quality. Have used this method with great success in understanding process inputs on quality outcomes!

[@sasharamirez2335]

fantastic video!!!!! I've done an informal version of the array stuff on my own, but I didn't have a clear approach in my head. This video makes it crystal clear!

[@Tictacpanter]

I appreciate the effort you have put into showing this method. This is great.

[@gabewrsewell]

this was very valuable, you’re one of my favourite creators on this platform. thanks again :)

[@conradwiebe7919]

Probably your most valuable video to date. Thanks as always :)

[@WilliamLee-bv4tv]

Always a treat to watch one of your videos. Eloquently narrated, and always some idea or take on a subject that i didn't know or hadn't considered. And I especially admire that you shout out other intelligent creators

[@bohemianist2082]

i loved this! thank you for articulation, this was exactly the right moment to stumble upon in my life with my biochemie experiments which leaning towards to be more complex gradually. hero!

[@gutsngorrrr]

This is very interesting, thank you for sharing. Also loved seeing the senko hanabi again, they are so beautiful.

[@y29k15]

This was fascinating. Thanks for covering this really cool idea.

[@manatoa1]

I think this will make a huge difference in a bunch of my hobbies. Thank-you so much. Challenges that were very daunting seem doable now.

[@Cybornut]

This is mind blowing stuff, and changes everything in how I would think of experimentations, thanks for the insight and the amazing tool.

[@davidbatchelor3264]

When you repeat your orthogonal array experiments, you can also use simplex optimization techniques to choose how much to vary each input.

[@prof.markbachman]

A great way to explain the use of Taguchi arrays in process optimization! Bravo!

[@NickCombs]

Taguchi arrays are new to me. I will try it out next time I need to debug code and there are multiple possible causes. Thanks for sharing this concept. It's sure to help a lot of people.

[@korebeast973]

I learned this on accident through baking and applied it to engineering projects on motors and engines. Most helpful combos of batteries, charge times, engines, start times, motors, and max rpm are a lot of variables. This process works amazing for testing anything

[@Leadvest]

This motivates the method well enough to make it seem like the necessity it is. Rather than the menial curiosity it feels like in most presentations. As per the MLA Handbook you don't need detractive statements in your scripts. Your work speaks for itself.

[@xoltri]

Wow, thanks so much Ben! You were the reason that I started making these fireworks and I brute forced my way through hundreds (?) of hours of making them to get to this point, a fraction of the time you spent I'm sure. This method is absolutely a game changer. It's hard to describe that feeling of trying to solve a problem and being completely lost with no direction on where to go next. So happy you've improved the recipe and I can't wait to try it, and to try to improve on it with this method!!

[@jaysonlee8625]

Your presentation and delivery has really improved over the years. Thumbs up!!

[@ragingwillie483]

watching was time well spent , as usual. THANK YOU!!

[@lel7531]

Very cool video thank you for sharing your experimentation process and your trade secret for sparklers !

[@miklov]

Thank you, this is very useful and your presentation is excellent as always!

[@ch1pnd413]

Fantastic methodology! Excellent video! Great project 👍🏻 beautiful outcome ❤

[@Domm_Dynamite]

This was a great video! Such a useful technique. I think I've witnessed this before but did not know the details or how the logic worked. Thank you for sharing this. I am going to be using this in the future!

[@pandit7130]

It was really an eye opening approach that I had never thought of. Thanks for exposing your viewers to such a great idea. ❤

[@chopper3lw]

This is an incredibly interesting method that has applications across so many domains. I had no idea this existed, so THANKS!

[@JasonPatz]

This is the best thing I've watched on KZbin in some time. Thank you for sharing it.

[@user-sb3wh3dd4v]

I'm impressed! Your explanation of this process is MANY times more concise and MUCH clearer than the dreary explanations I learned in school. THANK YOU!

[@JasonMisterDude]

Oh man. This is so awesome to play with! I could use this to make rapid improvements with tuning and drag racing. So much information being spit out already, up to us to analyze it!