Faster Subframe Selection With the Sigma Formula

[@MazzifLOL]

Thanks for sharing this. Simplifying and demystifying SFS is always useful.

[@MrGeert1972]

It could be all very interesting. But you make it like rocket science. I just do a quick rundown off my subs and those with clouds in it I remove. All the others will be average out by WBPP. I just use flats darks and light frames. And 5hst works just fine for me.

[@SKYST0RY]

I understand where you are coming from but the Sigma formula is very simple. You don't even have to understand it; you can just copy and paste it into your Subframe Selector script window and SFS will do the rest for you. The reality is that it is not a great idea to rely on WBPP to remove your flawed subs. It is very far from perfect. For one, doing this leaves you with no direct control of what standards are applied to culling subs. Another reasons is WBPP will consistently miss errors such as satellite and airplane scars. One should bear in mind the term "averaging out" is inaccurate. Averaging doesn't remove bad information; it simply attempts to bury it in good data, often with deleterious results. Contrary to myth, WBPP won't average out (or remove) flaws. The link below portrays an image of the Whirlpool Galaxy from a person who also told me that WBPP "averages out" flaws. A quick glance at the image revealed 5 satellite scars in the image. WBPP did not "average them out". A closer inspection will reveal more. It is a common misunderstanding that WBPP is a good culling tool. It is a good stacking tool, and I use it myself for stacking. As a culling tool, it is mediocre. www.astrobin.com/6bhx4c/

[@AstroIsland]

Great video. Any particular reason you reduced your Subframe Scale from 1.00 to 0.61?

[@SKYST0RY]

That should be adjusted to match the arcseconds/pixel as indicated by your focal length to sensor pixel size.

[@waltlickteig1105]

What is the name of the app you use to perform your visual inspection?

[@SKYST0RY]

It's called Irfanview.

[@gregerianne3880]

So useful! Thank you so much. I just have a couple of questions/observations about SFS and would like your opinion. I noticed that some of my subs with streaked stars seems to make it through FWHM, Eccentricity, Stars, and Median. I found that PSF Flux caught them. Have you ever used that criterion/parameter? Do you think it's useful? Also, I routinely make flats for each of my sessions since I shoot from my backyard and never know what might have gotten on the telescope lenses on a given night. This leads to a question or two. Do you find when you do SFS there is any benefit to importing ALL the subframes for each filter from multiple sessions, or do you do this per session and per filter? If there's a benefit to doing all the subframes for a particular project (by filter), is there a way to keep them sorted so that the correct flats are applied -- without having to go through the output (accepted) folder and manually sorting them? Thanks so much again for all the great information.

[@SKYST0RY]

A lot of questions there. I haven't really messed with PSF Flux.,TBH, I have found PixInsight promises a lot of performance that doesn't come through in the end. I visually preview images in Irfanview for obvious problems and cull those, then run SFS on the remaining images. You don't need to separate your flats and subs into separate folders as long as you instructed your sequencer to name them. If using the Weighted Batch Preprocessor, the subs and calibration frames will be sorted to where they need to be. But you do need to separate the frames into their sessions, i.e., folders marked Night_1, Night_2 and so on.

[@gregerianne3880]

@@SKYST0RY Thanks for your reply! Yes, I separate my subframes into sessions (and sometimes sessions AND scopes when I use two scopes for the same target) which is why I was asking the question about keeping them sorted. My flats and lights could very well be lumped together, but it's easier working with the lights if I keep the flats in a separate folder (by session or by session/scope) and let WBPP sort it out when I use my keywords. I've used IrfanView before, but not for astrophotography files was going to give it a try -- and ran into a major problem. My ASIAir puts a .FIT suffix on my subs, not .FITS and, unfortunately, IrfanView doesn't seem to recognize .FIT (only .FITS). I checked the plugins but I don't see a .FITS-containing plugin similar to FORMATS.DLL. It's just too cumbersome to rename all my subs to .FITS, I think.

[@gregerianne3880]

Ah, never mind about the .fit vs. .fits suffix!! I just found the way to add an extension in IrfanView. Thanks again!

[@SKYST0RY]

@@gregerianne3880 Glad you got it resolved. I use NINA for imaging. NINA allows you to choose file formats. It's always worked with Irfanview. I think NINA, being designed for broad, multi-platform use, just aims to do things in broadly compatible ways.

[@Calzune]

Can you show us how you use irfanview? I think most of us have never used that program.

[@SKYST0RY]

I should have referenced the video I made on it. You can find it here: kzbin.info/www/bejne/jWjOf3yGnt2cqa8si=XX2hWjsywsuswRCD

[@Calzune]

@@SKYST0RY thanks =)

[@JerryNewman-w8p]

Great job as always. I have also noticed that many of my subs are above the 2 sigma of Median and are almost always at the being of the shoot. This happens on nearly every night. I don’t think it is from clouds as I don’t see them in the images. I think it’s more related to the darkness of the sky as I usually shoot a half hour or so after sunset. I think the sky is too light when I start shooting. I’m also wondering if Median should be used at all. If you have cloudy images, the star count should be down. I’d like to hear if others are seeing the same thing and have an opinion about what might be the cause.

[@SKYST0RY]

A lighter sky will affect your median and possibly eccentricity as there is less differentiation between light and dark among pixels. Also, if your target is lower on the horizon at that time, the seeing and turbulence will be affected by more atmosphere. As it gets darker, and the target moves toward the center of the sky, your guiding should improve.

[@JerryNewman-w8p]

@@SKYST0RY That’s what I thought was going on as the Median values tend to decrease as time passes, because the sky is getting darker. Thanks again, you’re a class act.

[@NevadaDesertSkies]

Two comments: 1. Are these numbers in the expression formula good for everyone or are they dependent on the individual's equipment/seeing conditions? 2. I tried saving the process icons and also saving the blank project with the expressions window filled out but every time I re-open PI, the expressions window is blank. Is there a trick to keeping it saved as you mention around 04:28 ?

[@frankenstein77777]

Once you save the expression in SFS, drag the icon onto the Pixinsight desktop, rename it and save with the rest of your usual process icons. This saves the settings for reuse next time .

[@SKYST0RY]

The sigma formula should be good for any developing equipment. To get it to save the formula, all I did was save the project after I entered the formula. I am at a bit of a loss as to why it won't save for you. You can also screen capture the formula and save the image onto your desktop so you can write it in every time, but that's a bit time consuming.

[@NevadaDesertSkies]

​@@SKYST0RYthanks. I have a notepad doc that I keep on my desktop for repeat stuff like this. I'll just add this to the list so it's a simple copy/paste action.

[@jonathanpearceff]

This is interesting, but I don’t know (or understand) why the second standard deviation is importance of this. Why 2nd and not 1st?

[@SKYST0RY]

You would lose a lot of potentially useful subs, but you would get better data were you to keep the acceptable limit to the 1st std deviation. It's just a matter of choice. Weighing pros vs cons.

[@robertarmstrong9406]

I'm sorry. But I tried playing back a number of time, but still don't understand what your using instead of Blink.

[@darkoskrlec3101]

Probably IrfanView

[@SKYST0RY]

Sorry, I don't think I mentioned the name of my viewer alternative. I use Irfanview which reads FITS files fine. I have a video on using it. kzbin.info/www/bejne/jWjOf3yGnt2cqa8si=XX2hWjsywsuswRCD

[@markmruczek4391]

Great video and very helpful. I have been using a weighing formula that I got from Adam Block that works very well for me and wanted to share with you in case you hadn't seen the video. kzbin.info/www/bejne/nWe2hnV4bdqoaqcsi=BYKgTCJziLK7XyF0 I find that this weighing usually catches all of the subs that the formulas you have here except perhaps the median. I usually go back and add the median formula of just manually select them. This formula keeps the subs but gives them the weight that they should have, as Adam says it punishes the subs. Would love to hear your thoughts if you get a chance and thanks again for all the great information!

[@Mr77pro]

Don't confuse a weighting formula with culling bad subs. I definitely don't want a sub with egg-shaped stars to be in my stack no matter how low the score that a weighting formula gave it

[@SKYST0RY]

I haven't used Adam's weighing formula. He notes in the description it has become dated since SFS now includes new weighing and normalization that have improved things. Regardless, I have found that leaving all the culling to formulae misses things regardless. In my experience, SFS remains poor at detecting falling stars, aircraft and satellites scarring a sub. Since I shoot pretty short subs and after a winter night can easily end up with nearly 1000 subs, I wish there was a 100% reliable automated way to catch those, but I haven't found it yet. SFS also doesn't catch the occasional double exposure which results from a glitch between NINA and PHD2 when resuming guiding. Some persons say that these don't matter because they are "averaged out" in stacking, but that is a misunderstanding. Averaging doesn't remove glitches; it averages them into the data, essentially embedding them into the master. The problem subs may show up faintly in the master, but they nonetheless show up.