SPSS: How to identify an optimum Cut-Off Point on a Psychometric Scale - Part 1

This short video details how to find an optimum cut-off point on a Psychometric Scale using IBM SPSS. In particular, the video details how to generate a Receiver Operator Characteristics Curve and using a measure of Sensitivity and Specificity find an optimum scale cut-off point. This cut-off point maximising the True Positive and True Negative detection rates.

[@maysjamal9136]

May I say thank you sooooooomuch u just save another master student during my ROC confusion, have I mention I did study ROC, specificity and sensitivity and calculate them (took quiz online and was good) yet I didn't know how to actually compute my data ,since, all videos out there regarding +ve and -ve values not on scale data. soooo massive thank u :) pls make more vid for those like me (state of confucianism or confusionysis) its a real disease with massive headache. lasty, thanks :)

[@MathsAndStats]

That's great to hear Mays and thank you for the nice comment. Jonathan.

[@darnishanair]

count in another master student! I've been searching everywhere for this. Thank you so much!!

[@yasinsaglam4158]

Thanks for the video. How can I find cut off point if my test variable is Ordinal likert scale. It is 0-1-2-3-4

[@AnushthaMishra]

What if my state variable is not dichotomous? I am studying the prevalence of stress and the subjective report answers were in mild, moderate and severe. How do I identify an optimum cut off point in this case?

[@MathsAndStats]

Interesting! I am not aware of any technique that supports a non-dichotomous state variable. The ROC curve is specifically designed for a dichotomous state variable. One possible solution, although; I do not have any reference for it's validity would be to recode your state variable into a dichotomous state variable. Say, 0 'not-severe' and 1 for 'severe'. Or some other alternative. If you do find a procedure, I would be interested in getting the reference to it from you. Regards Jonathan

[@roverspinne]

@@MathsAndStats Thanks for your video. Your scale seems to have a dichotomous item that asks if they are satisfied or not overall, but most scales do not have this. I have two continuous scales and want to use one (predictor) to predict the other (outcome), so the outcome needs to have a set cutoff. However, it doesn't have this "are you satisfied" type dichotomous item to set the cutoff by. So, probably like other people in the comments, the measure I want to set a cutoff for is the variable I want to be my 'state' variable (i.e., dichotomize into presence (1) or absence (0) of the construct measured by the scale itself.

[@vanessachen2313]

sir could i ask, what if your test variable is categorical? for example, if under job satisfaction - 1 = satisfied, 2 = mildly satisfied, 3 = very satisfied etc, how do you do the ROC analysis?

[@lizaakter3732]

I am wondering to know whether the score of the column of job satisfaction is the mean score?

[@LadyProserphina]

What should I do when I don't have a state variable?

[@MathsAndStats]

Hi. Unfortunately the Receiver Operator Characteristics curve relies upon the state variable to estimate true/false positives and true/false negatives and then to do the optimisation. A lot of student studies involve assessing participants on specific scales that are constructed from multiple items, but most students fail to ask the actual state question. For example, if you are measuring extraversion using an accepted inventory, why not ask the actual question 'Do you classify yourself as extrovert?' This would then act as the state variable. If you don't have this, maybe consider some other question that you have asked that could be considered to be highly correlated with the scale you measured. This would more than likely be frowned upon, but in the abscence of anything else it could work. Once you fully document the approach and the limitations of it. Alternatively, you could consider different cut-off points at different quartile locations and see how they partition your data. Alternatively, and this would probably be the best bet in the absence of a state variable; is to consider the literature associated with your scale and to identify from the literature cut-off points that had been identified in those studies. at least then you are building on top of peer reviewed results. I hope this opinion helps. Kindest regards. Jonathan

[@alecandreva]

is it possible to compute Delong test between two ROC curves in SPSS?

[@temamibrahim593]

How to determine the optimum catoffvalue for ELISA? PLEASE HELP ME

[@MathsAndStats]

Hi Temam. I assume you are talking about the ELISA assay? Maybe a small paragraph about your experiment. Regards. Jonathan

[@choyhh3639]

Good day to u. Dear friend, what if i don't have a criterion variable for ROC curve? Well, the dependent variable itself (PTTD) is the one which I want to determine its cut-off point. My research is about patient's trust towards physician (PTTD), and i have a self-reported Likert scale questionnaire which consists of 13 questions, 5 points (1-Strongly disagree/never, right up to 5-Strongly agree/All the time). The issue was there ain't any golden standard or proposed cut-off point from previous researches, they just leave PTTD as a continuous variable. (1) What should i do if i want to consider using ROC curve to determine its cut-off point? For now I used quartile/percentile as my cut-off point. To be precise, the 3rd quartile (75%). (2) What are your thoughts about this? Thank u !

[@choyhh3639]

Well, i didn't ask them a straight-forward question, similarly as to your proposed idea for a criterion variable above , i.e. "Do u trust your current tending physician" or "Do u generally trust physician, in the past/current".

[@MathsAndStats]

Hi Choy, it seems like you are in a tricky situation. You are trying to develop a scale - composed of your 13 items (PTTD). Without the 'state variable,' I do not know any way to calculate the actual true positive rate (TPR) and the false positive rate (FPR) that are needed for ROC. So I don't think I can be of any real help. I was going to suggest the literature and undertaking a meta-analysis of cut-off points that have been previously reported. Jones et. al. consider this in the following: www.annalsthoracicsurgery.org/article/S0003-4975(04)01986-1/pdf They define a new technique called SROC, but this relies upon previous studies. Your quartile approach seems reasonable - but could be open to the question of why 75th percentile and not the 80th etc. If you exclude the question 'Do u trust your current tending physician,' which seems very close to the required state question, does your scale remain reliable/consistent? If it does you could use this as your state variable. That's all I can think of at this stage - but I will ponder it a bit more. Sorry for the delay in replying. Regards. Jonathan.

[@choyhh3639]

Thank you Jonathan for the recommendation & response. From our research culture, we try to silence direct question or 'make-or-break' type of question as to "Do u trust X or not" (Y/N), whether in quanti or even qualitative research. That is why we built a scale to indirectly, yet reliably measure such outcome. Imagine if the patient know u r measuring his/her trust towards the current tending physician. Biases shall definitely occur hence dampening the respondent's honest responses which we are desperately seeking for. Such question usually won't be included into the scale/overall research questionnaire, we don''t use its result, but with an exception to your new input about ROC. Basically, all Yes/No questions are avoided. Haha, u caught me on the 3rd quartile approach to justify a cut-off point. I remember such idea, it was widely used but I couldn't recall the its evidences - am looking for them. Kindly share with me if u encounter any. There is suggestion to use the summation of all the mean scores for each item in the scale (13x) as the cut-off point. What do u think about this approach? Its so happen that , i dun know why, the respective total mean score for this PTTD is the same as the average of the total score for all respondents (total score for all X respondents divided y the x number of respondents).

[@mohamedhabashy1262]

You speak too Fast. Do you know how many times you said okay?