Back to PLS HelpPosted on 04/03/16 06:28:25
Number of posts: 6
Dear all,
I have aquestion with regard to non-significant p-values in split-half routine.
I want to run PLS correlation on gray matter volumes of ~400 subjects on one side and a set of 82 highly correlated (together and to age as well as gender) bihavoural questionaires.
As input of PLS structural, I used residuals of GMV corrected for age and sex (using general linear model).
The behavioural measures are not-normally desitributed. Yet, I used general linear model and corrected each of the questionaire for age and sex and used the residuals as input for PLS. (NOTE: the residulas are still correated with age using non-parametric correlations).
I ran PLS on these datasets (one sessions data), using 100 permutations and 50 splits.
As the result, I get 82 components.
LV1, explains ~38% of variance, has a permutation p-value < 0.000 , the U-corr p-val (GMV) <0.0000 but the v-corr p-val (behavioural) =0.18.
I am not so sure if I am allowed to use the data and interprete the components, as one of the p-values are above 0.05.
- Am I right to think that in the case of very highly correlated measurs (such as behavioural dataset of mine), split half should not be used?
- Do you have any suggestions, how to proceed from here? Do I have a better option to deal with age and sex correlations, which are also probably increasing the correlations of split halves of the permuted data?
Thanks a lot,
Shahrzad
Replies:
Posted on 04/04/16 03:00:21
Number of posts: 394
We have been looking more closely into the spiit-half procedure and I would suggest focusing on both overall permutation and the split-half together. Split-half is for reliabilty really and not significance per se. LV1 is interpretabie for sure.
We are working to revise the PLS code and give better documentation on the split-half procedure. i hope to have this done by summer
Posted on 04/04/16 04:21:33
Number of posts: 6
Dear Randy , thanks a lot.
With regard to age and sex, do you recommend to add them as a variable to PLS or try to regress them out of behavioural and brain data, before PLS?
Bests
Shahrzad
Posted on 04/04/16 04:24:15
Number of posts: 394
Dear Randy , thanks a lot.
With regard to age and sex, do you recommend to add them as a variable to PLS or try to regress them out of behavioural and brain data, before PLS?
Bests
Shahrzad
I prefer to include them in the analysis to help understand if and how the variables are related to your measures - it helps with interpretation.
Posted on 04/04/16 04:31:50
Number of posts: 6
Thanks a lot. And my last question, lack of normal distribution of the data (my behavioural data in this case) is not an issue for PLS? I have transformed the data to make it more normally distributed, yet it is very skewed as many subjects have zero value for most of the questions.
Bests
Shahrzad
Posted on 04/04/16 05:29:49
Number of posts: 394
the non-normality won't be a problem, but if you have a lot of people wih the same value (i.e., 0), you could have restricted range issues, which is why the split-half assessment suggests the behavior side of the analysis is less reliable
Posted on 04/04/16 05:37:56
Number of posts: 6
So since this is the nature of the data, I hope it will not be an issue.
Thanks a lot for your fast and really helpful responses,
Shahrzad
Posted on 04/08/16 16:51:44
Number of posts: 6
Dear Randy, I have an additional question with regard to the permutation procedure, implemeneted within the toolbox.
As I read from your papers before 2013, it is always suggested to perform "procrustes rotation" in both permutation and bootstrap procedures (see http://dx.doi.org/10.1016/j.neuroimage.2010.07.034 for example).
However, in the 2013 paper, where spli half procedure is tested (http://www.scopus.com/inward/record.url?eid=2-s2.0-84886038301&partnerID=tZOtx3y1), actually also permutations are compared with rotation and without rotation and it seems from the tables that the permutation with rotaion gives significant results when looking at total noise.
So, My questions are:
1) what is your latest suggestion on this issue?
and 2) what is the implemented within the PLS matlab toolbox? Do the permutations also apply the rotaion or not?
Thanks a lot,
Shahrzad
Posted on 04/09/16 10:50:32
Number of posts: 394
Dear Randy, I have an additional question with regard to the permutation procedure, implemeneted within the toolbox.
As I read from your papers before 2013, it is always suggested to perform "procrustes rotation" in both permutation and bootstrap procedures (see http://dx.doi.org/10.1016/j.neuroimage.2010.07.034 for example).
However, in the 2013 paper, where spli half procedure is tested (http://www.scopus.com/inward/record.url?eid=2-s2.0-84886038301&partnerID=tZOtx3y1), actually also permutations are compared with rotation and without rotation and it seems from the tables that the permutation with rotaion gives significant results when looking at total noise.
So, My questions are:
1) what is your latest suggestion on this issue?
and 2) what is the implemented within the PLS matlab toolbox? Do the permutations also apply the rotaion or not?
Thanks a lot,
Shahrzad
Thanks for the question, Shahrzad. I think the link to the Springer source is incorrect and you meant to point to the chapter that introduced the split-half procedure.
At any rate to answer your question:
1) we do use the procrustes for the current overall permutation test. As we noted in the split-half paper, there is a conflict between false-positive and false-negative for the procedures with and without rotation. Without rotation has a reasonable false positive rate, but horrible false-negative - with rotation has higher false-positive but a far better false negative rate
2) In our McIntosh & Lobaugh 2004 review paper in Neuroimage, permutation test is only part of the solution, the bootstrap procedure (and now split-half) need to be considered as providing supporting information for the interpretation of the PLS results. A p-value is meaningless if the parameter estimates are not reliable.
Posted on 04/10/16 03:07:58
Number of posts: 6
Dear Randy, sorry for the incorrect link. Yes, I was reffering to that chapter. thanks a lot for these very helpful information.
So, we report P-value of the latent variable and also bootstrap ratios for reliability.
Thanks a lot
Shahrzad
Login to reply to this topic.
