Back to PLS HelpI'm Online
Posted on 06/08/18 10:15:36
Number of posts: 11
Dear PLS experts,
I'm using PLS to investigate group differences in resting state EEG connectivity. I have 3 groups, in a single condition. I'm interested in the group differences only for this analysis. For this, I'm using the command line. I ran 1000 permutations and bootstrapping resamples using the mean-centering PLS with mean centering across groups (method =1, meancentering_type =1). The first latent variable (LV) explains 50% of the variance and its singular value is significantly different from the permutted data (p=0.04). I then proceeded to inspect the bootstrapping results of this LV. This is what I obtained:
[result.boot_result.orig_usc(:,1) result.boot_result.llusc(:,1) result.boot_result.ulusc(:,1)]
orig_usc llusc ulusc
Group 1 -2.5289 -3.9131 -2.9501
Group 2 1.6371 1.7881 2.6432
Group 3 0.8918 0.7921 1.6773
From the numbers above, the confidence intervals do not cross zero (which is good) but the lower and upper level of the confidence interval do not cover the value of orig_usc. The skewness of the distribution shows that they're between -1 and 1, which is not very skewed (skewness(squeeze(result.boot_result.distrib(group, 1, 2:end))) ). I ran this with a PLS version from 2015 and one from 2017 with similar results.
Does this mean that the significant LV is not reliable? Or is there a problem with the bootstrapping resamples or the mean centering?
Thank you in advance for your help.
Best regards,
Laura
Replies:
Posted on 06/09/18 08:18:53
Number of posts: 394
Dear PLS experts,
I'm using PLS to investigate group differences in resting state EEG connectivity. I have 3 groups, in a single condition. I'm interested in the group differences only for this analysis. For this, I'm using the command line. I ran 1000 permutations and bootstrapping resamples using the mean-centering PLS with mean centering across groups (method =1, meancentering_type =1). The first latent variable (LV) explains 50% of the variance and its singular value is significantly different from the permutted data (p=0.04). I then proceeded to inspect the bootstrapping results of this LV. This is what I obtained:
[result.boot_result.orig_usc(:,1) result.boot_result.llusc(:,1) result.boot_result.ulusc(:,1)]
orig_usc llusc ulusc
Group 1 -2.5289 -3.9131 -2.9501
Group 2 1.6371 1.7881 2.6432
Group 3 0.8918 0.7921 1.6773
From the numbers above, the confidence intervals do not cross zero (which is good) but the lower and upper level of the confidence interval do not cover the value of orig_usc. The skewness of the distribution shows that they're between -1 and 1, which is not very skewed (skewness(squeeze(result.boot_result.distrib(group, 1, 2:end))) ). I ran this with a PLS version from 2015 and one from 2017 with similar results.
Does this mean that the significant LV is not reliable? Or is there a problem with the bootstrapping resamples or the mean centering?
Thank you in advance for your help.
Best regards,
Laura
HI Laura - this is an issue we've seen before, though less frequently with task PLS. It doesn't mean the results are unstable, however.
A couple of things to check:
1) in the result.boot_result there are variable ulusc_adj and llusc_adj, which are meant to re-centre the bootstrap distribution around the mean assuming the distrivution is normal. Check if that helps
2) generate a histogram of the bootstrap distrivbutions to see where the mean lies relative to the full distribution. If you do that, please post or send me a screenshot as I'd be interested to see this (rmcintosh@research.baycrest.org)
I'm Online
Posted on 06/11/18 06:32:08
Number of posts: 11
Dear Prof. McIntosh,
Thank you for your quick response. The adjusted confidence intervals are:
orig_usc llusc ulusc llusc_adj ulusc_adj
Group 1 -2.5289 -3.9131 -2.9501 NaN NaN
Group 2 1.6371 1.7881 2.6432 1.4325 1.5249
Group 3 0.8918 0.7921 1.6773 0.3104 1.0711
In the function rri_distrib, lines 16 to 21, it is checked how many bootsrapping samples result in a usc value less or equal to the one obtained from the original data. Then, an if statement checks if the proportion is 0 (i.e. all bootstrapping samples > original usc) or 1 (i.e. all bootstrapping samples < original usc). On this data for Group 1, the proportion is 1, so the adjusted confidence value is set to NaN (no calculation of the inverse cumulative gaussian is done).
Without the original data, the ranges of the bootstrapped samples per group are:
Group 1: [-4.711, -2.58]
Group 2: [1.4325, 3.4098]
Group 3: [0.3104, 2.3497]
I'll send the histogram by email. Thank you again for your help.
Best regards,
Laura
Posted on 06/11/18 07:35:07
Number of posts: 394
At the moment, I am not 100% sure what the source of the problem is, but my guess is that it may be the mean-centring. Can you try the analysis as a "non-rotated" using the contrast specified by your current LV1? The bootstrap routine for non-rotated does not do mean centering so it will be an easy way to check whether thats the problem.
Randy
I'm Online
Posted on 06/12/18 04:26:15
Number of posts: 11
Dear Prof. McIntosh,
I followed your suggestion and ran the non-rotated task PLS using the design obtained from the mean PLS ([-2.53; 1.64; 0.89]). The new vector result.v is: [-0.8048; 0.5217; 0.2831]. In this case, the bootstrapped confidence intervals do cover the estimate:
orig_usc llusc ulusc llusc_adj ulusc_adj
Group 1 -13.8587 -17.5622 -11.8718 -15.0553 -3.8640
Group 2 -9.6918 -14.2328 -6.2211 -11.7696 -1.1256
Group 3 -10.4398 -14.9072 -7.3931 -12.3198 -1.6060
Since the mean centering option was 0 for mean PLS and non-rotated task PLS, I don't understand your comment on the effect of mean-centering on mean PLS but not on the non-rotated PLS.
Best regards,
Laura
Posted on 06/12/18 08:57:01
Number of posts: 394
Dear Prof. McIntosh,
I followed your suggestion and ran the non-rotated task PLS using the design obtained from the mean PLS ([-2.53; 1.64; 0.89]). The new vector result.v is: [-0.8048; 0.5217; 0.2831]. In this case, the bootstrapped confidence intervals do cover the estimate:
orig_usc llusc ulusc llusc_adj ulusc_adj
Group 1 -13.8587 -17.5622 -11.8718 -15.0553 -3.8640
Group 2 -9.6918 -14.2328 -6.2211 -11.7696 -1.1256
Group 3 -10.4398 -14.9072 -7.3931 -12.3198 -1.6060
Since the mean centering option was 0 for mean PLS and non-rotated task PLS, I don't understand your comment on the effect of mean-centering on mean PLS but not on the non-rotated PLS.
Best regards,
Laura
Well this is a helpful outcome! If you look back in the forum, we modified the non-rotated PLS to not use mean centriing as we were getting weird results and since contrasts are used it unnecessary.
This does give me a place to start exploring how we may be able to remedy this issue in future releases. Would you be williing to send me your matlab results file so that I can take a look at the bootstrap distribution in more detail?
Thanks
Randy
Login to reply to this topic.
