Back to PLS Help

Mean correlation out of bound
Eric
Posted on 11/28/22 02:27:43
Number of posts: 14
Eric posts:

To whomever this might concern,

I have run a behavior PLS using 36 behavioral measures and got 2 significant latent variables (LV). For both LVs, some of the behavior correlations (orig_corr) lie outside of the upper and lower bounds as calculated by the bootstrapping (ulcorr and llcorr). Please see below the sample results for a few of such behavioral measure for one latent variable.

orig_corr ulcorr llcorr ulcorr_adj llcor_adj
-0.10906 -0.13331 -0.21737 -0.10215 -0.10215
-0.13448 -0.16314 -0.24496 -0.13435 -0.13435
-0.13606 -0.15863 -0.24106 -0.12028 -0.12028

 

I have searched the forum and seemed like this has happened before. Any idea of how to best tackle this problem would be much appreciated!

Replies:

Untitled Post
rmcintosh
Posted on 12/01/22 15:24:40
Number of posts: 392
rmcintosh replies:

quote:

To whomever this might concern,

I have run a behavior PLS using 36 behavioral measures and got 2 significant latent variables (LV). For both LVs, some of the behavior correlations (orig_corr) lie outside of the upper and lower bounds as calculated by the bootstrapping (ulcorr and llcorr). Please see below the sample results for a few of such behavioral measure for one latent variable.

orig_corr ulcorr llcorr ulcorr_adj llcor_adj
-0.10906 -0.13331 -0.21737 -0.10215 -0.10215
-0.13448 -0.16314 -0.24496 -0.13435 -0.13435
-0.13606 -0.15863 -0.24106 -0.12028 -0.12028

 

I have searched the forum and seemed like this has happened before. Any idea of how to best tackle this problem would be much appreciated!

Hi Eric,

 

This is a known issue for which we haven't found a good solution. It does happen with relatively small sample sizes but that does not mean the point estimates (correlations) are unreliable.

 

An easy fix is to use the bootstrap standard error to compute the confidence intervals rather than the percentile (whicih is what we do).  That can be calculate from the field boot_result.distrib, which has the entire distribution for the behavior side of the analysis.  

 

hope that helps

Randy



Untitled Post

I'm Online
nlobaugh
Posted on 12/01/22 15:59:18
Number of posts: 228
nlobaugh replies:

You might also want to check these outputs:  - the fact that the *adj limits are identical is odd

num_LowVariability_behav_boots:    display numbers of low
                 variability resampled hehavior data in
                 bootstrap test
         badbeh:        display bad behav data that is caused by
                 bad re-order (with 0 standard deviation)
                 which will further cause divided by 0
         countnewtotal:    count the new sample that is re-ordered
                 for badbeh



Untitled Post
Eric
Posted on 12/02/22 00:30:01
Number of posts: 14
Eric replies:

Dear all,

Thank you for your replies!

The data I used have large sample size (N=3000+) so I don't think that would be an issue here. The behavior data is however in Likert scale (0-5) and skewed toward 0, which might be a problem? However, I did the PLS analysis with similar behavior data with much smaller sample size (N=100+) without encountering such issues.

Regarding the fact that the *adj limits are identical, I did a quick check for the 1st LV. Below are all the troublesome corr which lies out of bound, quite a few of them do have identical adj limits, but some also have diffrent adj limits.:

orig_corr ulcorr llcorr ulcorr_adj llcor_adj
-0.10906 -0.13331 -0.21737 -0.1021512 -0.1021512
-0.13448 -0.16314 -0.24496 -0.1343458 -0.1343458
-0.13606 -0.15863 -0.24106 -0.1202817 -0.1202817
-0.11955 -0.14697 -0.22594 -0.1142138 -0.1142138
-0.07491 -0.08849 -0.16343 -0.0486365 -0.0651988
-0.09647 -0.11874 -0.19807 -0.0954431 -0.0954431
-0.11839 -0.1381 -0.21888 -0.1041022 -0.1059555
-0.09412 -0.11646 -0.20263 -0.0924281 -0.0924281
-0.08441 -0.10899 -0.20175 -0.0738357 -0.0738357
-0.07165 -0.08109 -0.16333 -0.0490865 -0.0666741
-0.07854 -0.09344 -0.16793 -0.0602779 -0.0685815
-0.0935 -0.1022 -0.18938 -0.0592278 -0.0903954
-0.09013 -0.11226 -0.20569 -0.0806942 -0.0807793
-0.10346 -0.12547 -0.20638 -0.0822778 -0.0822778
-0.08193 -0.09083 -0.16295 -0.0654829 -0.0811599
-0.08694 -0.09354 -0.17455 -0.0548671 -0.0838504
-0.08051 -0.08216 -0.1496 -0.0568116 -0.0872811
-0.08605 -0.10171 -0.17889 -0.0703527 -0.0703527
-0.07481 -0.08818 -0.17049 -0.0449204 -0.0725828

 

I have checked num_LowVariability_behav_boots, badbeh and countnewtotal. All these show either 0 or [], which is normal?

I'm looking forward to seeing if you have any other ideas why this issue might have happened and how to address them!

Much thanks!

 



Untitled Post
rmcintosh
Posted on 12/06/22 20:48:27
Number of posts: 392
rmcintosh replies:

quote:

Dear all,

Thank you for your replies!

The data I used have large sample size (N=3000+) so I don't think that would be an issue here. The behavior data is however in Likert scale (0-5) and skewed toward 0, which might be a problem? However, I did the PLS analysis with similar behavior data with much smaller sample size (N=100+) without encountering such issues.

Regarding the fact that the *adj limits are identical, I did a quick check for the 1st LV. Below are all the troublesome corr which lies out of bound, quite a few of them do have identical adj limits, but some also have diffrent adj limits.:

orig_corr ulcorr llcorr ulcorr_adj llcor_adj
-0.10906 -0.13331 -0.21737 -0.1021512 -0.1021512
-0.13448 -0.16314 -0.24496 -0.1343458 -0.1343458
-0.13606 -0.15863 -0.24106 -0.1202817 -0.1202817
-0.11955 -0.14697 -0.22594 -0.1142138 -0.1142138
-0.07491 -0.08849 -0.16343 -0.0486365 -0.0651988
-0.09647 -0.11874 -0.19807 -0.0954431 -0.0954431
-0.11839 -0.1381 -0.21888 -0.1041022 -0.1059555
-0.09412 -0.11646 -0.20263 -0.0924281 -0.0924281
-0.08441 -0.10899 -0.20175 -0.0738357 -0.0738357
-0.07165 -0.08109 -0.16333 -0.0490865 -0.0666741
-0.07854 -0.09344 -0.16793 -0.0602779 -0.0685815
-0.0935 -0.1022 -0.18938 -0.0592278 -0.0903954
-0.09013 -0.11226 -0.20569 -0.0806942 -0.0807793
-0.10346 -0.12547 -0.20638 -0.0822778 -0.0822778
-0.08193 -0.09083 -0.16295 -0.0654829 -0.0811599
-0.08694 -0.09354 -0.17455 -0.0548671 -0.0838504
-0.08051 -0.08216 -0.1496 -0.0568116 -0.0872811
-0.08605 -0.10171 -0.17889 -0.0703527 -0.0703527
-0.07481 -0.08818 -0.17049 -0.0449204 -0.0725828

 

I have checked num_LowVariability_behav_boots, badbeh and countnewtotal. All these show either 0 or [], which is normal?

I'm looking forward to seeing if you have any other ideas why this issue might have happened and how to address them!

Much thanks!

 

I did a bit more digging and it does seem that many have found this problem with percentile method of computing CI's from bootstrap estimation. I am going to try a few other options - like Efron's BCA verision. Would you be interested in helping out?  I can either send you test code or you can send the data with the weird CI estimations



Untitled Post
Eric
Posted on 12/07/22 21:05:00
Number of posts: 14
Eric replies:

Hi Randy,

Thank you so much for your reply! I'd love to help out. If you can send me the test code so that I can try it out it would be great!



Untitled Post
rmcintosh
Posted on 12/09/22 14:50:11
Number of posts: 392
rmcintosh replies:

quote:

Hi Randy,

Thank you so much for your reply! I'd love to help out. If you can send me the test code so that I can try it out it would be great!

Cool.  Would you mind sending me the results file from your first example? I want to look at the histograms to see if the ideas I have will work.  you can email me the link randy_mcintosh@sfu.ca

 

thanks

Randy



Untitled Post
Eric
Posted on 12/09/22 21:52:49
Number of posts: 14
Eric replies:

Dear Randy,

I have sent you the PLS mat file. Do let me know if you have any trouble accessing it.

Sophia



Untitled Post
Eric
Posted on 12/09/22 21:53:33
Number of posts: 14
Eric replies:

Dear Randy,

I have sent you the PLS mat file. Do let me know if you have any trouble accessing it.

Sophia



Untitled Post
rmcintosh
Posted on 12/11/22 14:28:31
Number of posts: 392
rmcintosh replies:

Thanks for the file. Getting the code together will take a while so in the interim, you can use the bootstrap SE to get the confidence intervals.

e.g., 

selv11=std(result1_test.boot_result.distrib(1,1,:))

ulCI=result1_test.boot_result.orig_corr(1,1)+(selv11*1.96)

llCI=result1_test.boot_result.orig_corr(1,1)-(selv11*1.96)

or depending on your plotting program you can enter the SE directly into the program 




Login to reply to this topic.

  • Keep in touch

Enter your email above to receive electronic messages from Baycrest, including invitations to programs and events, newsletters, updates and other communications.
You can unsubscribe at any time.
Please refer to our Privacy Policy or contact us for more details.

  • Follow us on social
  • Facebook
  • Instagram
  • Linkedin
  • Pinterest
  • Twitter
  • YouTube

Contact Us:

3560 Bathurst Street
Toronto, Ontario
Canada M6A 2E1
Phone: (416) 785-2500

Baycrest is an academic health sciences centre fully affiliated with the University of Toronto