Back to PLS HelpBetween vs Within groups PLS
Posted on 10/24/07 08:03:34
Number of posts: 6
Replies:
Between vs Within subjects PLS comparison
Posted on 10/24/07 08:28:21
Number of posts: 6
Dear PLS experts,
sorry about the blank post above - I had some trouble with the interface!
I have a question regarding inputting conditions as between or within groups: when I run PLS (non-rotated, on ERP data) with for example 4 within-subjects conditions, but also want to analyse how these conditions interact with trial block (e.g. first vs second half of trials), is it appropriate to create 2 datamats with 4 conditions for each half separately and then input these as 'groups' in the PLS analysis, or should I enter "half" as a within ss factor (giving me 8 repeated measures conditions).
I realise that when I enter "half" as between groups, the permutation test somehow discounts the main effect of half for the significance test so that half is nowhere near significant, although it still assigns it a large share of the cross-block covariance. But if I enter half as a repeated measure, the amount of cross-block covariance remains the same for each variable but the significance values are assigned differently, with half now being highly significant and the other effects less significant.
I was hoping that someone could please explain to me how and on what basis PLS excludes the between subjects factor in the permutation test since because of this, paradoxically, a between subjects design is actually more powerful for detecting the other effects as significant than a within subjects design. I was also wondering whether you think it is justifiable to input block of presentations as a "group" to exclude it in this way, since I'm not actually interested in the main effect of half - only the interaction between half and the other conditions, and excluding half gives me more power for the interactions. Although of course I don't want to inflate the P values if this is incorrect!
Apologies if this is question is answered somewhere else but I haven't been also to find it in the manual or the papers.
Thank you very much,
Zara
sorry about the blank post above - I had some trouble with the interface!
I have a question regarding inputting conditions as between or within groups: when I run PLS (non-rotated, on ERP data) with for example 4 within-subjects conditions, but also want to analyse how these conditions interact with trial block (e.g. first vs second half of trials), is it appropriate to create 2 datamats with 4 conditions for each half separately and then input these as 'groups' in the PLS analysis, or should I enter "half" as a within ss factor (giving me 8 repeated measures conditions).
I realise that when I enter "half" as between groups, the permutation test somehow discounts the main effect of half for the significance test so that half is nowhere near significant, although it still assigns it a large share of the cross-block covariance. But if I enter half as a repeated measure, the amount of cross-block covariance remains the same for each variable but the significance values are assigned differently, with half now being highly significant and the other effects less significant.
I was hoping that someone could please explain to me how and on what basis PLS excludes the between subjects factor in the permutation test since because of this, paradoxically, a between subjects design is actually more powerful for detecting the other effects as significant than a within subjects design. I was also wondering whether you think it is justifiable to input block of presentations as a "group" to exclude it in this way, since I'm not actually interested in the main effect of half - only the interaction between half and the other conditions, and excluding half gives me more power for the interactions. Although of course I don't want to inflate the P values if this is incorrect!
Apologies if this is question is answered somewhere else but I haven't been also to find it in the manual or the papers.
Thank you very much,
Zara
Untitled Post
Posted on 10/24/07 10:22:16
Number of posts: 394
Hi Zara
If I understand your design, its completely within subjects. You have 4 conditions by two trial blocks. This means that you should treat it as 8 conditions for PLS. The problem with "tricking" the program to consider it as a mixed design is that the permutation test will be invalid because you may have a subject appear more than once in the same condition by chance in the random reassignment, which violates the assumption of exchangibility in permutation testing. If its treated as completely within subject, then the permutation test maintains the repeated-measures aspect of the design in the permutation.
To use the non-rotated PLS to assess the interaction term, you can create a set of contrast for the interaction terms. I would leave the main effect of block as one of the contrasts so you have coded the full design (e.g, block main effect, condition main effect, blockXcondition interactions).
Randy
If I understand your design, its completely within subjects. You have 4 conditions by two trial blocks. This means that you should treat it as 8 conditions for PLS. The problem with "tricking" the program to consider it as a mixed design is that the permutation test will be invalid because you may have a subject appear more than once in the same condition by chance in the random reassignment, which violates the assumption of exchangibility in permutation testing. If its treated as completely within subject, then the permutation test maintains the repeated-measures aspect of the design in the permutation.
To use the non-rotated PLS to assess the interaction term, you can create a set of contrast for the interaction terms. I would leave the main effect of block as one of the contrasts so you have coded the full design (e.g, block main effect, condition main effect, blockXcondition interactions).
Randy
Untitled Post
Posted on 10/24/07 11:09:54
Number of posts: 6
quote:
Hi Zara
If I understand your design, its completely within subjects. You have 4 conditions by two trial blocks. This means that you should treat it as 8 conditions for PLS. The problem with "tricking" the program to consider it as a mixed design is that the permutation test will be invalid because you may have a subject appear more than once in the same condition by chance in the random reassignment, which violates the assumption of exchangibility in permutation testing. If its treated as completely within subject, then the permutation test maintains the repeated-measures aspect of the design in the permutation.
To use the non-rotated PLS to assess the interaction term, you can create a set of contrast for the interaction terms. I would leave the main effect of block as one of the contrasts so you have coded the full design (e.g, block main effect, condition main effect, blockXcondition interactions).
Randy
Hi Randy, Hi Zara
If I understand your design, its completely within subjects. You have 4 conditions by two trial blocks. This means that you should treat it as 8 conditions for PLS. The problem with "tricking" the program to consider it as a mixed design is that the permutation test will be invalid because you may have a subject appear more than once in the same condition by chance in the random reassignment, which violates the assumption of exchangibility in permutation testing. If its treated as completely within subject, then the permutation test maintains the repeated-measures aspect of the design in the permutation.
To use the non-rotated PLS to assess the interaction term, you can create a set of contrast for the interaction terms. I would leave the main effect of block as one of the contrasts so you have coded the full design (e.g, block main effect, condition main effect, blockXcondition interactions).
Randy
thanks very much for your quick reply - I understand now and I'll make sure to use the within subjects design for this study!
I just have one further question: when group is a factor (I'm also running some between groups experiments) why are the p-values for group always (at least in my experience) so non-significant even though the cross-block covariance accounted for by group is really large (e.g. over 50% of the variance)? Is the PLS somehow excluding group from the permutation analysis, or is it just underpowered to detect effects of a between groups factor? Running univariate ANOVAs on my ERP data, I do normally detect between groups main effects. Although these group differences are not interesting, I still would like to know why they don't show up in the PLS result?
Thank you!
Zara
Untitled Post
Posted on 10/24/07 12:28:23
Number of posts: 394
I am not sure about the group main effects. It does seem strange to get different results, though I admit we haven't explored that part too closely since we seldom focus on main effects alone. Would it be possible to get a bit more information on the data set you analyzed? Did you try non-rotated or mean-centred?
Randy
Randy
Untitled Post
Posted on 10/24/07 13:15:29
Number of posts: 6
Hi,
sure: I'm running non-rotated PLS on ERP data with a 2x2x2 factorial design. If I input the conditions with 2 groups (data for groups organised in 2 datamats with 4 repeated measures conditions in each) with the following design scores:
I get the following results:
constrast:
1: cross-block = 35.95% P = 0.959
2: cross-block = 21.23% P = 0.001
3: cross-block = 12.44% P = 0.024
4: cross-block = 6.28% P = 0.380
etc.
However, if I reorganise the data into one datamat with 8 repeated measures conditions, and use the corresponding within subjects contrasts (same as above but contrast 1 is main effect of another repeated measures factor), I get the following results:
1: cross-block = 35.95% P = 0.000
2: cross-block = 21.23% P = 0.010
3: cross-block = 12.44% P = 0.092
4: cross-block = 6.28% P = 0.585
etc.
I.e. the cross-block accounted for by each contrast is identical, but the p value is changed. I guess the discrepancy between the repeated measures contrasts (Contrast 2 onwards) is likely caused by the problem you explained in the previous message, that the same people might be treated as independent in the permutation test when I put the first factor as between groups, and therefore their p-values are inflated in the first analysis. however, I can't understand why the between groups factor in analysis 1 is so far from significant in the permutation test, even though it accounts for such a large proportion of variance (which comes out as P = 0.000 when it is entered as a within groups factor)?
I hope this makes sense. Thanks for all your help! Zara
sure: I'm running non-rotated PLS on ERP data with a 2x2x2 factorial design. If I input the conditions with 2 groups (data for groups organised in 2 datamats with 4 repeated measures conditions in each) with the following design scores:
| Contrast 1 (main effect group) | 1 | 1 | 1 | 1 | -1 | -1 | -1 | -1 |
| Contrast 2 (Main effect RM factor 1) | 1 | 1 | -1 | -1 | 1 | 1 | -1 | -1 |
| Contrast 3 (Main effect RM factor 2) | 1 | -1 | 1 | -1 | 1 | -1 | 1 | -1 |
| Contrast 4 (2-way Interaction contrast 1 x 2) | 1 | 1 | -1 | -1 | -1 | -1 | 1 | 1 |
| Contrast 5 (2-way Interaction contrast 1 x 3) | 1 | -1 | 1 | -1 | -1 | 1 | -1 | 1 |
| Contrast 6 (2-way Interaction contrast 2 x 3) | 1 | -1 | -1 | 1 | 1 | -1 | -1 | 1 |
| Contrast 7 (3-way Interaction contrast 1 x 2 x 3) | 1 | -1 | -1 | 1 | -1 | 1 | 1 | -1 |
I get the following results:
constrast:
1: cross-block = 35.95% P = 0.959
2: cross-block = 21.23% P = 0.001
3: cross-block = 12.44% P = 0.024
4: cross-block = 6.28% P = 0.380
etc.
However, if I reorganise the data into one datamat with 8 repeated measures conditions, and use the corresponding within subjects contrasts (same as above but contrast 1 is main effect of another repeated measures factor), I get the following results:
1: cross-block = 35.95% P = 0.000
2: cross-block = 21.23% P = 0.010
3: cross-block = 12.44% P = 0.092
4: cross-block = 6.28% P = 0.585
etc.
I.e. the cross-block accounted for by each contrast is identical, but the p value is changed. I guess the discrepancy between the repeated measures contrasts (Contrast 2 onwards) is likely caused by the problem you explained in the previous message, that the same people might be treated as independent in the permutation test when I put the first factor as between groups, and therefore their p-values are inflated in the first analysis. however, I can't understand why the between groups factor in analysis 1 is so far from significant in the permutation test, even though it accounts for such a large proportion of variance (which comes out as P = 0.000 when it is entered as a within groups factor)?
I hope this makes sense. Thanks for all your help! Zara
Login to reply to this topic.
