Back to PLS Helpfeature detection & overfitting in PLS
Posted on 02/23/07 11:13:10
Number of posts: 41
hi all ~
i have some lovely results from a behavioral PLS (but a messy result using task PLS - noisy data) - and i have encountered two questions posed by my audience (whom are not too familiar with dimension reduction techniques) - i am currently doing some reading to figure out the best answers but perhaps you can point me in the right direction...
1. would it be possible, useful and/or feasible to use some feature-selection step before doing PLS? i.e., are we missing results in the task-pls b/c of noise etc..
* i think the answer to this is no it's not useful to do so - b/c averaging across task condition already constrains the solutions - can you comment from a technical perspective?
2. is over-fitting an issue when using PLS - are our results 'real' basically? this is a concern from having so many voxels for so few conditions...
* this one i don't know - is PLS prone to overfitting - or, again, is the fact that we're restricting to patterns across task conditions a sufficient criterion to take care of the concern!
many thanks & best regards!
agatha
i have some lovely results from a behavioral PLS (but a messy result using task PLS - noisy data) - and i have encountered two questions posed by my audience (whom are not too familiar with dimension reduction techniques) - i am currently doing some reading to figure out the best answers but perhaps you can point me in the right direction...
1. would it be possible, useful and/or feasible to use some feature-selection step before doing PLS? i.e., are we missing results in the task-pls b/c of noise etc..
* i think the answer to this is no it's not useful to do so - b/c averaging across task condition already constrains the solutions - can you comment from a technical perspective?
2. is over-fitting an issue when using PLS - are our results 'real' basically? this is a concern from having so many voxels for so few conditions...
* this one i don't know - is PLS prone to overfitting - or, again, is the fact that we're restricting to patterns across task conditions a sufficient criterion to take care of the concern!
many thanks & best regards!
agatha
Replies:
Untitled Post
Posted on 02/23/07 19:39:06
Number of posts: 394
Hi Agatha;
1. would it be possible, useful and/or feasible to use some feature-selection step before doing PLS? i.e., are we missing results in the task-pls b/c of noise etc..
* i think the answer to this is no it's not useful to do so - b/c averaging across task condition already constrains the solutions - can you comment from a technical perspective?
It is feasible but not part of the current PLS code. We assume that you can handle the feature extraction part and we'll take care of the PLS part. For example, ICA denoising, or even analysing the independent components could be alternatives.
2. is over-fitting an issue when using PLS - are our results 'real' basically? this is a concern from having so many voxels for so few conditions...
* this one i don't know - is PLS prone to overfitting - or, again, is the fact that we're restricting to patterns across task conditions a sufficient criterion to take care of the concern!
If we did not use permutation and bootstrap tests then the answer would be yes. We use the tests to guard against overfitting.
1. would it be possible, useful and/or feasible to use some feature-selection step before doing PLS? i.e., are we missing results in the task-pls b/c of noise etc..
* i think the answer to this is no it's not useful to do so - b/c averaging across task condition already constrains the solutions - can you comment from a technical perspective?
It is feasible but not part of the current PLS code. We assume that you can handle the feature extraction part and we'll take care of the PLS part. For example, ICA denoising, or even analysing the independent components could be alternatives.
2. is over-fitting an issue when using PLS - are our results 'real' basically? this is a concern from having so many voxels for so few conditions...
* this one i don't know - is PLS prone to overfitting - or, again, is the fact that we're restricting to patterns across task conditions a sufficient criterion to take care of the concern!
If we did not use permutation and bootstrap tests then the answer would be yes. We use the tests to guard against overfitting.
thank you !
Posted on 02/26/07 10:04:09
Number of posts: 41
that's very useful!
Login to reply to this topic.
