What's your thoughts on plate/batch-wise whitening for batch effect?

[hellowangqian]

I'm thinking about the batch effect handling part of the code. What it does in the code is quite simple: learning the parameters with all DMSO data and applying the whitening model to all data (DMSO and treatments). My question is why not do this plate-wisely or batch-wisely for such plate/batch-effect correction?

[Arkkienkeli]

We do this transformation (sphering) after the features are extracted and aggregated to a well-level.
See example here.

[hellowangqian]

Thanks for your prompt reply, but my question is: in step 4 of the example, the dataframe "wells" contain well-level data of all plates, why not do the sphering for each plate separately? The pseudo-code is as follows:

for plate in allPlates:
     do Sphereing on wells_from_this_plate
concatenate Sphered wells_from_each_plate 

[Arkkienkeli]

We specifically correct for plate variation.
If you try to correct each plate separately, you are going to correct other confounding effects (for instance, well-position effects), though in this case you will have very few data points, so probably you will need to go down to image-level or single-cell level.
I don't think we tried it.