I'm not a statistics expert, but I would argue that using a t-test is not appropriate in this setting where you are alternating melatonin and non-melatonin days. There are two potential reasons for this:
1. One night's sleep is likely to influence the next night, and this cannot really be controlled for in the current design.
2. I think there is 'information' or statistical power available in the scenario where you sleep poorly on a melatonin off day after sleeping well on a melatonin on day, because this supports the hypothesis that melatonin helps sleep rather than the null. The key to making n=1 experiments more relevant is harnessing this fact statistically. Not sure how one would do this however.
> I'm not a statistics expert, but I would argue that using a t-test is not appropriate in this setting where you are alternating melatonin and non-melatonin days.
I'm not an expert either; I just do the best I can, and using a t-test is better than not using any test, y'know?
> 1. One night's sleep is likely to influence the next night, and this cannot really be controlled for in the current design.
Seth Roberts actually said something like that about my vitamin D setup, arguing there could be spillover effects to the next day, which is why I spent any time looking at the 'lagged nights' to see if there was any odd behavior in the following nights. His suggestion was to randomize blocks of nights or even weeks, rather than individual days. Of course, it was a bit late to start over and I didn't really feel like running another experiment...
1. One night's sleep is likely to influence the next night, and this cannot really be controlled for in the current design.
2. I think there is 'information' or statistical power available in the scenario where you sleep poorly on a melatonin off day after sleeping well on a melatonin on day, because this supports the hypothesis that melatonin helps sleep rather than the null. The key to making n=1 experiments more relevant is harnessing this fact statistically. Not sure how one would do this however.