I. On Interpreting Experimental Results
1. Most experiments fail -- that is, do not satisfy expectations. This is because at least one of the underlying assumptions was in error. Validating assumptions (like testing hypotheses) is rational science (progress).
2. Most experiment succeed -- that is, provide new information. However, the information is usually not what was hoped for. Results which fundamentally challenge assumptions are discoveries. Therefore, love all data and mistrust all assumptions.
3. The data are always right. Data that do not make sense are telling you that you do no understand your system. You, not they, are the problem.
4. You can use the dissonance between the "unwanted" data and your expectations to discover new truths. Learn to play with the dissonance in your mind until you resolve it. Never suppress dissonant data; discoveries lie therein.
5. You cannot do the same experiment twice and get different results. Irreproducibility comes from not understanding the variables (hence controlling them).
6. It is hard to do the same experiment twice; i.e., to control all and not change any of the variables.
II. On Planning Protocols
1. The better you (and the experiment) get, the more time you will spend planning experiments and the less time executing them.
2. Never start an experiment without a complete, written protocol. A minimal protocol is one which can be properly executed by someone else entirely from what is written. It should be complete, formal and explicit. The more that is specified, the better the results. It is also a primary historical record of great value.
3. Variables are the axes of an n-dimensional space in which the experiment is performed. How do you find optimal conditions and the biggest window? Isolate and characterize each variable and the interplay between them. Change no more (and no less) than one variable per experiment.
III. On Getting Famous
1. Treat all surprises as discoveries.
2. Seek new paradigms in your data.
3. Set aesthetic standards for you research.
4. Do not assume your project is complete until every dissonance in resolved and no new conclusions are reached during a significant period of vigorous data gathering and analysis.
5. Never record data on paper towels.