Exploratory Data Analysis
Do Collect All Data Possible …. (within reason)
- One year Santa Claus had to repeat all our experiments
- paper deadline just after new year!
- We had collected number of branches in search tree
- but not the number of backtracks
- performance scaled with backtracks, not branches
- all experiments had to be rerun
Don’t Kill Your Machines
- We have got into trouble with sysadmins
- … over experimental data we never used
- Often the vital experiment is small and quick