So I learned last night that our submission to NSDI was accepted. It described the methodology for a tool we built which we can Distalyzer. Distalyzer works to help developers diagnose performance problems in their systems. It works by utilizing a minimal amount of structure from the logs, and then doing two kinds of analysis (t-tests and dependency networks) to discover the most relevant and most divergent aspects of groups of log instances. We have used it to find bugs in Transmission and H-Base, and applied it to other another system’s problem (TritonSort) to show reductions in effort needed to diagnose the problem.
Soon there will be a blog post at our research group website which describes it in more detail, but I wanted to go ahead and post about the good news.
Congratulations!
Alex – thanks for your help! Incidentally, if you have other logs, past or present, that Distalyzer might be able to help with, we are working on an updated version for KDD, tailored more toward solving the problems of cluster configs rather than software issues. We’re working on it with MPI jobs running on clusters, but Tritonsort-or other-logs would be helpful. If you are having any current problems, that would be particularly good for us to work on. -Chip