Thinking about job histories and priorities

[sawyl]

I've recently spent some time churning through LoadLeveler's job history file, trying to tease out some trends from the mass of data. What I really want is a way to quantify the relationship between project priority and step queue time, in the hope that this will show that higher priority projects spend less time queuing. Unfortunately the picture is greatly complicated by the fact that step sizes and durations differ between projects, and that the backfill scheduling algorithm is more likely to select smaller, shorter jobs regardless of their priority.

If only I had a better grasp of statistical methods...