Always read the small print

[sawyl]

Having made a few changes to LoadLeveler to enable preemption for reservations, I ran a few naive tests. These apparently confirmed my expectations: running jobs were bumped and put back into the queue in order to allow a new reservation to be forced in. But when others started to use preemption in earnest, I was surprised to discover that their llmkres commands had failed with insufficient resource errors.

Doing some digging, I finally isolated the cause: I could create preempting reservations because I had LoadLeveler admin rights while all normal users simple got an error indicating that the reservation could not be created. Sure enough, when I checked the documentation, I found the answer staring me in the face:

# RESERVATION_PRIORITY to define whether LoadLeveler administrators may reserve nodes on which running jobs are expected to end after the start time for the reservation.

...

This keyword value applies only for LoadLeveler administrators; other reservation owners do not have this capability.

Most unfortunate.