I have noticed this same issue, both in 7.04 and 8.10. The "first time" after
a boot that I activate the the "shutdown" panel, it takes at least 2 to 3
minutes to initiate. I'm usually doing this for a screen lock rather than a
shut down.

Now after this first time - the panel activates within a second or two after
selection. Acts normally until the next restart, then back to several minutes
for the next "first time". I suspect there is an underlying service which
controls the shutdown/logoff/screen lock panel which is not starting at boot,
and is only called and loaded when it's asked for. The issue is a bit
irritating, but not to the point I've gone digging to fix it.