delete queued alerts

We had add an 'a-ha' moment the other day when our Hobbit server lost network connectivity, and once rebooted it blasted out basically every alert configured.  If this were to happen again, is there a way to silence or delete queued alerts before rebooting?

I'm also curious how this community handles a case like that.