If your audio midi setup takes an age to load, and you get the message ‘the midi server cant be opened’

it might be to do with protools.
i run protools 8
and osx lion.

everything worked fine if I was plugged into my protools interface, but when the interface was unplugged, I got the above error.

but when my protools hardware was unplugged I couldnt open audio midi setup, and any app that I opened took an age because it had to wait for the timeout to occur because it couldnt find the protools hardware.

i simply went to /Library/Audio/Midi Drivers

and deleted the digidesign files in there.

it means that my midi interface isnt available when running protools, but I use ableton for my midi things anyway.
it turns out that there isnt a driver update either, so you have to upgrade to protools 9 or protools 10 if you want to get it all to work properly.