The way it works, Dan, is like this:
You drive up and press the button that says "Press To Order". Pressing it multiple times before anyone answers your call doesn't do anything. Do not let anyone tell you it does, because it doesn't.
After you press the button, your order is placed in line to be taken. If there is no line, of course you should be answered promptly. You are perfectly within your right to become irritated if they don't answer within about 15 seconds. Sonic has set the ideal reply time (the time it takes for someone to answer you) to 8 seconds with over 30 seconds being ridiculously slow. Each location has, on average, three speakers with which to answer calls.
However, if you do move to another stall after waiting for a few seconds or minutes, it will increase the amount of wait time you must experience because the system is not able to choose which orders it answers. It always just takes the next in line according to how long that particular stall has been waiting. When you moved, you were placed at the end of the line again.
However, I still say that you have a right to be irritated because the service at this location seems to be terribly slow.