You signed a financial agreement stating if you withdrew from class after a set number of days or weeks, you understood you would be financially liable. The contract didn't say "unless you really can't help it". The Federal Student Loan you took out (and it is a governemnt loan, or they wouldn't garnish your tax refund) also has a contract, that you cannot take a leave of absence of over 28 days or you will no longer be considered a student and will have to begin to repay your loan in full. Because you dropped out (regardless of the reason) your student loan would not pay for the class you were in at the time, that's why the school had to collect it from you directly. Any school will do this, not just University of Phoenix.
