Now the Secret Service payroll tops 4,500 ( most of them agents ) , and the annual budget exceeds $ 500 million ( up 300 percent just since 1980 ) .
The Secret Service annual budget is over $ 500 million , said the report .