The natural foods and organic retailer, which has outlets in the UK, is said to have overstated weights of pre-packaged meat, dairy and baked products
New York's Department of Consumer Affairs (DCA) commissioner Julie Menin said it was "incredibly troubling".
Whole Foods said it rejected the DCA's "overreaching allegations" and that customers were its number one priority.
The New York City consumer affairs office said on Wednesday it had found mislabelled weights on 80 types of pre-packaged products it tested.
"As a large chain grocery store, Whole Foods has the money and resources to ensure greater accuracy and to correct what appears to be a widespread problem,'' Ms Menin said.
Last year, Whole Foods agreed to pay $800,000 in penalties and improve pricing accuracy after an investigation into alleged pricing irregularities in California.
Whole Foods dismissed the allegations, saying that it cooperated fully with the consumer affairs department until it made "grossly excessive monetary demands" to settle the dispute.
The company has a reputation for high prices, with some people referring to it as "Whole Paycheque". However, Whole Foods has announced plans for a new chain of smaller stores with lower prices.
Upmarket US grocery chain Whole Foods has been accused in New York of routinely overcharging customers.