We did that but the dogs got really good at catching them. Turns out if you splash the water out of the trough the fish come with and are really easy to catch.

It was a fun game of adding more bricks until splashing was harder to do, but the bold fish always got eaten. We kept a timid one for years