Shiroyama Park (southwest of Kagoshima Station) is up on a hill, giving you a fine view of the city and Kagoshima Bay through the archway of Nanshu Shrine.
The most rewarding view of Kagoshima Bay is from Shiroyama Park.