The long-term proposals for Hay Castle in Hay-on-Wye would allow it to open to the public for the first time.
Hay Castle Trust has been awarded £528,600 to help develop the plans which include an arts centre.
The castle was built in the late 12th Century by the powerful Norman Lord William de Braose.
It was sacked the following century and rebuilt by Henry III.
Castle House, the adjoining Jacobean mansion, was built in 1660 but was severely damaged by fire in 1939 and 1977.
The castle has been owned by Richard Booth, the man credited with transforming Hay into a global attraction for second-hand book lovers, but was purchased in 2011 by the Hay Castle Trust.
The Heritage Lottery Fund said the grant would help the trust to develop its plans.
It eventually hopes to secure more than £4.8m to put towards proposals which include conserving the medieval and Jacobean buildings, and creating a new centre for arts and culture.

The 26-year-old centre-back makes the move from Portuguese top-flight club subject to international clearance.
Igor Rossi told Hearts' website: "I am a leader, I play aggressively, with passion, and I like the ball at my feet to play from the back.
"From what I have seen, this is the Hearts way."
Igor Rossi spent four years in Portugal after moving from Brazilian club Internacional, where he was part of the side that won the 2010 Copa Libertadores.
But he only made two starts and two substitute appearances for their first team last season, while he made 10 appearances for Maritimo's second string in the Portuguese second tier.
Igor Rossi revealed that Hearts' director of football, Craig Levein, was pivotal in bringing him to Robbie Neilson's side.
"I had seen Scottish sides in Europe, particularly Celtic and the Scottish national side, and, when I was aware of Hearts' interest, I read up quite a bit about them and saw that they won the league by a huge margin over Rangers, so I was very impressed," he said.
"Robbie was very impressive and he explained to me his ambition for the club, where he expects us to be in the future and how he wants the team to play.
"Craig also spoke very well to me and I was immediately convinced that I should come to Edinburgh and play for the Hearts.
"I believe I will bring good experience from my five years playing in the Portuguese Division 1 and in the Europa League.
"I want to help the club challenge in every tournament we play in and I want to help my new team-mates.
"I'm also hoping to score some goals as I'm good in the air."

Slogans were painted on the monument at White Brae, Ligoniel, on the outskirts of north Belfast and two wreaths were removed.
The monument is a tribute to teenage brothers John and Joseph McCaig, and their colleague Dougald McCaughey, 23.
The off-duty fusiliers were shot dead at the site by the IRA in March 1971.
James Wright, chairman of Oldpark and Cavehill branch of the Royal British Legion, said: "Just like the cowards who in the dead of night murdered John, Joe and Dougald, these lowlife cowards cannot show their faces, they have to carry out their attacks under the cover of darkness.
"As a branch we will not be deterred in ensuring that the murder of these young soldiers who were off-duty and out socialising in Belfast will never be forgotten."
Sinn FÃ©in councillor for north Belfast, Gerry McCabe, also condemned the latest incident.
"The ongoing attacks on this memorial must stop.  They have gone on too long," he said.
"The actions of those involved in this are not representative of the people of this community and whoever is behind this should question what they are achieving."
A medieval castle which stands over a Powys town is to receive more than £500,000 as part of plans to restore it.
[NEXT_CONCEPT]
Hearts have added to their defensive options with the signing of Igor Rossi Branco on a one-year deal after the Brazilian's exit from Maritimo.
[NEXT_CONCEPT]
A memorial to three Scottish soldiers who were murdered by the IRA has been attacked for a 19th time, the Royal British Legion has said.