For 25 years, the Charles Dickens Centre in Rochester, Kent, has paid tribute to Britain's foremost author of the Victorian era.
The centre has celebrated Dickens' strong connections with Medway
However, a decline in visitor numbers has led to a plans by Medway Council to close the centre in October and relocate Rochester Library to the site.
The council says it is not financially viable to keep Eastgate House open.
News of the closure has been greeted by a mixture of resigned understanding and confusion by Dickens enthusiasts.
Lee Auld, curator at Dickens House in nearby Broadstairs, said she was "mystified" by the decision.
She said visitors to Dickens House - once the home of a woman on whom Dickens based the character Betsey Trotwood in his novel David Copperfield - had expressed concern about the closure.
"It doesn't make sense - Dickens is incredibly popular and we have seen visitor numbers increase significantly over the past four years, so I can't believe it wouldn't be financially viable to keep the centre open," she said.
"I don't think Dickens will ever be forgotten but I think it's important to question the loss of a museum that recalls our heritage."
Medway Council said the Grade I listed building - used by Dickens as The Nuns' House in The Mystery of Edwin Drood and Westgate House Seminary for young ladies in The Pickwick Papers - would serve the community better as a library and literary centre.
It also cited increasing competition from other attractions as a reason for declining visitor numbers.
Andrew Xavier, director of the Dickens House Museum in London, said he understood the threat posed by other attractions.
He said Dickens remained very popular in the United States but in the UK, the large number of heritage attractions made it difficult to stand out.
He also said it was difficult to blame attractions for falling visitor numbers following recent world events such as 9/11 and the war in Iraq.
Visitors will still be able to see the Swiss Chalet where Dickens worked
"To keep interest alive in an age of computer games, Harry Potter and the internet, it is important for heritage attractions to evolve and stay relevant.
"We do things like visit schools, hold late night readings and offer children the chance to handle objects like quills.
"We are also considering holding a modern exhibition that has nothing whatsoever to do with Dickens so we can invite new audiences through our doors."
Medway Council said that while the closure was regrettable, Dickens' connections with the Medway area - where the author spent much of his life - would be maintained.
In the grounds of Eastgate House, visitors will still be able to see the Swiss chalet where the author worked on The Mystery of Edwin Drood before his death in 1870.
Dickens festivals will also continue to be held twice a year in Rochester.
Work on the new library will start immediately after the closure of the Dickens Centre on 31 October.
Before then, the council will conduct a public consultation and hold a three-day exhibition outlining its plans for the library.