Plastic bricks with an aluminum foil or tin cover, or "plastic concrete".
Plastic Bricks
A plastic brick building in Columbia from here.
This example is small scale, but there is no good reason that bigger blocks perhaps as big as those in the pyramids or medieval castles, but much lighter to place, couldn't be used. For example, a version of this technology developed in New Zealand can be designed to use much larger, less finely polished bricks. Imagine cubic yard sized bales of plastic turned into smooth bricks.
The world is full of plastic crap that would be readily available in a post-apocalyptic world (it could be mined from landfills or gathered from floating plastic in the sea, if necessary), can often be compressed, melted and reformed into convenient shapes. This technique is used to make plastic bricks in some developed countries. Landfills are more ubiquitous than quality quarries or old growth forests which would be good in a time period when long distance transportation of low value building materials might be a problem.
Also, once placed, the plastic bricks can be welded into a single solid surface with heat. And, plastic is forever. 561
Most kinds don't biodegrade over time. Animals don't eat it. It isn't friendly to plants that could root in it. If it's thick enough it can handle rough weather. It isn't as brittle as glass or ceramic or brick which would be good in places with earthquakes (perhaps fracking induced as people chase scare hydrocarbon supplies) or major storms.
Also, plastic is water proof which would be attractive in coastal areas put at risk of periodic storm surges in a world where sea levels had risen due to global warming.
While the bricks in the image above are bland and gray, plastic can come in many colors and random mix of pre-apocalypse marketing driven colors could provide an ugly/beautiful aesthetic that characters could muse upon adding "local color" to the story.
The (optional) aluminum or tin cover foil would prevent it from melting in the sun or from minimal stray fire exposure, but with sufficient ignition and intentionally poked holes in the metal cover it could burn.
Plastic Concrete
A wall made of "plastic concrete":
Recent RPI Masters of Architecture graduate <PERSON> has devised a way to reuse waste plastic as an aggregate in cement, circumventing the energy-intensive process of plastic recycling. By grinding up landfill-bound plastic and mixing it with portland cement, <PERSON> was able to create a material just as strong as traditional concrete made with mined aggregate.
The trouble with plastic concrete is that the temperature at which the aggregate melts is much lower than conventional aggregates used in concrete. You'd want to have your "poor" protagonists set up fires along the base like an old stone oven or fire pit and then assault the weakened material after it had baked for a few days. 561