Is it crazy to not rely on a caching system like memcached nowadays ( for dynamic sites )?
Caching, when it works right (==high hit rate), is one of the few general-purpose techniques that can really help with latency -- the harder part of problems generically describes as "performance". You can enhance QPS (queries per second) measures of performance just by throwing more hardware at the problem -- but latency doesn't work that way (i.e., it doesn't take just one month to make a babies if you set nine mothers to work on it;-).
However, the main resource used by caching is typically memory (RAM or disk as it may be). As you mention in a comment that the only performance problem you observe is memory usage, caching wouldn't help: it would just earmark some portion of memory to use for caching purposes, leaving even less available as a "general fund". As a resident of California I'm witnessing first-hand what happens when too many resources are earmarked, and I couldn't recommend such a course of action with a clear conscience!-)
I was just reviewing one of my client's applications which uses some old outdated php framework that doesn't rely on caching at all and is pretty much completely database dependent.
I figure I'll just rewrite it from scratch because it's really outdated and in this rewrite I want to implement a caching system. It'd be nice if I could get a few pointers if anyone has done this prior.

Rewrite will be done in either PHP or Python
Would be nice if I could profile before and after this implementation
I have my own server so I'm not restricted by shared hosting