Question: I wish to write software which could essentially profile the CPU cache (L2,L3, possibly L1) and the memory, to analyze performance.

Am I right in thinking this is un-doable because there is no access for the software to the cache content? 

Another way of wording my Q: is there any way to know, from the OS/Application level, what data has been loaded into cache/memory?

EDIT: Operating System Windows or Linux and CPU Intel Desktop/Xeon

Answer: 
Cache performance is generally measured in terms of hit rate and miss rate.

There are many tools to do this for you. Check how [Valgrind does cache profiling](http://tldp.org/HOWTO/Valgrind-HOWTO/deeper.html). 

Also ***cache performance is generally measured on a per program basis***. Well written programs will result in a fewer cache misses and better cache performance and vice versa for poorly written code.

Measuring the **actual cache speed** is the headache of the hardware manufacturers and you can refer their manuals to know this value.

[Callgrind](http://valgrind.org/docs/manual/cl-manual.html)/[Cachegrind](http://wwwpub.zih.tu-dresden.de/~mlieber/practical_performance/04_callgrind.pdf) combination can help you track cache hits/misses
[This](https://stackoverflow.com/questions/10082517/simplest-tool-to-measure-c-program-cache-hit-miss-and-cpu-time-in-linux) has some examples.
TAU, an open-source profiler which works using PAPI can also be used. 

If however, you want to write a code to measure the cache statistics you can write a program using PAPI. PAPI allows the user to access the hardware counters without any need to know system architecture.
PMU uses Model Specific Registers, hence you must have the knwoledge of the registers to be used.

Perf allows for measurement of L1 and LLC (which is L2), Cachegrind, on the other hand allows the user to measure L1 and LLC (which can be L2 or L3, whichever the highest level cache is). Use Cachegrind only if you have no need of faster results because Cachegrind runs the program about 10X slower.