Can a dedicated GPU use/share system RAM in addition to the VRAM included with it? The Macbook Pro laptop, running OS X, has a AMD GPU which includes graphics memory of 256 MB. Is it possible for the machine to run an application that requires 512 MB video memory by sharing some of the system RAM?

(There is also an integrated graphics card but this has a max (shared) VRAM of around 380 MB. AFAIK it cannot go over this.)

Is there any way an application with video memory requirements of 512MB could be run on this machine?
I believe "NO" is also an answer :(

The physical and design of the discrete (AMD in your case, nVidia in other MBPs) is such that their VRAM is on separate bus connected only to the GPU.

The IGPs (which are part of the CPU's die, ie. physical chip) shared the DRAM of the computer. That is where the term "shared" memory comes in, because the CPU and IGP have a shared access to the "computer's" RAM :)

Thus, if your application needs 512MB display memory, then you are out of luck with your current MBP ;(