How to access system display memory / frame buffer in a Java program?
directly access system display memory on Linux

You can't. Linux is a memory protected virtual address space operating system. Ohh, the kernel gives you access to the graphics memory through some node in /dev but that's not how you normally implement this kind of thing.
Also in Linux you're normally running a display server like X11 (or in the future something based on the Wayland protocol) and there might be no system graphics memory at all.

I have researched a bit and found that one way to achieve this is to capture the screen at a high frame rate (screen shot), convert it into RAW format, compress it and store it in an ArrayList.

That's exactly how its done. Use the display system's method to capture the screen. It's the only reliable way to do this. Note that if conversion or compression is your bottleneck, you'd have that with fetching it from graphics memory as well.
I am trying to create my own VNC client and would like to know how to directly access system display memory on Linux? So that I can send it over a Socket or store it in a file locally.
I have researched a bit and found that one way to achieve this is to capture the screen at a high frame rate (screenshot), convert it into RAW format, compress it and store it in an ArrayList.
But, I find this method a bit too resource heavy. So, was searching for alternatives.
Please, let me also know if there are other ways for the same (using Java or Python only)?