You can use both. The default one is the full version (float32). To use the slim version, you can do the following:
from transformers import GPTJForCausalLM
import torch

model = GPTJForCausalLM.from_pretrained("EleutherAI/gpt-j-6B", revision="float16", torch_dtype=torch.float16, low_cpu_mem_usage=True)