Tried importing Java 8 JDK for PySpark, but PySpark still won't let me start a session
This problem is coming with spark 2.4. please try spark 2.3.
Ok here's my basic information before I go on:
MacBook Pro: OS X 10.14.2
Python Version: 3.6.7
Java JDK: V8.u201
I'm trying to install the Apache Spark Python API (PySpark) on my computer. I did a conda installation: conda install -c conda-forge pyspark
It appeared that the module itself was properly downloaded because I can import it and call methods from it. However, opening the interactive shell with myuser$ pyspark gives the error: 
No Java runtime present, requesting install.
Ok that's fine. I went to Java's download page to get the current JDK, in order to have it run, and downloaded it on Safari. Chrome apparently doesn't support certain plugins for it to work (although initially I did try to install it with Chrome). Still didn't work.
Ok, I just decided to start trying to use it.
from pyspark.sql import SparkSession It seemed to import the module correctly because it was auto recognizing SparkSession's methods. However,
spark = SparkSession.builder.getOrCreate() gave the error:
Exception: Java gateway process exited before sending its port number
Reinstalling the JDK doesn't seem to fix the issue, and now I'm stuck with a module that doesn't seem to work because of an issue with Java that I'm not seeing. Any ideas of how to fix this problem? Any and all help is appreciated.