Within Databricks, it is possible to import JAR libraries and run the classes in them on your clusters. I will create a very simple piece of Scala code to print out the first 100 elements of the Fibonacci series as
BigInt values, locally on my Centos Linux server. I will compile my class into a JAR file using SBT, run it locally to check the result, and then run it on my Databricks cluster to compare the results. The code looks as following:
import org.apache.spark.SparkContext import org.apache.spark.SparkContext._ import org.apache.spark.SparkConf object db_ex1 extends App { val appName = "Databricks example 1" val conf = new SparkConf() conf.setAppName(appName) val sparkCxt = new SparkContext(conf) var seed1:BigInt = 1 var ...
No credit card required