Error initializing SparkContext : ‘sparkDriver’ failed after 16 retries!

Hi All,

I just came across this error ‘sparkDriver’ failed after 16 retries!’ while starting the spark shell.

spark-scala

From the above screenshot, I came to know the error occurred due to the SparkUI port was not available. For this scenario I used Cloudera sandbox using Oracle VirtualBox.

I just solved this issue by the following steps

  1. Go to /usr/lib/spark/bin (path for cloudera sandbox)
  2. Open load-spark-env.sh and add this export SPARK_LOCAL_IP=’127.0.0.1′ (you need to be root user)
  3. Save the file and type ‘spark-shell’ to get sparkcontext initialized

Thanks for reading this post. Subscribe my blog for  some awesome posts.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s