Error initializing SparkContext : ‘sparkDriver’ failed after 16 retries!

Hi All,

I just came across this error ‘sparkDriver’ failed after 16 retries!’ while starting the spark shell.


From the above screenshot, I came to know the error occurred due to the SparkUI port was not available. For this scenario I used Cloudera sandbox using Oracle VirtualBox.

I just solved this issue by the following steps

  1. Go to /usr/lib/spark/bin (path for cloudera sandbox)
  2. Open and add this export SPARK_LOCAL_IP=’′ (you need to be root user)
  3. Save the file and type ‘spark-shell’ to get sparkcontext initialized

Thanks for reading this post. Subscribe my blog for  some awesome posts.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.