Error initializing SparkContext : ‘sparkDriver’ failed after 16 retries!

Hi All,

I just came across this error ‘sparkDriver’ failed after 16 retries!’ while starting the spark shell.

spark-scala

From the above screenshot, I came to know the error occurred due to the SparkUI port was not available. For this scenario I used Cloudera sandbox using Oracle VirtualBox.

I just solved this issue by the following steps

  1. Go to /usr/lib/spark/bin (path for cloudera sandbox)
  2. Open load-spark-env.sh and add this export SPARK_LOCAL_IP=’127.0.0.1′ (you need to be root user)
  3. Save the file and type ‘spark-shell’ to get sparkcontext initialized

Thanks for reading this post. Subscribe my blog for  some awesome posts.

Solve for India – Chennai

Hi Readers,

Today, I went to solve for India workshop conducted by GDG India. The workshop started with Welcome Note from GDG speakers and continued with sessions like designing for Indian Users target 2020, building for billion framework, kubernetes, firebase, GCP console and a 45 minutes open house session.

#SolveForIndia

Machine Learning Roadshow – Chennai

Hi Readers,

Today, I have attended Machine Learning workshop conducted by Google. It was a full day tech session. Morning session covered machine learning basics, use cases explained, tensorflow 101 and afternoon session covered with machine learning & tensorflow code labs (hands-on in real GCP environment).

#MLRoadshowIndia