Setup Apache Spark On Windows

download spark and unzip E.g

spark needs hadoop jars. download hadoop binaries for windows (hadoop 2.6.0) from

unzip hadoop at some locaiton e.g.


If your java_home or hadoop_home path contains space charcters in it ' ', you will need to convert to path to short paths:

  • Create a batch script with following contents

echo %~s1

  • Run the above batch script file from java_home directory to get the short path for java-home
  • Run the above batch script file from hadoop_home directory to get the short path for hadoop_home

set java_home=short path obtained from above command
set hadoop_home=short path obatained from above command.

Run following command and copy the classpath generated by the command for next step

         %HADOOP_HOME%\bin\hadoop classpath

under spark_home\conf, create a file named "spark-env.cmd" like below

@echo off
set HADOOP_HOME=D:\Utils\hadoop-2.7.1

on Command prompt

          cd %spark_home%\bin
          set SPARK_CONF_DIR=%SPARK_HOME%\conf
          spark-shell.cmd //To start spark shell
          spark-submit.cmd   //To submit spark job

Refer below to create a spark word count example

To run a spark job (written using Java) word count example from above URL

          spark-submit --class org.sparkexample.WordCount --master local[2]  your_spark_job_jar  Any_additional_parameters_needed_by_your_job_jar

References :