Hadoop jar - System cannot find the path specified

Question

I'm trying to run a jar file with hadoop using command

hadoop jar test.jar org.ipiran.hadoop.sample.TestMapReduce /user/data.txt /output/1.txt 4

But it returns error

Exit code: 1 Exception message: System cannot find the path specified

I tried without specifying the class but got the same result. I have file data.txt. My main java code

public int run(String[] args) throws Exception {
    Configuration conf = this.getConf();
    conf.set("fs.defaultFS", "hdfs://localhost:9000/");
    conf.set("fs.hdfs.impl", org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());
    conf.set("fs.file.impl", org.apache.hadoop.fs.LocalFileSystem.class.getName());
    conf.set("source", args[2]);
    long numUpdated = 1;
    int code = 0;
    int numIterations = 1;
    FileSystem hdfs = FileSystem.get(conf);
    while (numUpdated > 0) {
        logger.info("Iteration: " + numIterations);
        String input, output;
        Job job = Job.getInstance(conf, "word count");
        if (numIterations == 1) {
            input = args[0];
        } else {
            input = args[1] + "-" + (numIterations - 1);
        }
        output = args[1];// + "-" + numIterations;

        job.setJarByClass(TestMapReduce.class);
        job.setMapperClass(testmap.class);
        job.setReducerClass(TestReduce.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(Text.class);
        FileInputFormat.addInputPath(job, new Path(input));
        FileOutputFormat.setOutputPath(job, new Path(output));
        code = job.waitForCompletion(true) ? 0 : 1;

        Counters jobCounters = job.getCounters();
        numUpdated = jobCounters.
            findCounter(MoreIterations.numUpdated).getValue();
        if (numIterations > 1) {
            hdfs.delete(new Path(input), true);
            logger.info("Updated: " + numUpdated);
        }
        numIterations += 1;
    }
    return code;
}

I'm using windows 10. Does anybody know what is wrong here?

Answer 1

You are using Windows 10, however you are treating it is if you are using Linux.

In Linux "/someDir/someFile means that there is a file under a directory that is under the root directory.

You have to change the paths of your data files to some location that is compitable with windows.

By the way, do you have hadoop installed on your machine ?

I would recommend using a linux VM, you can use Cloudera VM as it has everything installed

https://www.cloudera.com/downloads/quickstart_vms/5-12.html