Project1/log_test_mapreduce.txt at master · BigData-MielPops/Project1 · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
bash-4.1# yarn jar /home/workspace/Project1-mapreduce/target/project1-0.0.1-SNAPSHOT.jar project1.Job1 input/1999_2006.csv output/test_utilizzo_mapreduce
No year arguments given, will use only 1 Reducer.
17/05/06 11:09:56 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/05/06 11:09:57 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
17/05/06 11:09:57 INFO input.FileInputFormat: Total input paths to process : 1
17/05/06 11:09:57 INFO mapreduce.JobSubmitter: number of splits:1
17/05/06 11:09:57 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1494079809711_0014
17/05/06 11:09:57 INFO impl.YarnClientImpl: Submitted application application_1494079809711_0014
17/05/06 11:09:57 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1494079809711_0014/
17/05/06 11:09:57 INFO mapreduce.Job: Running job: job_1494079809711_0014
17/05/06 11:10:03 INFO mapreduce.Job: Job job_1494079809711_0014 running in uber mode : false
17/05/06 11:10:03 INFO mapreduce.Job:  map 0% reduce 0%
17/05/06 11:10:08 INFO mapreduce.Job:  map 100% reduce 0%
17/05/06 11:10:14 INFO mapreduce.Job:  map 100% reduce 100%
17/05/06 11:10:15 INFO mapreduce.Job: Job job_1494079809711_0014 completed successfully
17/05/06 11:10:16 INFO mapreduce.Job: Counters: 49
        File System Counters
                FILE: Number of bytes read=241748
                FILE: Number of bytes written=695841
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=5174717
                HDFS: Number of bytes written=5919
                HDFS: Number of read operations=6
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=2
        Job Counters
                Launched map tasks=1
                Launched reduce tasks=1
                Data-local map tasks=1
                Total time spent by all maps in occupied slots (ms)=3702
                Total time spent by all reduces in occupied slots (ms)=3863
                Total time spent by all map tasks (ms)=3702
                Total time spent by all reduce tasks (ms)=3863
                Total vcore-seconds taken by all map tasks=3702
                Total vcore-seconds taken by all reduce tasks=3863
                Total megabyte-seconds taken by all map tasks=3790848
                Total megabyte-seconds taken by all reduce tasks=3955712
        Map-Reduce Framework
                Map input records=8823
                Map output records=8823
                Map output bytes=224096
                Map output materialized bytes=241748
                Input split bytes=116
                Combine input records=0
                Combine output records=0
                Reduce input groups=7082
                Reduce shuffle bytes=241748
                Reduce input records=8823
                Reduce output records=73
                Spilled Records=17646
                Shuffled Maps =1
                Failed Shuffles=0
                Merged Map outputs=1
                GC time elapsed (ms)=50
                CPU time spent (ms)=4590
                Physical memory (bytes) snapshot=462798848
                Virtual memory (bytes) snapshot=1476661248
                Total committed heap usage (bytes)=297795584
        Shuffle Errors
                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
                WRONG_MAP=0
                WRONG_REDUCE=0
        File Input Format Counters
                Bytes Read=5174601
        File Output Format Counters
                Bytes Written=5919