The I/O performance of HDFS with TestDFSIO.
% hadoop jar hadoop-test-0.20.2-cdh3u3b.jar TestDFSIO -write -nrFiles 10 -fileSize 1000
----- TestDFSIO ----- : write
Date & time: Thu Jul 26 18:50:11 PDT 2012
Number of files: 10
Total MBytes processed: 10000.0
Throughput mb/sec: 163.4360801490537
Average IO rate mb/sec: 167.77435302734375
IO rate std deviation: 25.658150459575825
Test exec time sec: 19.329
hadoop jar hadoop-test-0.20.2-cdh3u3b.jar TestDFSIO -read -nrFiles 10 -fileSize 1000
----- TestDFSIO ----- : read
Date & time: Thu Jul 26 19:22:14 PDT 2012
Number of files: 10
Total MBytes processed: 10000.0
Throughput mb/sec: 374.6721618583739
Average IO rate mb/sec: 375.14581298828125
IO rate std deviation: 13.625353109608241
Test exec time sec: 17.311
The communication performance of Apache Hama with Bench tool.
% hama jar hama-examples-0.5.0.jar bench 16 100000 32
...
12/07/26 21:12:16 INFO bsp.BSPJobClient: Current supersteps number: 30
12/07/26 21:12:19 INFO bsp.BSPJobClient: Current supersteps number: 31
12/07/26 21:12:22 INFO bsp.BSPJobClient: Current supersteps number: 32
12/07/26 21:12:25 INFO bsp.BSPJobClient: The total number of supersteps: 32
12/07/26 21:12:25 INFO bsp.BSPJobClient: Counters: 8
12/07/26 21:12:25 INFO bsp.BSPJobClient: org.apache.hama.bsp.JobInProgress$JobCounter
12/07/26 21:12:25 INFO bsp.BSPJobClient: LAUNCHED_TASKS=162
12/07/26 21:12:25 INFO bsp.BSPJobClient: org.apache.hama.bsp.BSPPeerImpl$PeerCounter
12/07/26 21:12:25 INFO bsp.BSPJobClient: SUPERSTEPS=32
12/07/26 21:12:25 INFO bsp.BSPJobClient: SUPERSTEP_SUM=5184
12/07/26 21:12:25 INFO bsp.BSPJobClient: MESSAGE_BYTES_TRANSFERED=10404951552
12/07/26 21:12:25 INFO bsp.BSPJobClient: TIME_IN_SYNC_MS=10300386
12/07/26 21:12:25 INFO bsp.BSPJobClient: TOTAL_MESSAGES_SENT=1036800000
12/07/26 21:12:25 INFO bsp.BSPJobClient: TOTAL_MESSAGES_RECEIVED=518400000
12/07/26 21:12:25 INFO bsp.BSPJobClient: MESSAGE_BYTES_RECEIVED=10404951552
Job Finished in 93.33 seconds
Smart post admin
ReplyDeleteI hope to visit my blog and subscribe to me :)
Gardnerella Vaginalis and Mucopurulent Cervicitis