Posts

Showing posts from September, 2012

OSS App Hackathon @ National Information Society Agency

Image
Yesterday, there was a OSS App Hackathon arranged by the NIA (National Information Society Agency) in Seoul. I attended as a panel of judges w/ Prof. Lee of the Next, NHN University. A lot of people were in there.


You can read more details (Korean news) here:
 - http://news.naver.com/main/read.nhn?mode=LSD&mid=sec&sid1=105&oid=138&aid=0001997038

Fault-tolerance in Hama

Recently, Hama core committers Suraj Menon and Thomas Jungblut are working on Fault-tolerant BSP system. And I am trying to read the source code. Their design describes the new BSP computing system and API enabling checkpoint-based recovery. Furthermore, describes the confined recovery, which can be used to improve the cost and latency of recovery.

I didn't fully understand and test yet but quite nice!

PageRank with Apache Hama

The dataset contains 5,716,808 pages and 130,160,392 links and is unzipped ~1gb large. [oracle@bda03 hama-0.6.0-SNAPSHOT]$ hadoop dfs -ls edward Found 1 items -rw-r--r-- 3 oracle hadoop 1058414409 2012-08-28 22:16 /user/oracle/edward/links-simple-sorted.txt [oracle@bda03 hama-0.6.0-SNAPSHOT]$ bin/hama jar hama-examples-0.6.0-SNAPSHOT.jar pagerank /user/oracle/edward/links-simple-sorted.txt out 12/09/10 18:44:25 INFO bsp.FileInputFormat: Total input paths to process : 1 12/09/10 18:44:25 INFO net.NetworkTopology: Adding a new node: /switch1/192.168.10.3:50010 12/09/10 18:44:25 INFO net.NetworkTopology: Adding a new node: /switch1/192.168.10.5:50010 12/09/10 18:44:25 INFO net.NetworkTopology: Adding a new node: /switch1/192.168.10.16:50010 12/09/10 18:44:25 INFO net.NetworkTopology: Adding a new node: /switch1/192.168.10.14:50010 12/09/10 18:44:25 INFO net.NetworkTopology: Adding a new node: /switch1/192.168.10.15:50010 12/09/10 18:44:25 INFO net.NetworkTopology: Adding a new node:…