July 20, 2009

Hamburg, a graph computing framework on Hadoop

As mentioned ago, I've been forming up the Hamburg project with Hyunsik Choi. Let's see more detail in the diagram of computing method of Hamburg based on BSP model.




Each worker will process the data fragments stored locally. And then, We can do bulk synchronization using collected communication data. The 'Computation' and 'Bulk synchronization' can be performed iteratively, Data for synchronization can be compressed to reduce network usage.

Plainly, It aims to improve the performance of traverse operations in Graph computing. For example, to explores all the neighboring nodes from the root node using Map/Reduce (FYI, Breadth-First Search (BFS) & MapReduce), We need a lot of iterations to get next vertex per-hop time.

If (same condition as before) do BFS using Hamburg, It will cause a lowering the cost of iterations. Let's assume the graph looks like presented below:




The graph was stored in Hbase on distributed system as above. The root is 1. Then, we need only one 'Bulk synchronization' between server2 and server3 with Hamburg. Rests will be calculated on local machine.




Almost graph algorithms are similar with this case.

Updated - 26 August, 2009 : See also, Inference anatomy of the Google Pregel

July 7, 2009

Massive DDoS attacks

The targets as described below hit by attacks :

Korea sites:

www.president.go.kr, www.mnd.go.kr, www.mofat.go.kr, www.assembly.go.kr, www.usfk.mil, blog.naver.com, mail.naver.com, banking.nonghyup.com, ezbank.shinhan.com, ebank.keb.co.kr, www.hannara.or.kr, www.chosun.com, www.auction.co.kr

US sites:

www.whitehouse.gov, www.faa.gov, www.dhs.gov, www.state.gov, www.voanews.com, www.defenselink.mil, www.nyse.com, www.nasdaq.com, finance.yahoo.com, www.usauctionslive.com, www.usbank.com, www.washingtonpost.com, www.ustreas.gov

It seems, are of a political character. Anyway, please check the 'msiexec2.exe' on your computer! -- Trojan.Win32.DDoS-Agent.33841

- S. Korean Web Sites Hit by Cyber Attacks
- MS제로데이 취약 패치 공개...DDoS 원인일 수 있어