Hamburg, a graph computing framework on Hadoop
As mentioned ago, I've been forming up the Hamburg project with Hyunsik Choi. Let's see more detail in the diagram of computing method of Hamburg based on BSP model.
Each worker will process the data fragments stored locally. And then, We can do bulk synchronization using collected communication data. The 'Computation' and 'Bulk synchronization' can be performed iteratively, Data for synchronization can be compressed to reduce network usage.
Plainly, It aims to improve the performance of traverse operations in Graph computing. For example, to explores all the neighboring nodes from the root node using Map/Reduce (FYI, Breadth-First Search (BFS) & MapReduce), We need a lot of iterations to get next vertex per-hop time.
If (same condition as before) do BFS using Hamburg, It will cause a lowering the cost of iterations. Let's assume the graph looks like presented below:
The graph was stored in Hbase on distributed system as above. The root is 1. Then, we need only one 'Bulk synchronization' between server2 and server3 with Hamburg. Rests will be calculated on local machine.
Almost graph algorithms are similar with this case.
Updated - 26 August, 2009 : See also, Inference anatomy of the Google Pregel
Each worker will process the data fragments stored locally. And then, We can do bulk synchronization using collected communication data. The 'Computation' and 'Bulk synchronization' can be performed iteratively, Data for synchronization can be compressed to reduce network usage.
Plainly, It aims to improve the performance of traverse operations in Graph computing. For example, to explores all the neighboring nodes from the root node using Map/Reduce (FYI, Breadth-First Search (BFS) & MapReduce), We need a lot of iterations to get next vertex per-hop time.
If (same condition as before) do BFS using Hamburg, It will cause a lowering the cost of iterations. Let's assume the graph looks like presented below:
The graph was stored in Hbase on distributed system as above. The root is 1. Then, we need only one 'Bulk synchronization' between server2 and server3 with Hamburg. Rests will be calculated on local machine.
Almost graph algorithms are similar with this case.
Updated - 26 August, 2009 : See also, Inference anatomy of the Google Pregel
Server2 needs to tell Server3 that the vertex 6 distance is 2 so that Server3 can assign distance 3 for vertex 7.
ReplyDeleteSince Server2 can only see partial view for vertex 2 neighbors, I think Server2 also has to tell Server1 about the distance of vertex 2 (which is 2). Then Server1 decides the final distance of vertex 2 (which is 1).
Or the other way around: Server1 already decides the distance of vertex 2 is 1 and needs to tell Server2 about it (since from Server2 point of view, the disntance of vertex 2 is 2).
Eitherway, Server1 and Server2 has to communicate, isn't it?
The start node assumed as a vertext 1. Hence, Server1's processing will be locally done w/o any communicate. There is one communication between Server2 and Server3.
ReplyDeleteNice info regarind graph computing framework on Hadoop my sincere thanks for sharing please continue to share this post
ReplyDeleteHadoop Training in Chennai
Hello admin, I have read your blog, it was very nice to read & I am getting useful information’s through your blog. Keep update your blog. AWS Training in Bangalore | Big Data Hadoop Training in Bangalore
ReplyDeleteThe share you really give us excitement. Thanks for your sharing. If you feel tired at work or study try to participate in our games to bring the most exciting feeling. Thank you!
ReplyDeletePython Online Training | Learn Python Online
niceblog
ReplyDeleteVery good blog helpful to everyone AWS training in bangalore
ReplyDeleteHadoop training in bangalore
Tableau training in bangalore
PHP training in bangalore
Android training in bangalore
Digital marketing training in bangalore
Nice blogiteanz
ReplyDeletemachine learning training in bangalore
DEvops Training in Bnagalore
powershell training in bangalore
Iot Training in Bnagalore
Great information. Thank you for sharing such post
ReplyDeleteVery nice post. Awesome article... Really helpful
ReplyDeleteNice Article Keep Posting it
ReplyDeleteIot training in Bangalore
Iteanz
very nice blog it was useful
ReplyDeletevery nice blog It was useful
ReplyDeletenice blog
ReplyDeletemyTectra Profile | Trainingindustry.com
myTectra | Instagram
myTectra | Youtube
Nice blog
ReplyDeletejquery interview questions
sql interview questions
Nice blog
ReplyDeletetalend training
docker training
Great post and informative blog.it was awesome to read, thanks for sharing this great content to my vision.
ReplyDeleteGood discussion.
PMP Training Course in Bangalore
PMP Training Course in Dammam
PMP Training Course in Dubai
PMP Training Course in Jeddah
PMP Training Course in Riyadh
Thanks for sharing this blog, I am reading your post from the beginning, it was so interesting to read. Visit for
ReplyDeleteWeb Design Company in Delhi
best hiv doctor in delhi
ReplyDeleteEnglish Song lyrics
ReplyDeletelg fridge repair in gurgaon
ReplyDeletePEP Treatment in delhi
ReplyDelete