Hamburg, a graph computing framework on Hadoop

As mentioned ago, I've been forming up the Hamburg project with Hyunsik Choi. Let's see more detail in the diagram of computing method of Hamburg based on BSP model.




Each worker will process the data fragments stored locally. And then, We can do bulk synchronization using collected communication data. The 'Computation' and 'Bulk synchronization' can be performed iteratively, Data for synchronization can be compressed to reduce network usage.

Plainly, It aims to improve the performance of traverse operations in Graph computing. For example, to explores all the neighboring nodes from the root node using Map/Reduce (FYI, Breadth-First Search (BFS) & MapReduce), We need a lot of iterations to get next vertex per-hop time.

If (same condition as before) do BFS using Hamburg, It will cause a lowering the cost of iterations. Let's assume the graph looks like presented below:




The graph was stored in Hbase on distributed system as above. The root is 1. Then, we need only one 'Bulk synchronization' between server2 and server3 with Hamburg. Rests will be calculated on local machine.




Almost graph algorithms are similar with this case.

Updated - 26 August, 2009 : See also, Inference anatomy of the Google Pregel

Comments

  1. Server2 needs to tell Server3 that the vertex 6 distance is 2 so that Server3 can assign distance 3 for vertex 7.

    Since Server2 can only see partial view for vertex 2 neighbors, I think Server2 also has to tell Server1 about the distance of vertex 2 (which is 2). Then Server1 decides the final distance of vertex 2 (which is 1).

    Or the other way around: Server1 already decides the distance of vertex 2 is 1 and needs to tell Server2 about it (since from Server2 point of view, the disntance of vertex 2 is 2).

    Eitherway, Server1 and Server2 has to communicate, isn't it?

    ReplyDelete
  2. The start node assumed as a vertext 1. Hence, Server1's processing will be locally done w/o any communicate. There is one communication between Server2 and Server3.

    ReplyDelete
  3. Nice info regarind graph computing framework on Hadoop my sincere thanks for sharing please continue to share this post
    Hadoop Training in Chennai

    ReplyDelete
  4. Hello admin, I have read your blog, it was very nice to read & I am getting useful information’s through your blog. Keep update your blog. AWS Training in Bangalore | Big Data Hadoop Training in Bangalore

    ReplyDelete
  5. The share you really give us excitement. Thanks for your sharing. If you feel tired at work or study try to participate in our games to bring the most exciting feeling. Thank you!
    Python Online Training | Learn Python Online

    ReplyDelete
  6. Great post and informative blog.it was awesome to read, thanks for sharing this great content to my vision.
    Good discussion.
    PMP Training Course in Bangalore
    PMP Training Course in Dammam
    PMP Training Course in Dubai
    PMP Training Course in Jeddah
    PMP Training Course in Riyadh

    ReplyDelete
  7. Thanks for sharing this blog, I am reading your post from the beginning, it was so interesting to read. Visit for
    Web Design Company in Delhi

    ReplyDelete
  8. THANKS FOR INFORMATION

    you can search low-cost website with high-quality website functions.
    Today Join Us
    Call: +91 - 8076909847

    website designing company in india

    levantro
    best interior designer in delhi


    livewebindia
    best website designing company in delhi

    seo company delhi

    Best It Service Provider:

    1. Website Designing And Development.
    2. SEO Services.
    3. Software Development.
    4. Mobile App Development.

    ReplyDelete
  9. Welcome to AZLyrics! It's a place where all searches end! We have a large, legal, every day growing universe of punjabi song lyrics where stars of all genres and ages shine.

    ReplyDelete
  10. I was looking at some of your posts on this website and I conceive this web site is really instructive! Keep putting up..
    danish

    ReplyDelete
  11. Hi,
    Thanks for sharing such a useful information,
    If anyone is looking out for a SEO service in Delhi,NCR then there is only one name which you can trust i;e,Nexcuit.com is one of the best SEO company in Delhi. Be Digitally Visible at the Exact Right time you are Being Searched For! We are the pioneer seo company in Delhi and we implement tactical planning and positioning to make stronger your brand presence worldwide and offer the essential impetus to bring up rankings. Call @ 919910326510 and get more insights concerning their charges, Get best SEO Package from the Best SEO Company in Delhi NCR


    seo services in India
    seo service in Delhi
    seo company in Delhi

    ReplyDelete
  12. This comment has been removed by the author.

    ReplyDelete
  13. Thanks for sharing such an awesome Information with us

    I Got Job in my dream company with decent 12 Lacks Per Annum salary, I have learned this world most demanding course out there in the current IT Market from the python training in btm experts who helped me a lot to achieve my dreams comes true. Really worth trying

    ReplyDelete
  14. This is a very informative content, I appreciate that author has taken time for research and content writing. Thanks
    Website designing company in Delhi

    ReplyDelete
  15. Bharat CSP is nothing but a business contributor and technology services provider to different kind of banks. Online Bank CSP Registration, Official Site for apply online CSP. Start Your Business. Mini Bank & Start Earning from 1st Month Low Investment Easy Process..

    Apply CSP
    CSP registration
    CSP provider
    Top CSP Provider in India
    Apply Online For Bank CSP
    Online Money Transfer

    ReplyDelete
  16. Thank you so much for this useful article. Visit OGEN Infosystem for Web Designing and SEO Services in Delhi, India.
    SEO Service in Delhi

    ReplyDelete
  17. Awesome and Valuable information in this blog. Pure Digital covers a wide range of services from hosting, building your first website to Website integrations and backend systems development.

    Website Design Company in New Zealand
    Website Development Company in New Zealand
    Hosting Company New Zealand
    Website Revamp Company in New Zealand
    Web Hosting New Zealand

    ReplyDelete
  18. A large number of people, particularly the migrant laborers and factory workers do not have a saving account and even not able to open an account due to lack of valid address and ID proof. As a result they face difficulties to save their earnings in a safe place and look out for solution to send money to their families.

    CSP Apply
    CSP Online Application
    Apply for CSP
    Top CSP Provider in India
    Apply Online For Bank CSP
    Online Money Transfer

    ReplyDelete
  19. Very useful information, thanks for sharing with us

    Corporate Training | Best Corporate Training | RAMSOL

    ReplyDelete
  20. The content is very clear.Thanks for posting.I always like your posts.Keep updating
    Best Python Training in BTM Layout

    ReplyDelete
  21. Thank You for Providing Such insightful information. If someone is looking for the Website Designing in Faridabad , connect with jeewangarg.com as we are the leading Website Designing Company.

    Website Designing
    Website Designing Company in Faridabad

    ReplyDelete
  22. We are the largest shot blasting machine manufacturers & Suppliers in India. Get the all types of Shot blasting machines and their equipments @ best prices & unmatched services. Call us @ 9811083740.

    ReplyDelete
  23. Nice Blog, keep it up for more updates about this type of blog.Carolina Classics is the manufacturer of best F-100 Classic Ford Truck Parts | Buy Ford F100 truck parts online at Carolina Classics.

    Classic Ford Truck Parts
    F-100 Ford Truck Parts
    Ford Truck Parts
    Exterior Parts for Ford f-100
    Interior Parts for Ford f-100

    ReplyDelete
  24. TVH is the best video & film Production houses in Delhi, India. We are Creative, positive and productive and create videos like Documentaries, Corporate films, Explainer videos etc. Moreover, we do Event management, Printing and Designing of all IEC, BCC and SBCC Material for all India regions. Being the best video production company in Delhi NCR, we aim to add creativity in each of our service offerings. If you want to make your company/brand stand out, Choose TVH as your partner communication management agency in Delhi NCR.

    ReplyDelete

Post a Comment

Popular posts from this blog

음성 인공지능 스타트업의 기회 분석

일본만화 추천 100선

공유 모빌리티 회사로 합류