Hamburg, a graph computing framework on Hadoop

As mentioned ago, I've been forming up the Hamburg project with Hyunsik Choi. Let's see more detail in the diagram of computing method of Hamburg based on BSP model.




Each worker will process the data fragments stored locally. And then, We can do bulk synchronization using collected communication data. The 'Computation' and 'Bulk synchronization' can be performed iteratively, Data for synchronization can be compressed to reduce network usage.

Plainly, It aims to improve the performance of traverse operations in Graph computing. For example, to explores all the neighboring nodes from the root node using Map/Reduce (FYI, Breadth-First Search (BFS) & MapReduce), We need a lot of iterations to get next vertex per-hop time.

If (same condition as before) do BFS using Hamburg, It will cause a lowering the cost of iterations. Let's assume the graph looks like presented below:




The graph was stored in Hbase on distributed system as above. The root is 1. Then, we need only one 'Bulk synchronization' between server2 and server3 with Hamburg. Rests will be calculated on local machine.




Almost graph algorithms are similar with this case.

Updated - 26 August, 2009 : See also, Inference anatomy of the Google Pregel

105 comments:

  1. Server2 needs to tell Server3 that the vertex 6 distance is 2 so that Server3 can assign distance 3 for vertex 7.

    Since Server2 can only see partial view for vertex 2 neighbors, I think Server2 also has to tell Server1 about the distance of vertex 2 (which is 2). Then Server1 decides the final distance of vertex 2 (which is 1).

    Or the other way around: Server1 already decides the distance of vertex 2 is 1 and needs to tell Server2 about it (since from Server2 point of view, the disntance of vertex 2 is 2).

    Eitherway, Server1 and Server2 has to communicate, isn't it?

    ReplyDelete
  2. The start node assumed as a vertext 1. Hence, Server1's processing will be locally done w/o any communicate. There is one communication between Server2 and Server3.

    ReplyDelete
  3. Nice info regarind graph computing framework on Hadoop my sincere thanks for sharing please continue to share this post
    Hadoop Training in Chennai

    ReplyDelete
  4. The share you really give us excitement. Thanks for your sharing. If you feel tired at work or study try to participate in our games to bring the most exciting feeling. Thank you!
    Python Online Training | Learn Python Online

    ReplyDelete
  5. Thanks for sharing this blog, I am reading your post from the beginning, it was so interesting to read. Visit for
    Web Design Company in Delhi

    ReplyDelete
  6. THANKS FOR INFORMATION

    you can search low-cost website with high-quality website functions.
    Today Join Us
    Call: +91 - 8076909847

    website designing company in india

    levantro
    best interior designer in delhi


    livewebindia
    best website designing company in delhi

    seo company delhi

    Best It Service Provider:

    1. Website Designing And Development.
    2. SEO Services.
    3. Software Development.
    4. Mobile App Development.

    ReplyDelete
  7. Hi,
    Thanks for sharing such a useful information,
    If anyone is looking out for a SEO service in Delhi,NCR then there is only one name which you can trust i;e,Nexcuit.com is one of the best SEO company in Delhi. Be Digitally Visible at the Exact Right time you are Being Searched For! We are the pioneer seo company in Delhi and we implement tactical planning and positioning to make stronger your brand presence worldwide and offer the essential impetus to bring up rankings. Call @ 919910326510 and get more insights concerning their charges, Get best SEO Package from the Best SEO Company in Delhi NCR


    seo services in India
    seo service in Delhi
    seo company in Delhi

    ReplyDelete
  8. This comment has been removed by the author.

    ReplyDelete
  9. Bharat CSP is nothing but a business contributor and technology services provider to different kind of banks. Online Bank CSP Registration, Official Site for apply online CSP. Start Your Business. Mini Bank & Start Earning from 1st Month Low Investment Easy Process..

    Apply CSP
    CSP registration
    CSP provider
    Top CSP Provider in India
    Apply Online For Bank CSP
    Online Money Transfer

    ReplyDelete
  10. Thank you so much for this useful article. Visit OGEN Infosystem for Web Designing and SEO Services in Delhi, India.
    SEO Service in Delhi

    ReplyDelete
  11. Very useful information, thanks for sharing with us

    Corporate Training | Best Corporate Training | RAMSOL

    ReplyDelete
  12. TVH is the best video & film Production houses in Delhi, India. We are Creative, positive and productive and create videos like Documentaries, Corporate films, Explainer videos etc. Moreover, we do Event management, Printing and Designing of all IEC, BCC and SBCC Material for all India regions. Being the best video production company in Delhi NCR, we aim to add creativity in each of our service offerings. If you want to make your company/brand stand out, Choose TVH as your partner communication management agency in Delhi NCR.

    ReplyDelete

  13. Wow!!! It was really an Informational Article which provide me with much Insightful Information. I would like to first thank the author for such an Insightful Content. If someone here is looking for a Website Designing Company in India, then look no further than Jeewangarg.com as we are the best website designing company in Delhi working in the arena for the last 8 years.
    Best website designing company in India
    website designing company in Delhi NCR
    Google Partner in India

    ReplyDelete
  14. Hi,
    Its a really useful Blog. Using this information with your given guidelines we can reduce usage of data network. This graph computing framework on hadoop is very easy and useful.

    ReplyDelete
  15. This comment has been removed by the author.

    ReplyDelete
  16. Thanks For Sharing the Articles,just Thanks Is not enough for all Keep Updating The Articles looking towards more

    To Learn Python and get a Consequence with Happy Then visit same
    python training in chennai | python training in annanagar | python training in omr | python training in porur | python training in tambaram | python training in velachery

    ReplyDelete
  17. Thank you for the amazing value you are providing through your blog.

    ReplyDelete
  18. Thanks for any other wonderful post. Where else may just anyone get that type of info in such a perfect means of writing? I’ve a presentation next week, and I am on the look for such information.
    java training in chennai

    java training in velachery

    aws training in chennai

    aws training in velachery

    python training in chennai

    python training in velachery

    selenium training in chennai

    selenium training in velachery



    ReplyDelete
  19. Get instant assignment help service in Australia. We are in this service from last ten years and provide best assistance to all our clients. If you are in Australia and pursuing your graduation or post-graduation from over there, then you can get assistance from our experts. We believe in providing best to our clients to maintain a long relationship with our clients. You can expect from us a genuine work with zero plagiarism.
    Online Assignment Help

    ReplyDelete
  20. Pretty! This was an extremely wonderful post. Thanks for providing this information.
    Website: panel chart

    ReplyDelete
  21. this article is very good. It is very so much valuable content. I hope these Commenting lists will help to my website
    Website :- Electric hot water heater service in Nagpur

    ReplyDelete
  22. Great post, I think blog owners should larn a lot from this web blog its very user friendly.
    Visit here :- Seo Packages In Gwalior

    ReplyDelete
  23. Very excellent post!!! Thank you so much for your great content. Keep posting.....

    Python Training Institute in Pune
    Best Python Classes in Pune

    ReplyDelete
  24. Nice post! Thanks for your great content and I obtain a huge of knowledge from your best post...
    Selenium with C# Training
    Selenium with Python Training
    Selenium with Java Training
    Learn Selenium with C# Training

    ReplyDelete
  25. I am sure this paragraph has touched all the internet visitors, its really really fastidious piece of writing on building up new weblog.
    online electronics store in india

    ReplyDelete
  26. Great information, i was searching of this kind of information, thank you very much for sharing with us.

    Website : Craigslist Posting Service for Car Dealers |

    ReplyDelete
  27. This was something I was looking for, really helpful, and great work is done. Thank you so much for sharing such valuable information.

    Wbsite : Car Auction Software |

    ReplyDelete
  28. https://www.mytectra.com/

    ReplyDelete
  29. Truly incredible blog found to be very impressive due to which the learners who ever go through it will try to explore themselves with the content to develop the skills to an extreme level. Eventually, thanking the blogger to come up with such an phenomenal content. Hope you arrive with the similar content in future as well.
    schΓΆnen muttertag gif

    ReplyDelete
  30. Extraordinary blog! I recently went over your website and have found some interesting content.I am glad I found this amazing website on the web.
    Website

    ReplyDelete
  31. I think that this article is very useful for the audience. It provides useful and quality information which is definitely going to help a lot of people.
    Click Here


    ReplyDelete
  32. Finally, I found good quality and decent post, it is very helpful to me. I am grateful and thankful for all your effort and dedication.
    Best CBSE School In Gorakhpur

    ReplyDelete
  33. This comment has been removed by the author.

    ReplyDelete
  34. If your answer is yes to any one of the above questions, well don’t worry anymore. Because, no matter what kind of security challenge or threat you are facing, we have a bunch of professionals ready to get you out of the way of any harm. Be it your protection, your family’s safety, your bodyguard company
    residence or office security and surveillance, your asset protection, your transportation-related security needs, or even your chauffeur requirements, here’s a team that offers you comprehensive, highly efficient yet affordable solutions.

    ReplyDelete
  35. Good information regarding graph computing framework on Hadoop my sincere. Thanks for sharing please continue to share this post. Kinemaster Gold

    ReplyDelete
  36. The share you really give us excitement. Thanks for your sharing. If you feel tired at work or study try to participate in our games to bring the most exciting feeling. Thank you! Kinemaster Gold

    ReplyDelete
  37. It was a great blog with so much information of the beautiful places to visit.
    Pest control provider in Nagpur

    ReplyDelete
  38. This is very useful software for you and me. No errors were found during the check.
    You can use it. I hope you like it.
    Marketing and Advertising Graphics

    ReplyDelete
  39. It was not first article by this author as I always found him as a talented author. John Wayne Jacket

    ReplyDelete
  40. Nice Article..!! Thanks for Sharing with us.
    For hand therapy exercise please visit squegg.

    ReplyDelete
  41. Thank you for sharing so insightful article. Rozana.in has a far-reaching presence across various cities in India.
    Visit for more info local grocery stores

    ReplyDelete
  42. Hello there, You have done a great job. As we all know how much Bill of Sale being used. This document is usually used for sale and purchase between two parties like Buyer & seller.
    Visit here colorado Bill of Sale

    ReplyDelete
  43. This post is so interactive and informative.keep update more information...
    AWS Training in Anna Nagar
    AWS Training in Chennai

    ReplyDelete
  44. Hey friend, it is very well written article, thank you for the valuable and useful information you provide in this post. Keep up the good work! FYI, Pet Care adda
    Credit card processing, wimpy kid books free
    ,Essay About Trip With Family

    ReplyDelete
  45. This post is so interactive and informative.keep update more information...
    hadoop training in tambaram
    Big data training in chennai

    ReplyDelete
  46. Nice Post, Good content
    Web development Company in Hyderabad- Inovies
    Ecommerce Development Company
    Social Media Marketing Company in Hyderabad


    Inovies is the best web development company in Hyderabad providing all types of services in web development, mobile app development, ios app development, ecommerce website development, web design.
    It also provides digital marketing services like SEO, SEM, SMM etc.


    ReplyDelete
  47. The great website and information shared are also very appreciable. Young John Dutton Jacket

    ReplyDelete
  48. Nice blog post so thanks a lot for sharing this great blog post.. keep more post for sharing.. have a nice day.
    Apply for Super Visa Canada

    ReplyDelete
  49. Thank You for Providing Such insightful information. If someone is looking for the Quickbooks Customer Service in US.

    ReplyDelete
  50. This comment has been removed by the author.

    ReplyDelete
  51. Best AWS Training provided by Vepsun in Bangalore for the last 12 years. Our Trainer has more than 20+ Years
    of IT Experience in teaching Virtualization and Cloud topics.. we are very delighted to say that Vepsun is
    the Top AWS cloud training Provider in Bangalore. We provide the best atmosphere for our students to learn.
    Our Trainers have great experience and are highly skilled in IT Professionals. AWS is an evolving cloud
    computing platform provided by Amazon with a combination of IT services. It includes a mixture of
    infrastructure as service and packaged software as service offerings and also automation. We have trained
    more than 10000 students in AWS cloud and our trainer Sameer has been awarded as the best Citrix and Cloud
    trainer in india.

    ReplyDelete
  52. Visit Quickbooks Customer Service in US and find out QuickBooks is a software that helps you manage your business and track your income and expenses. It also helps you plan for future transactions, which in turn saves you time, money, and effort.

    ReplyDelete
  53. Great post. keep sharing such a worthy information. Azure Data Factory Training in Hyderabad

    ReplyDelete
  54. This is great content for your readers. Thanks for sharing.
    flute for sale

    ReplyDelete
  55. Vedant Computer is Nagpur's 1st Ecommerce website for Refurbished Laptops, Refurbished Desktops, Providing consumers with an almost new-like experience.
    Visit Here::- [url=https://vedabyte.com/]old laptop[/url]

    ReplyDelete
  56. Vedant Computer is Nagpur's 1st Ecommerce website for Refurbished Laptops, Refurbished Desktops, Providing consumers with an almost new-like experience.
    Visit Here:- refurbished desktop

    ReplyDelete
  57. thanks for valuable information
    nice ariticle

    dellbhoomi training

    ReplyDelete
  58. We as an Adventure Travel company aim to show you nice people the most beautiful and pristine places in Karnataka. Preserving the cleanliness & beauty of our beautiful land is our topmost priority.
    kudremukha

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  59. A large portion of individuals have less number of jeans and more funny trending t-shirts. Thus, pants are normal generally speaking. Thus, remember you get the funny trending t-shirts that you wear with the entirety of your jeans. To close, these are a portion of the things that you ought to consider while you are looking for easygoing garments for men. You can go with the funny trending t- shirt assuming you need the best quality and fitting.

    ReplyDelete
  60. Join top-rated React Training in Hyderabad to boost your web development skills. Interactive sessions, real-world projects, and expert instructors. Enroll now.

    ReplyDelete