Hamburg, a graph computing framework on Hadoop

As mentioned ago, I've been forming up the Hamburg project with Hyunsik Choi. Let's see more detail in the diagram of computing method of Hamburg based on BSP model.




Each worker will process the data fragments stored locally. And then, We can do bulk synchronization using collected communication data. The 'Computation' and 'Bulk synchronization' can be performed iteratively, Data for synchronization can be compressed to reduce network usage.

Plainly, It aims to improve the performance of traverse operations in Graph computing. For example, to explores all the neighboring nodes from the root node using Map/Reduce (FYI, Breadth-First Search (BFS) & MapReduce), We need a lot of iterations to get next vertex per-hop time.

If (same condition as before) do BFS using Hamburg, It will cause a lowering the cost of iterations. Let's assume the graph looks like presented below:




The graph was stored in Hbase on distributed system as above. The root is 1. Then, we need only one 'Bulk synchronization' between server2 and server3 with Hamburg. Rests will be calculated on local machine.




Almost graph algorithms are similar with this case.

Updated - 26 August, 2009 : See also, Inference anatomy of the Google Pregel

160 comments:

  1. Server2 needs to tell Server3 that the vertex 6 distance is 2 so that Server3 can assign distance 3 for vertex 7.

    Since Server2 can only see partial view for vertex 2 neighbors, I think Server2 also has to tell Server1 about the distance of vertex 2 (which is 2). Then Server1 decides the final distance of vertex 2 (which is 1).

    Or the other way around: Server1 already decides the distance of vertex 2 is 1 and needs to tell Server2 about it (since from Server2 point of view, the disntance of vertex 2 is 2).

    Eitherway, Server1 and Server2 has to communicate, isn't it?

    ReplyDelete
  2. The start node assumed as a vertext 1. Hence, Server1's processing will be locally done w/o any communicate. There is one communication between Server2 and Server3.

    ReplyDelete
  3. Nice info regarind graph computing framework on Hadoop my sincere thanks for sharing please continue to share this post
    Hadoop Training in Chennai

    ReplyDelete
  4. Hello admin, I have read your blog, it was very nice to read & I am getting useful information’s through your blog. Keep update your blog. AWS Training in Bangalore | Big Data Hadoop Training in Bangalore

    ReplyDelete
  5. The share you really give us excitement. Thanks for your sharing. If you feel tired at work or study try to participate in our games to bring the most exciting feeling. Thank you!
    Python Online Training | Learn Python Online

    ReplyDelete
  6. Great post and informative blog.it was awesome to read, thanks for sharing this great content to my vision.
    Good discussion.
    PMP Training Course in Bangalore
    PMP Training Course in Dammam
    PMP Training Course in Dubai
    PMP Training Course in Jeddah
    PMP Training Course in Riyadh

    ReplyDelete
  7. Thanks for sharing this blog, I am reading your post from the beginning, it was so interesting to read. Visit for
    Web Design Company in Delhi

    ReplyDelete
  8. THANKS FOR INFORMATION

    you can search low-cost website with high-quality website functions.
    Today Join Us
    Call: +91 - 8076909847

    website designing company in india

    levantro
    best interior designer in delhi


    livewebindia
    best website designing company in delhi

    seo company delhi

    Best It Service Provider:

    1. Website Designing And Development.
    2. SEO Services.
    3. Software Development.
    4. Mobile App Development.

    ReplyDelete
  9. Welcome to AZLyrics! It's a place where all searches end! We have a large, legal, every day growing universe of punjabi song lyrics where stars of all genres and ages shine.

    ReplyDelete
  10. I was looking at some of your posts on this website and I conceive this web site is really instructive! Keep putting up..
    danish

    ReplyDelete
  11. Hi,
    Thanks for sharing such a useful information,
    If anyone is looking out for a SEO service in Delhi,NCR then there is only one name which you can trust i;e,Nexcuit.com is one of the best SEO company in Delhi. Be Digitally Visible at the Exact Right time you are Being Searched For! We are the pioneer seo company in Delhi and we implement tactical planning and positioning to make stronger your brand presence worldwide and offer the essential impetus to bring up rankings. Call @ 919910326510 and get more insights concerning their charges, Get best SEO Package from the Best SEO Company in Delhi NCR


    seo services in India
    seo service in Delhi
    seo company in Delhi

    ReplyDelete
  12. This comment has been removed by the author.

    ReplyDelete
  13. Thanks for sharing such an awesome Information with us

    I Got Job in my dream company with decent 12 Lacks Per Annum salary, I have learned this world most demanding course out there in the current IT Market from the python training in btm experts who helped me a lot to achieve my dreams comes true. Really worth trying

    ReplyDelete
  14. This is a very informative content, I appreciate that author has taken time for research and content writing. Thanks
    Website designing company in Delhi

    ReplyDelete
  15. Bharat CSP is nothing but a business contributor and technology services provider to different kind of banks. Online Bank CSP Registration, Official Site for apply online CSP. Start Your Business. Mini Bank & Start Earning from 1st Month Low Investment Easy Process..

    Apply CSP
    CSP registration
    CSP provider
    Top CSP Provider in India
    Apply Online For Bank CSP
    Online Money Transfer

    ReplyDelete
  16. Thank you so much for this useful article. Visit OGEN Infosystem for Web Designing and SEO Services in Delhi, India.
    SEO Service in Delhi

    ReplyDelete
  17. Awesome and Valuable information in this blog. Pure Digital covers a wide range of services from hosting, building your first website to Website integrations and backend systems development.

    Website Design Company in New Zealand
    Website Development Company in New Zealand
    Hosting Company New Zealand
    Website Revamp Company in New Zealand
    Web Hosting New Zealand

    ReplyDelete
  18. A large number of people, particularly the migrant laborers and factory workers do not have a saving account and even not able to open an account due to lack of valid address and ID proof. As a result they face difficulties to save their earnings in a safe place and look out for solution to send money to their families.

    CSP Apply
    CSP Online Application
    Apply for CSP
    Top CSP Provider in India
    Apply Online For Bank CSP
    Online Money Transfer

    ReplyDelete
  19. Very useful information, thanks for sharing with us

    Corporate Training | Best Corporate Training | RAMSOL

    ReplyDelete
  20. The content is very clear.Thanks for posting.I always like your posts.Keep updating
    Best Python Training in BTM Layout

    ReplyDelete
  21. Thank You for Providing Such insightful information. If someone is looking for the Website Designing in Faridabad , connect with jeewangarg.com as we are the leading Website Designing Company.

    Website Designing
    Website Designing Company in Faridabad

    ReplyDelete
  22. We are the largest shot blasting machine manufacturers & Suppliers in India. Get the all types of Shot blasting machines and their equipments @ best prices & unmatched services. Call us @ 9811083740.

    ReplyDelete
  23. Nice Blog, keep it up for more updates about this type of blog.Carolina Classics is the manufacturer of best F-100 Classic Ford Truck Parts | Buy Ford F100 truck parts online at Carolina Classics.

    Classic Ford Truck Parts
    F-100 Ford Truck Parts
    Ford Truck Parts
    Exterior Parts for Ford f-100
    Interior Parts for Ford f-100

    ReplyDelete
  24. TVH is the best video & film Production houses in Delhi, India. We are Creative, positive and productive and create videos like Documentaries, Corporate films, Explainer videos etc. Moreover, we do Event management, Printing and Designing of all IEC, BCC and SBCC Material for all India regions. Being the best video production company in Delhi NCR, we aim to add creativity in each of our service offerings. If you want to make your company/brand stand out, Choose TVH as your partner communication management agency in Delhi NCR.

    ReplyDelete
  25. Thank you for excellent article.You made an article that is interesting.
    AWS Solutions Architect courses in Bangalore with certifications.
    https://onlineidealab.com/aws-training-in-bangalore/

    ReplyDelete
  26. Great post! I really enjoyed reading it. Keep sharing such articles. Looking forward to learn more from you.
    Mobile app development company in mumbai

    ReplyDelete
  27. Nice and very informative post. A step up transformer is a transformer that step ups the voltage from the primary winding to the secondary winding while keeping the power same in both the windings at the rated frequency. If you looking for buying a Step Up Transformer, go through a complete guide to What is Step Up Transformer by Servostar before making a purchase.

    ReplyDelete

  28. Wow!!! It was really an Informational Article which provide me with much Insightful Information. I would like to first thank the author for such an Insightful Content. If someone here is looking for a Website Designing Company in India, then look no further than Jeewangarg.com as we are the best website designing company in Delhi working in the arena for the last 8 years.
    Best website designing company in India
    website designing company in Delhi NCR
    Google Partner in India

    ReplyDelete
  29. Thank You for providing us with such an insightful information through this blog. However, if you are in the search of finding Cab or Taxi Services in Melbourne? Your Search Ends here…
    silver service cabs is your answer for availing best silver taxi service in Melbourne. Reach us for the best Silver Service Cab Melbourne.

    ReplyDelete
  30. This is a great post. Your Blog the very informative i have learned some information about your blog. We are top Mobile App Development | Mobile App Development Company in India | Website Development Company in Delhi | Website Designing Company in Gurgaon.

    ReplyDelete
  31. Hi,
    Its a really useful Blog. Using this information with your given guidelines we can reduce usage of data network. This graph computing framework on hadoop is very easy and useful.

    ReplyDelete
  32. I really appreciate this wonderful post that you have provided for us. I assure this would be beneficial for most of the people. Thanks for sharing the information keep updating, looking forward to more posts. High Quality Product Images

    ReplyDelete
  33. This comment has been removed by the author.

    ReplyDelete
  34. Thanks For Sharing the Articles,just Thanks Is not enough for all Keep Updating The Articles looking towards more

    To Learn Python and get a Consequence with Happy Then visit same
    python training in chennai | python training in annanagar | python training in omr | python training in porur | python training in tambaram | python training in velachery

    ReplyDelete
  35. Pretty article! I found some useful information in your blog....

    so here we provide,

    We provide you with flexible services and complete hybrid network solutions. It can provide your organisation with exceptional data speeds, advanced external security protection, and high-resilience by leveraging the latest SD-WAN and networking technologies to monitor, manage and strengthening your organisation’s existing network devices.

    https://www.quadsel.in/networking/>
    https://twitter.com/quadsel/
    https://www.linkedin.com/company/quadsel-systems-private-limited/
    https://www.facebook.com/quadselsystems/

    #quadsel #network #security #technologies #managedservices #Infrastructure #Networking #OnsiteResources #ServiceDeskSupport #StorageServices #WarrantyAMCServices #datacentersolutions #DataCenterBuild #EWaste #InfraConsolidation #DisasterRecovery #NetworkingServices #ImagingServices #MPS #Consulting #WANOptimisation #enduserservices

    ReplyDelete
  36. Thank you for the amazing value you are providing through your blog.

    ReplyDelete
  37. Best Corporate Video Production Company in Bangalore and top Explainer Video Company in Bangalore , 3d, 2d Animation Video Makers in Chennai.
    Many many thanks for sharing such valuable tips. we really appreciate your nice explanation.

    ReplyDelete
  38. After reading your article I was amazed. I know that you explain it very well. And I hope that other readers will also experience how I feel after reading your article.
    Ethical Hacking Course in Bangalore
    Certified Ethical Hacker Course

    ReplyDelete
  39. Wow! Such an amazing and helpful post this is. I really really love it. I hope that you continue to do your work like this in the future also.
    Ethical Hacking Training in Bangalore
    Ethical Hacking Training

    ReplyDelete
  40. Thumbs up guys your doing a really good job. It is the intent to provide valuable information and best practices, including an understanding of the regulatory process.
    Cyber Security Course in Bangalore

    ReplyDelete
  41. I am impressed by the information that you have on this blog. Thanks for Sharing
    Ethical Hacking in Bangalore
    Certified Ethical Hacker Course

    ReplyDelete
  42. Thanks for any other wonderful post. Where else may just anyone get that type of info in such a perfect means of writing? I’ve a presentation next week, and I am on the look for such information.
    java training in chennai

    java training in velachery

    aws training in chennai

    aws training in velachery

    python training in chennai

    python training in velachery

    selenium training in chennai

    selenium training in velachery



    ReplyDelete
  43. it is really a great and helpful piece of info. I am glad that you shared this helpful information with us. Please keep us informed like this. Thank you for sharing.
    china visa application singapore

    ReplyDelete
  44. Good Post! , it was so good to read and useful to improve my knowledge as an updated one, keep blogging.After seeing your article I want to say that also a well-written article with some very good information which is very useful for the readers....thanks for sharing it and do share more posts likethis. https://www.3ritechnologies.com/course/angular-js-training-institute-in-pune/

    ReplyDelete
  45. I enjoyed over read your blog post. This was actually what i was looking for and i am glad to came here!
    Website: artificial jewellery

    ReplyDelete

  46. Amazing Article,Really useful information to all So, I hope you will share more information to be check and share here.thanks for sharing .
    Website: Vietnam Tour Packages

    ReplyDelete

  47. Tender Gourmet Butchery is the Butcher shop in Australia. With over 30 years collective experience in the industry and over 200 awards, we pride ourselves on stocking and supplying only the highest-grade Australian meats in our award-winning shops.

    ReplyDelete
  48. it is really a great and helpful piece of info. I am glad that you shared this helpful information with us. Please keep us informed like this. Thank you for sharing.
    malaysia visa online

    ReplyDelete
  49. Get instant assignment help service in Australia. We are in this service from last ten years and provide best assistance to all our clients. If you are in Australia and pursuing your graduation or post-graduation from over there, then you can get assistance from our experts. We believe in providing best to our clients to maintain a long relationship with our clients. You can expect from us a genuine work with zero plagiarism.
    Online Assignment Help

    ReplyDelete
  50. Thanks for sharing such a nice thinking, post is pleasant, thats why i have read it completely
    Website: cz jewellery

    ReplyDelete
  51. Pretty! This was an extremely wonderful post. Thanks for providing this information.
    Website: panel chart

    ReplyDelete
  52. this article is very good. It is very so much valuable content. I hope these Commenting lists will help to my website
    Website :- Electric hot water heater service in Nagpur

    ReplyDelete
  53. Great post, I think blog owners should larn a lot from this web blog its very user friendly.
    Visit here :- Seo Packages In Gwalior

    ReplyDelete
  54. Welcome to the Usma ayurvedic clinic in Delhi NCR. We are ?arguably the best Sexologist clinic in allover India. To all sexual related issues get in touch with us at ?https://www.usmaclinic.com or contact with us @? +91- 8750 800 100, ?+91- 8750 800 200. We provieds 100% ayurvedic treatment and free sexual therapies.

    ReplyDelete
  55. Very excellent post!!! Thank you so much for your great content. Keep posting.....

    Python Training Institute in Pune
    Best Python Classes in Pune

    ReplyDelete
  56. it is really a great and helpful piece of info. I am glad that you shared this helpful information with us. Please keep us informed like this. Thank you for sharing.
    Website: Dear Lottery

    ReplyDelete
  57. Nice post! Thanks for your great content and I obtain a huge of knowledge from your best post...
    Selenium with C# Training
    Selenium with Python Training
    Selenium with Java Training
    Learn Selenium with C# Training

    ReplyDelete
  58. Actually I read it yesterday I looked at most of your posts but I had some ideas about it . This article is probably where I got the most useful information for my research and today I wanted to read it again because it is so well written.
    Data Science Course in Bangalore

    ReplyDelete
  59. I am sure this paragraph has touched all the internet visitors, its really really fastidious piece of writing on building up new weblog.
    online electronics store in india

    ReplyDelete
  60. I always like to read a quality content having accurate information regarding the subject and the same thing I found in this post.

    Website Development Company |

    ReplyDelete
  61. Great information, i was searching of this kind of information, thank you very much for sharing with us.

    Website : Craigslist Posting Service for Car Dealers |

    ReplyDelete
  62. This is really amazing website that I have been found on google regarding website Blog Commenting sites. and I would like to thank admin who also given us to post the link on his side.

    Website : Lubbock moving company |

    ReplyDelete
  63. This was something I was looking for, really helpful, and great work is done. Thank you so much for sharing such valuable information.

    Wbsite : Car Auction Software |

    ReplyDelete
  64. It’s really a cool and helpful piece of information. I am glad that you shared this useful information with us. Please keep us up to date like this. Thanks for sharing.

    Website : Best CRM for Small Businesses |

    ReplyDelete
  65. I am really like it very much for the interesting info in this blog that to this website is providing the wonderful info in this blog that to utilize the great technology in this blog.

    Website : Web Development Company in Gwalior |

    ReplyDelete
  66. You rocked in this post; we are also work in the same field. Being one of the most trusted mobile app development company we are looking forward to serving clients with superb IT solutions. Also visit: top mobile app development company

    ReplyDelete
  67. https://www.mytectra.com/

    ReplyDelete
  68. Truly incredible blog found to be very impressive due to which the learners who ever go through it will try to explore themselves with the content to develop the skills to an extreme level. Eventually, thanking the blogger to come up with such an phenomenal content. Hope you arrive with the similar content in future as well.
    schönen muttertag gif

    ReplyDelete
  69. I was pleasantly delighted to discover this blog and what an excellent post it contains. Thank you for sharing.

    Digital Marketing In Telugu

    ReplyDelete
  70. เราคือผู้นำด้านเกมพนันออนไลน์ Major168 เราคือผู้ให้บริการ คาสิโนออนไลน์ ที่ได้รับรองว่าดีที่สุดในประเทศไทย มีค่ายเกมส์ให้เล่นมากมาย Sagaming, Sexy bacarat, Dreamgame, Ebet, Wm casino, Vivo gaming ไม่ผ่านเอเย่นต์ ระบบปลอดภัยมีทีมงานดูแลตลอด 24ชม.

    SAGAME88 แหล่งรวมเกมส์พนันออนไลน์ คาสิโนสด บาคาร่า กำถั่ว คาสิโนออนไลน์ ไฮโล รูเล็ต รับเครดิตฟรีเล่นได้ทุกเกมส์ โบนัสสมาชิกใหม่เพียบ พร้อมระบบฝากถอนออโต้ 10วิ เรามีทุกค่ายเกมส์ให้คุณเลือกเดิมพัน SA Game Sexy bacarat Dreamgame WM Casino VIVO Gaming Ebet เล่นได้ทุกเกมส์

    Our website ufabet provides betting services in the system of Auto Deposit-Withdrawal. Our members are not only in Thailand. Online football betting UEFA Bet market And this makes a guarantee that Ufabet168 is another reliable football betting website

    ufa through the website UFABET1688 a web gambling online , one that integrated all the bets from online casinos , online casino , online , and also have a game a lot more to be chosen to play such games.

    ReplyDelete
  71. Great article! This is the type of information that are meant to
    be shared across the internet. Thank you for sharing such a useful post. Very Interesting Post! I regularly follow this kind of Blog.
    click here

    ReplyDelete
  72. Wonderful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article resolved my all queries.

    RPA Training in Pune
    Best RPA Training in Pune

    ReplyDelete
  73. This concept is a good way to enhance knowledge. thanks for sharing..

    DevOps Training Institute in Pune

    ReplyDelete
  74. Extraordinary blog! I recently went over your website and have found some interesting content.I am glad I found this amazing website on the web.
    Website

    ReplyDelete
  75. I think that this article is very useful for the audience. It provides useful and quality information which is definitely going to help a lot of people.
    Click Here


    ReplyDelete
  76. Finally, I found good quality and decent post, it is very helpful to me. I am grateful and thankful for all your effort and dedication.
    Best CBSE School In Gorakhpur

    ReplyDelete
  77. I just like the helpful information you supply for your articles.
    I will bookmark your blog and test once more here frequently.
    I’m fairly certain I will learn many new stuff right
    here! Best of luck for the next!
    ufabet168

    ReplyDelete