OMG, my buddy bought some domain, there were existing mail boxes on Google Apps.
Do you use the Google Apps? Then, be careful when expiring your domain. Because, new owner of domain can access your old data including mail box of users, sites, ..., etc.
Summary of the Google Pregel
The paper of Google Pregel has been published. Here's my summary of the Pregel:
And, I was interested in this phrase:
"Assigning vertices to machines to minimize inter-machine communication is a challenge. Partitioning of the input graph based on topology may suffice if the topology corresponds to the message traffic, but it may not."
- Pregel is a scalable and fault-tolerant platform with an API that is sufficiently flexible to express arbitrary graph algorithms.
- Map/Reduce is one of distributed computing infrastructure, and Pregel is another one.
- Why did they make Pregel!?
- Building a custom distributed infrastructure typically requires a substantial implementation effort which must be repeated for each new algorithm or a graph representation.
- M/R framework isn't ideal for graph algorithms because it does not support communications among nodes.
- There is no such system for large scale graph computing.
- It's inspired by BSP (Bulk Synchronouse Parallel).
- User-defined function compute() is as below:
void Compute(MessageIterator* msgs) { // Receive current messages int mindist = IsSource(vertex_id()) ? 0 : INF; for (; !msgs->;Done(); msgs->;Next()) mindist = min(mindist, msgs->;Value()); if (mindist < GetValue()) { *MutableValue() = mindist; OutEdgeIterator iter = GetOutEdgeIterator(); for (; !iter.Done(); iter.Next()) SendMessageTo(iter.Target(), mindist + iter.GetValue()); // Send data to neighbor node } VoteToHalt(); // Superstep synchronization }
- Pregel system also uses the master/worker (slave) model.
- A master maintains worker, recovers faults of workers, and provides Web-UI monitoring tool of job progress.
- A worker processes its task and communicates with the other workers.
- Used for Shortest Path, PageRank, ..., etc.
- See also: BFS with MapReduce, PageRank Using The BSP, and Hama BSP.
And, I was interested in this phrase:
"Assigning vertices to machines to minimize inter-machine communication is a challenge. Partitioning of the input graph based on topology may suffice if the topology corresponds to the message traffic, but it may not."
A distributed caching mechanism to avoid Twitter's API request limit
Recently i made a twitter application which allows to find school friends. Development was simple, but API call limit and Slow speed were problematic. To solve these problems i added a caching layer which gathers&stores API result data from each clients using javascript and server-side scripts, and it is damn fast now!
Subscribe to:
Posts (Atom)
-
The Red : 4천만 MAU를 지탱하는 서비스 설계와 데이터 처리 기술 By 카카오페이지 기술전략이사 윤 당신의 인터넷 서비스는 안정적이고 확장성 있게 설계되어 있습니까? 아파치 하마 프로젝트 의장 활동, GSoC 멘토링 경험과 7개의 기업을 거치...
-
음성 인공지능 분야에서 스타트업이 생각해볼 수 있는 전략은 아마 다음과 같이 3가지 정도가 있을 것이다: 독자적 Vertical 음성 인공지능 Application 구축 기 음성 플랫폼을 활용한 B2B2C 형태의 비지니스 구축 기 음성 플랫폼...
-
개발자 컨퍼런스같은 것도 방문한게 언제인지 까마득합니다. 코로나로 왠지 교류가 많이 없어졌습니다. 패스트캠퍼스로부터 좋은 기회를 얻어 강연을 하나 오픈하였습니다. 제가 강연에서 주로 다룰 내용은, 인터넷 역사 이래 발전해온 서버 사이드 기술들에 대해 ...
-
공유 모빌리티 시장 여러 자료를 살펴보니 2010년부터 공유 모빌리티 시장이 빠르게 발전하기 시작했다. 최근에는 1천만명 이상으로 성장했고 Frost & Sullivan에 따르면 2025년까지 그 이용자 수는 3600만 명으로 연간 성장...
