Facebook大数据模块快速部署和实时更新 冯冀 (2)

2020-03-01 107浏览

  • 1.Service Deployment with Real Time and Large Volume Data Yi Feng 冯翼 Performance and Capacity Engineer, Facebook Inc. Facebook Inc. © All Rights Reserved
  • 2.
  • 3.Who am I? Before industry PhD, Computer Science, City University of New York Approximation Algorithm Prove theorems and disprove conjectures MSExchange Diagostics Production Debugging, Diagnostics, and bug fix Microsoft, LinkedIn Programmer – Monitoring and Diagnostic Agent Performance and Scalability Engineer – Infrastructure, Instagram Infra Distribution Service Cluster and Interface Optimization Facebook Inc. © All Rights Reserved
  • 4.Takeaways Make Synergetic Application in your practice Learn the principles and keys of similar practice Entertained and Delight Facebook Inc. © All Rights Reserved
  • 5.Challenge in Operations • Reliability • Agility • Efficiency • Procedure – engineering practice • Coordination – engineer practice • Engineering Effort – Operation Cost Facebook Inc. © All Rights Reserved
  • 6.About Facebook • Social network • Communication • Content Sharing Millions of Servers Tens of EB storage (1EB = 1000PB = 1M TB) • 2.42B Monthly Active Users (SEC Filing 10Q, Q2 2019) 10+ Geographical Regions Globally Facebook Inc. © All Rights Reserved
  • 7.Dedicated Cluster Infra Overview Co-allocated Infra Overview Load balance Desktop FE service Mobile Partner EdgeTransport Feed retrieval and ranking ADs FE 前端 Content Photo / Video Warm storage Replication Table storage （文件 表存 储） OS App Blob storage （二进制 存储块) Search Store Migration BE 后端 Facebook Inc. © All Rights Reserved
  • 8.Deployment Facebook Inc. © All Rights Reserved
  • 9.Deployment Challenge Amount of Servers – tens of thousands Data Reatime-ness Amount of Data in Memory Mega Tera Bytes Completion Time Availability During Deployment Incident Recovery time Facebook Inc. © All Rights Reserved
  • 10.Deployment Specification •Scope:Service clusters => geographical region, servers • Metaspec:current version, target version,steps:2%, 10%, 25%, 50%, 99%, 100% •Availability:concurrent down percentage •SLA:'>SLA: