刘梦馨 - AWS如何在云环境下改造传统数据库
2020-02-27 54浏览
- 1.AWS 是如何改造 MySQL ADD YOUR POWERPOINT TITLE HERE 演讲人：刘梦馨 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT 北京 中国大饭店 2017.07
- 2.Agenda • AWS Aurora background • Log as database • Durability and Availability • Fast recovery • High throughput • Performance metric 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 3.Aurora Background • First announce at Nov, 2014 and GA at Jul, 2015 • Compatible with MySQL 5.6 • 6 replica cross 3 AZ • 99.99% available • < 10s recovery • 5X throughput compare MySQL 5.6 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 4.Durable and Available • 6 replica cross 3 region • Use 6 nodes quorum protocol •Write:4/6 nodes •Read:3/6 nodes • Gossip to fill the gap • Async logs to S3 with 99.999999999% durable 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 5.Durable and Available • Multi tenant environment AZ1 AZ2 AZ3 • Every time some nodes down • If one AZ is un reachable … 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 6.Durable and Available • If one AZ and one instance is down • Write is unavailable, read still work • Use 3 read quorum to rebuild replica • If node get down during rebuilding replica … 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 7.Durable and Available • Split replica to 10GB segment • Only rebuild segment that not meet write quorom • With 10GB network recover in 10 seconds 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 8.Durable and Available • Operation benefit • Can rolling update system and software by just offline a node • Migrate user data by delete data • OverSale and keep performance by migrate data 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 9.Durable and Available • Performance impact • Read/Write need network IO • IO times * 6 • Slowest node will hang the request • How to achieve 5X throughput? 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 10.Log is database • MySQL write multiply times when commit a record • Undo/Redo log, binlog, data pages, double write frm … • Master and slave are in sync mode 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 11.• Think about 6 replica 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 12.Log is database • WAL can regenerate all data record begin • Like checkout from git by commit ids • Modern database use WAL to store data A=10 A+10 A-5 A=15 • Only redo log need to be replicated commit • Periodically checkpoint to reduce log length 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 13.• Transfer redo log and meta changes to replica to update cache 第四届中国数据分析师行业峰会 CHINA DATA ANALYST SUMMIT
- 14.Log is database • Write Operation •Client:make SQL changes and commit • Workerthread:'>thread:
