（吕毅）PHP做大数据量实时分析

2020-02-27 75浏览

  • 1.
  • 2.PHP؉य़හഝᰁਫ෸‫ړ‬ຉ चԭ෭பහഝጱਫ෸‫ړ‬ຉᔮᕹ
  • 3.‫ྫྷݾ‬ ᱾ਹᗑଘ‫ݣ‬๐‫ۓ‬ຝ຅૵ Ӟ‫ࣁݷ‬᪠Ӥጱᑕଧާ ԯᦇᓒၵ੠෸݇ӨୌᦡࢵٖᒫӞਹPaaSଘ‫ݣ‬SAE҅ ᑏۖၵ੠෸֢ԅ‫ڡ‬๗౮ާӨࢫᴚ಑᭜ԧࢵٖTop3ጱSuperAPPಋ๢ጯଶ҅ ᯿ଶ࣮ፗ෸դ‫فے‬᱾ਹᗑᨮᨱଘ‫ݣ‬۸๐‫ۓ‬ୌᦡ̶
  • 4.‫ྫྷݾ‬ ᱾ਹᗑଘ‫ݣ‬๐‫ۓ‬ຝ຅૵ Ӟ‫ࣁݷ‬᪠Ӥጱᑕଧާ ԯᦇᓒၵ੠෸݇ӨୌᦡࢵٖᒫӞਹPaaSଘ‫ݣ‬SAE҅ ᑏۖၵ੠෸֢ԅ‫ڡ‬๗౮ާӨࢫᴚ಑᭜ԧࢵٖTop3ጱSuperAPPಋ๢ጯଶ҅ ᯿ଶ࣮ፗ෸դ‫فے‬᱾ਹᗑᨮᨱଘ‫ݣ‬۸๐‫ۓ‬ୌᦡ̶
  • 5.᱾ਹᗑଘ‫ݣ‬๐‫ۓ‬ᕟ Pla$orm Services Group Sam Biwing Monika Rambone Skylee Kingho1d ਂ‫ؙ‬ො‫ݻ‬ᨮᨱՈ ᭗አਂ‫ؙ‬๐‫ۓ‬൉‫׀‬ຽ‫ٵ‬ ጱ෈կ඙֢ᔄള‫҅ݗ‬ਠ ᗦّ਻AWS S3‫̶ᦓܐ‬च ԭਂ‫ؙ‬๐‫ۓ‬ԏӤ҅൉‫׀‬ ࢶᇆ॒ቘ̵ᥤ᷇᫨Ꮁ๐ ‫ؙਂ҅ۓ‬ෆ֛ຝ຅ٖ۱ ‫ ތ‬C D N ̵ AT S ᖨ ਂ ᚆ ‫̵ێ‬ग़๢಄හഝ‫ྍݶ‬Ө ԰॓ᚆ‫̶ێ‬ ᦾ‫ۓ‬ଘ‫ݣ‬ᨮᨱՈ ᦾ‫ۓ‬ଘ‫ݣ‬ளഠള‫֟܈ف‬ ਹኪ‫מ‬๐‫׀ۓ‬ଫࠟ҅೮ ํ‫ݩ‬Ꮁᩒრ1kӡ̶ଘ‫ݣ‬ ᭗ᬦٖ᮱᧣ଶ̵‫ݩ‬Ꮁࢧ තӨ॔አ๢‫҅ګ‬ඪ೮‫ل‬ ‫ݪ‬ᕪᕉՈޮ᭗ᦾ30+ӡ ᭗҅ჿ᪃᱾ਹ10+ӡᕪᕉ Ոग़᯿‫ݩ‬Ꮁᵱ࿢҅ଚᓕ ഴ᭗ᦾ̶ හഝො‫ݻ‬ᨮᨱՈ चԭਫ෸‫ړ‬ຉᦇᓒ໛ຝ҅ ຅ୌ෭பਫ෸‫ړ‬ຉ๐‫ۓ‬ ᶎ‫ݻ‬ದ๞᥯ᜋ൉‫׀‬๐‫ۓ‬ ፊഴ̵ᶼᦄᚆ‫҅ێ‬൫ୌ ᕹᦇଘ‫ݣ‬๐‫ۓ‬ᶎ‫ݻ‬Ծߝ̵ ᬩ០᥯ᜋ൉‫׀‬Ծߝහഝ ඪඅ҅ଚአGoᥴ٬හഝ ଚ‫ړݎ‬ຉᚆ‫̶ێ‬ Ꮈ‫ݎ‬ૡᑕ૵ Ꮈ‫ݎ‬ૡᑕ૵ ਫԟૡᑕ૵ ඘ᳩPHP҅ᔜ᭗FE୏ ‫ํٍ҅ݎ‬හഝ‫ړ‬ຉ‫ݪل‬ ૡ֢ᙧว҅໒ୗ۸හഝ ᔜ‫઀ٵ‬ሿ̶ሿ݇‫̵ۓᦾے‬ හഝො‫ݻ‬ૡ֢҅ଚᨮᨱ Services Panelጱ୏‫ݎ‬ᖌ ಷ̶ ඘ᳩC++ٍ҅॓Go୏‫ݎ‬ ᚆ‫ࣁ้҅ێ‬Ӿᑀᴺᨮᨱ ᗑᕶၞᰁ‫ړ‬ຉૡ֢҅඘ ᳩᗑᕶ۱॒ቘӨၞᰁፊ ഴ҅ፓ‫ڹ‬ಭ᫝ԭਂ‫ؙ‬ො ‫ݻ‬Ꮈ‫̶ݎ‬ C/C++च๜‫ۑ‬ಎਫ҅य़ ਍ӣଙ݇‫ے‬ACMᒋ ᩦ҅ᆧఀPHP୏‫҅ݎ‬ഩ ൎӞਧጱFEದᚆ҅ፓ ‫̵ۓᦾے݇ڹ‬හഝො‫ݻ‬ ૡ̶֢
  • 6.᱾ਹᗑଘ‫ݣ‬๐‫ۓ‬ᕟ Pla$orm Services Group Sam Biwing Monika Rambone Skylee Kingho1d ਂ‫ؙ‬ො‫ݻ‬ᨮᨱՈ ᭗አਂ‫ؙ‬๐‫ۓ‬൉‫׀‬ຽ‫ٵ‬ ጱ෈կ඙֢ᔄള‫҅ݗ‬ਠ ᗦّ਻AWS S3‫̶ᦓܐ‬च ԭਂ‫ؙ‬๐‫ۓ‬ԏӤ҅൉‫׀‬ ࢶᇆ॒ቘ̵ᥤ᷇᫨Ꮁ๐ ‫ؙਂ҅ۓ‬ෆ֛ຝ຅ٖ۱ ‫ ތ‬C D N ̵ AT S ᖨ ਂ ᚆ ‫̵ێ‬ग़๢಄හഝ‫ྍݶ‬Ө ԰॓ᚆ‫̶ێ‬ ᦾ‫ۓ‬ଘ‫ݣ‬ᨮᨱՈ ᦾ‫ۓ‬ଘ‫ݣ‬ளഠള‫֟܈ف‬ ਹኪ‫מ‬๐‫׀ۓ‬ଫࠟ҅೮ ํ‫ݩ‬Ꮁᩒრ1kӡ̶ଘ‫ݣ‬ ᭗ᬦٖ᮱᧣ଶ̵‫ݩ‬Ꮁࢧ තӨ॔አ๢‫҅ګ‬ඪ೮‫ل‬ ‫ݪ‬ᕪᕉՈޮ᭗ᦾ30+ӡ ᭗҅ჿ᪃᱾ਹ10+ӡᕪᕉ Ոग़᯿‫ݩ‬Ꮁᵱ࿢҅ଚᓕ ഴ᭗ᦾ̶ හഝො‫ݻ‬ᨮᨱՈ चԭਫ෸‫ړ‬ຉᦇᓒ໛ຝ҅ ຅ୌ෭பਫ෸‫ړ‬ຉ๐‫ۓ‬ ᶎ‫ݻ‬ದ๞᥯ᜋ൉‫׀‬๐‫ۓ‬ ፊഴ̵ᶼᦄᚆ‫҅ێ‬൫ୌ ᕹᦇଘ‫ݣ‬๐‫ۓ‬ᶎ‫ݻ‬Ծߝ̵ ᬩ០᥯ᜋ൉‫׀‬Ծߝහഝ ඪඅ҅ଚአGoᥴ٬හഝ ଚ‫ړݎ‬ຉᚆ‫̶ێ‬ Ꮈ‫ݎ‬ૡᑕ૵ Ꮈ‫ݎ‬ૡᑕ૵ ਫԟૡᑕ૵ ඘ᳩPHP҅ᔜ᭗FE୏ ‫ํٍ҅ݎ‬හഝ‫ړ‬ຉ‫ݪل‬ ૡ֢ᙧว҅໒ୗ۸හഝ ᔜ‫઀ٵ‬ሿ̶ሿ݇‫̵ۓᦾے‬ හഝො‫ݻ‬ૡ֢҅ଚᨮᨱ Services Panelጱ୏‫ݎ‬ᖌ ಷ̶ ඘ᳩC++ٍ҅॓Go୏‫ݎ‬ ᚆ‫ࣁ้҅ێ‬Ӿᑀᴺᨮᨱ ᗑᕶၞᰁ‫ړ‬ຉૡ֢҅඘ ᳩᗑᕶ۱॒ቘӨၞᰁፊ ഴ҅ፓ‫ڹ‬ಭ᫝ԭਂ‫ؙ‬ො ‫ݻ‬Ꮈ‫̶ݎ‬ C/C++च๜‫ۑ‬ಎਫ҅य़ ਍ӣଙ݇‫ے‬ACMᒋ ᩦ҅ᆧఀPHP୏‫҅ݎ‬ഩ ൎӞਧጱFEದᚆ҅ፓ ‫̵ۓᦾے݇ڹ‬හഝො‫ݻ‬ ૡ̶֢
  • 7.
  • 8.ٖ਻൹ᥝ Summary % ໜֺCase PHP $ " PHP‫ړ‬ຉᑕଧ ෆ֛ຝ຅ᦡᦇ ਫ෸හഝතᵞ # ਫ෸‫ړ‬ຉ
  • 9.
  • 10.% ໜֺCase Show Case
  • 11.᱾ਹᗑ ᕹᦇଘ‫ݣ‬ Sta7s7cal service pla$orm
  • 12.᱾ਹᗑ ᕹᦇଘ‫ݣ‬ Sta7s7cal service pla$orm
  • 13.᱾ਹᗑ ෭ப‫ړ‬ຉ๐‫ۓ‬ Real-7me log analyse service
  • 14.
  • 15.$ ෆ֛ຝ຅ᦡᦇ Architecture design
  • 16.ӱ ‫ۓ‬ ӱ‫ۓ‬A ᕚ idc 1 idc 2 idc 3 ӱ‫ۓ‬B idc 1 idc 2 idc 2 ӱ‫ۓ‬C idc 1 idc 2 Ka+a Topic 1 Topic 2 Topic 3 Storm Topology 1 Spout & Bolt Count Monitor PHP PHP-FPM PHP PHP PHP PHP PHP ಸᦄ๢‫ګ‬ DB UI PHP ള‫ݗ‬ හഝᔄ API ඙֢ᔄ API Cache
  • 17.ӱ ‫ۓ‬ ӱ‫ۓ‬A ᕚ idc 1 idc 2 idc 3 ӱ‫ۓ‬B idc 1 idc 2 idc 2 ӱ‫ۓ‬C idc 1 idc 2 Ka+a Topic 1 Topic 2 Topic 3 Storm Topology 1 Spout & Bolt Count Monitor PHP PHP-FPM PHP PHP PHP PHP PHP ಸᦄ๢‫ګ‬ DB $ ෆ֛ຝ຅ᦡᦇ UI PHP ള‫ݗ‬ හഝᔄ API ඙֢ᔄ API Cache
  • 18.
  • 19.ᔮᕹ: CentOS 7 ௔ᚆፘ‫ى‬හഝ Performance dataCPU:2᷋ Intel E5 ٖਂ: 128G ྯᑁ30Kᤈ෭ப CPU idle
  • 20.
  • 21.ᔮᕹ: CentOS 7 ௔ᚆፘ‫ى‬හഝ Performance dataCPU:2᷋ Intel E5 ٖਂ: 128G ྯᑁ30Kᤈ෭ப ᗑᕶIO ‫֖ܔ‬ғKB
  • 22.
  • 23.PHP PHP‫ړ‬ຉᑕଧ php analysis program
  • 24.ӱ ‫ۓ‬ ӱ‫ۓ‬A ᕚ idc 1 idc 2 idc 3 ӱ‫ۓ‬B idc 1 idc 2 idc 2 ӱ‫ۓ‬C idc 1 idc 2 Ka+a Topic 1 Topic 2 Topic 3 Storm Topology 1 Spout & Bolt Count Monitor PHP PHP-FPM PHP PHP PHP PHP PHP ಸᦄ๢‫ګ‬ DB $ ෆ֛ຝ຅ᦡᦇ UI PHP ള‫ݗ‬ හഝᔄ API ඙֢ᔄ API Cache
  • 25.
  • 26.Storm ग़᧍᥺ඪ೮ Storm mul7 language support PHP Shell & ' Storm use STDIN & STDOUT support multi language Python JS
  • 27.ᬳള Storm Ө PHP Connect Storm to PHP STDOUT & PHP STDIN ( Traditional mode PHP & Connect Pool FastCGI PHP PHP-FPM PHP PHP ( Transformed model
  • 28.ᬳള Storm Ө PHP Connect Storm to PHP STDOUT & PHP STDIN ( Traditional mode PHP & Connect Pool FastCGI PHP PHP-FPM PHP PHP ( Transformed model Nginx FastCGI PHP-FPM ( LNMP model PHP PHP PHP
  • 29.
  • 30.PHP 7
  • 31.
  • 32.PHPᥴຉ෭பᑕଧ PHP program for parse log ᑕଧ෫ᇫா PHPᥴຉ෭பጱդᎱᑕଧ෫ܲ‫ݥ‬ᇫ ா҅ૡᕟٖ਻Րԅᥴຉ෭ப෈๜አ Inputහഝጱ໊ḵ փ‫௳מف‬ጱٖ਻‫ض‬؉໒ୗ໊ḵ‫ݸ‬ٚ ؉ᥴຉ᭦ᬋ҅ӱ‫ۓ‬ᕚӨӾᳵሾᜓ਻ ฃԾኞᚍහഝ դᎱ᪃ड़ᔜᓌ ֵአPHPٖᗝ‫ڍ‬හ॒ቘ෈๜හഝ҅ ӧୌᦓ୚‫ف‬໛ຝ҅ᔜᓌդᎱ๶‫ᦤכ‬ ‫ེ॒ܔ‬ቘපሲ̶੝‫ސ‬አPHPಘ઀ PHP ) PHP-FPMᯈᗝ ୌᦓdynamicᓕቘPHPᬰᑕ҅ pm.max_children٬ਧ፳ෆ֛॒ቘ᭛ ଶ҅ଫ‫ڥړش‬አ๢࢏ᚆ‫ێ‬ ᭿‫ع‬ᗑᕶ୏ᲀ ӧᥝࣁPHPᥴຉᑕଧӾ؉ᗑᕶ᧗࿢ ᔄ඙֢҅ই඙֢DB̵Cacheᒵ҅਻ ฃᴥलStormձ‫ۓ‬ ᕹӞ‫ݗف‬ ᔄ֒ԭwebᶱፓ҅ଫ୮ํᕹӞጱ‫ف‬ ‫෈ݗ‬կ؉‫॒ول‬ቘ᮱‫ٍ֛҅ړ‬෭ப ᥴຉᥢ‫ڞ‬ኧ෭ப୭ંӱ‫ۓ‬ᕚRDᖫٟ
  • 33.
  • 34." ਫ෸හഝතᵞ Real 7me data collec7on
  • 35.KaAa —— ‫૲ړ‬ୗၾ௳ᔮᕹ a high-throughput distributed messaging system source from LinkedIn හഝ‫ړ‬ຉᶱፓӾ Producer ጱḒᭌѺ KaQaრԭLinkedIn҅ሿંApache୏რᶱፓ̶ਙࣁय़ᰁහഝრӨ‫ݱ‬ᔄၾᩇ ᘏᑕଧԏᳵ຅ୌԧᑞࢴ‫ݢ‬ᶌጱဋ᭗໫໴҅ள᭛̵ᅾಘ઀̵೮ԋ۸ᒵᇙࣳᦏ KaQa౮ԅ୮ӥහഝ‫ړ‬ຉᶱፓӾ Producer ጱḒᭌၾ௳ᔮᕹ̶
  • 36.KaAa —— ‫૲ړ‬ୗၾ௳ᔮᕹ a high-throughput distributed messaging system source from LinkedIn හഝ‫ړ‬ຉᶱፓӾ Producer ጱḒᭌѺ KaQaრԭLinkedIn҅ሿંApache୏რᶱፓ̶ਙࣁय़ᰁහഝრӨ‫ݱ‬ᔄၾᩇ ᘏᑕଧԏᳵ຅ୌԧᑞࢴ‫ݢ‬ᶌጱဋ᭗໫໴҅ள᭛̵ᅾಘ઀̵೮ԋ۸ᒵᇙࣳᦏ KaQa౮ԅ୮ӥහഝ‫ړ‬ຉᶱፓӾ Producer ጱḒᭌၾ௳ᔮᕹ̶ ( Model Twomodels:queuing and publish-subscribe. the laRer is more commonly used.
  • 37.KaAa —— ‫૲ړ‬ୗၾ௳ᔮᕹ a high-throughput distributed messaging system source from LinkedIn හഝ‫ړ‬ຉᶱፓӾ Producer ጱḒᭌѺ KaQaრԭLinkedIn҅ሿંApache୏რᶱፓ̶ਙࣁय़ᰁහഝრӨ‫ݱ‬ᔄၾᩇ ᘏᑕଧԏᳵ຅ୌԧᑞࢴ‫ݢ‬ᶌጱဋ᭗໫໴҅ள᭛̵ᅾಘ઀̵೮ԋ۸ᒵᇙࣳᦏ KaQa౮ԅ୮ӥහഝ‫ړ‬ຉᶱፓӾ Producer ጱḒᭌၾ௳ᔮᕹ̶ ( Model Twomodels:queuing and publish-subscribe. the laRer is more commonly used. * Feature Scalability & Durability & Reliability & Performance & Faulttolerant
  • 38.KaAa —— ‫૲ړ‬ୗၾ௳ᔮᕹ a high-throughput distributed messaging system source from LinkedIn හഝ‫ړ‬ຉᶱፓӾ Producer ጱḒᭌѺ KaQaრԭLinkedIn҅ሿંApache୏რᶱፓ̶ਙࣁय़ᰁහഝრӨ‫ݱ‬ᔄၾᩇ ᘏᑕଧԏᳵ຅ୌԧᑞࢴ‫ݢ‬ᶌጱဋ᭗໫໴҅ள᭛̵ᅾಘ઀̵೮ԋ۸ᒵᇙࣳᦏ KaQa౮ԅ୮ӥහഝ‫ړ‬ຉᶱፓӾ Producer ጱḒᭌၾ௳ᔮᕹ̶ ( Model Twomodels:queuing and publish-subscribe. the laRer is more commonly used. * Feature Scalability & Durability & Reliability & Performance & Faulttolerant " high-level abstracSon Each Topic corresponds to one or more log files. Each Topic has one or more parWWons.
  • 39.KaAa —— ‫૲ړ‬ୗၾ௳ᔮᕹ a high-throughput distributed messaging system source from LinkedIn හഝ‫ړ‬ຉᶱፓӾ Producer ጱḒᭌѺ KaQaრԭLinkedIn҅ሿંApache୏რᶱፓ̶ਙࣁय़ᰁහഝრӨ‫ݱ‬ᔄၾᩇ ᘏᑕଧԏᳵ຅ୌԧᑞࢴ‫ݢ‬ᶌጱဋ᭗໫໴҅ள᭛̵ᅾಘ઀̵೮ԋ۸ᒵᇙࣳᦏ KaQa౮ԅ୮ӥහഝ‫ړ‬ຉᶱፓӾ Producer ጱḒᭌၾ௳ᔮᕹ̶ ( Model Twomodels:queuing and publish-subscribe. the laRer is more commonly used. * Feature Scalability & Durability & Reliability & Performance & Faulttolerant " high-level abstracSon Each Topic corresponds to one or more log files. Each Topic has one or more parWWons. + Ecosystem Stream processing systems, Hadoop integraSon, monitoring, and deployment tools.
  • 40.ӱ ‫ۓ‬ ӱ‫ۓ‬A ᕚ idc 1 idc 2 idc 3 ӱ‫ۓ‬B idc 1 idc 2 idc 2 ӱ‫ۓ‬C idc 1 idc 2 Ka+a Topic 1 Topic 2 Topic 3 Storm Topology 1 Spout & Bolt Count Monitor PHP PHP-FPM PHP PHP PHP PHP PHP ಸᦄ๢‫ګ‬ DB $ ෆ֛ຝ຅ᦡᦇ UI PHP ള‫ݗ‬ හഝᔄ API ඙֢ᔄ API Cache
  • 41.
  • 42.KaAa ਫ᪢Ӿጱဳ఺ᶱ should be no7ced when using KaAa , ၾ௳ᇫா KaQaӾጱoffsetֵአᵱࣁ Consumer‫׆‬؉অഴ‫҅ګ‬᭿‫ع‬ၾ௳ᤩ ᯿॔ၾᩇ̶ ಢᰁ‫ݎ‬ᭆ ‫ݢ‬զ᭗ᬦᯈᗝᦡᗝӞਧጱBuffer҅ ಢᰁ॒ቘၾ௳҅‫ݢ‬զ൉‫܋‬ӧᲙጱප ሲ҅֕ဳ఺౼ෙᳯ̶᷌ ‫ݳ‬ቘ‫ڥ‬አܴᖽ KaQaඪ೮gzip̵snappy̵lz4ӣᐿ ܴᖽᓒဩܴ҅ᖽԾኞጱCPU୏ᲀᵱ ᥝ໑ഝఘ‫٭‬ᘍᡤฎ‫ސވ‬አܴᖽ̶ ၾ௳೮ԋ۸ ኞԾᘏ‫ݻ‬KaQaփ᭓ጱၾ௳տᤩKaQa ؉೮ԋ۸ҁ‫ݢ‬ᯈ҂҅ᅎၚᯈᗝ೮ԋ۸ ෸ᳩ҅᭿‫ع‬KaQa៧ࣈහഝᬦय̶़ ၾ௳෸ᳵଧ ྯӻTopicӾጱྯӻParWWonٖጱٖ਻ฎ ୩෸ᳵᶲଧጱ҅‫̶ڊضفض‬ग़ParWWon ጱTopicٖ҅ParWWonᳵ‫ݢ‬ᚆԤଧ̶ Zookeeper KaQaׁᩢZookeeperᬰᤈ᮱ᗟ҅ࣁ‫כ‬ ᵑग़ZKᜓᅩጱ‫ݶ‬෸҅୮ஞࢩZKᜓᅩ ᬦग़੕ᛘහഝ‫ྍݶ‬ጱ෸ᳵ୏ᲀ̶
  • 43.
  • 44." ਫ෸‫ړ‬ຉ Real-7me analysis
  • 45.Storm —— ‫૲ړ‬ୗਫ෸ᦇᓒᔮᕹ a distributed real-7me computa7on system source from TwiHer ຄ᭛॒ቘय़හഝጱਫ෸ᦇᓒᔮᕹѺ ‫׀‬ᘳԭBackTypeጱnathanmarz୏‫ݎ‬ԧStorm෱๗ᇇ๜҅‫ݸ‬ᵋ፳BackType ᤩTwiZerතᨻ҅StormᬩአࣁTwiZerஉग़ਫ෸ᔮᕹӾ҅ଚࣁTwiZer๗ᳵ୏ რ౮ԅApacheᶮᕆᶱፓѺ
  • 46.Storm —— ‫૲ړ‬ୗਫ෸ᦇᓒᔮᕹ a distributed real-7me computa7on system source from TwiHer - Topology of one Spout ຄ᭛॒ቘय़හഝጱਫ෸ᦇᓒᔮᕹѺ ‫׀‬ᘳԭBackTypeጱnathanmarz୏‫ݎ‬ԧStorm෱๗ᇇ๜҅‫ݸ‬ᵋ፳BackType ᤩTwiZerතᨻ҅StormᬩአࣁTwiZerஉग़ਫ෸ᔮᕹӾ҅ଚࣁTwiZer๗ᳵ୏ რ౮ԅApacheᶮᕆᶱፓѺ
  • 47.Storm —— ‫૲ړ‬ୗਫ෸ᦇᓒᔮᕹ a distributed real-7me computa7on system source from TwiHer - Topology of one Spout ຄ᭛॒ቘय़හഝጱਫ෸ᦇᓒᔮᕹѺ ‫׀‬ᘳԭBackTypeጱnathanmarz୏‫ݎ‬ԧStorm෱๗ᇇ๜҅‫ݸ‬ᵋ፳BackType ᤩTwiZerතᨻ҅StormᬩአࣁTwiZerஉग़ਫ෸ᔮᕹӾ҅ଚࣁTwiZer๗ᳵ୏ რ౮ԅApacheᶮᕆᶱፓѺ * Feature Highly scalable Fault-tolerant Guarantees processing Language agnosSc
  • 48.Storm —— ‫૲ړ‬ୗਫ෸ᦇᓒᔮᕹ a distributed real-7me computa7on system source from TwiHer - Topology of one Spout ຄ᭛॒ቘय़හഝጱਫ෸ᦇᓒᔮᕹѺ ‫׀‬ᘳԭBackTypeጱnathanmarz୏‫ݎ‬ԧStorm෱๗ᇇ๜҅‫ݸ‬ᵋ፳BackType ᤩTwiZerතᨻ҅StormᬩአࣁTwiZerஉग़ਫ෸ᔮᕹӾ҅ଚࣁTwiZer๗ᳵ୏ რ౮ԅApacheᶮᕆᶱፓѺ * Feature Highly scalable Fault-tolerant Guarantees processing Language agnosSc
  • 49.ӱ ‫ۓ‬ ӱ‫ۓ‬A ᕚ idc 1 idc 2 idc 3 ӱ‫ۓ‬B idc 1 idc 2 idc 2 ӱ‫ۓ‬C idc 1 idc 2 Ka+a Topic 1 Topic 2 Topic 3 Storm Topology 1 Spout & Bolt Count Monitor PHP PHP-FPM PHP PHP PHP PHP PHP ಸᦄ๢‫ګ‬ DB $ ෆ֛ຝ຅ᦡᦇ UI PHP ള‫ݗ‬ හഝᔄ API ඙֢ᔄ API Cache
  • 50.Storm ᵞᗭᕟ౮ Storm cluster organiza7on structure Storm UI ᇫா‫௳מ‬ ඙֢ Storm Nimbus ‫૲ݎ‬ Topology ձ‫ړۓ‬ᯈ Zookeeper ឴‫ݐ‬ձ‫ۓ‬ ᇫாӤಸ Storm Supervisor - UI‫׎‬ԭັ፡Ө඙֢ / Nimbus؉ԅຑᕘ᧣ଶձ‫ ۓ‬/ ZKᨮᨱ‫ ௳מྍݶ‬/ Supervisorԅਫᴬᬩᤈᦡ॓
  • 51.
  • 52.? Q&A