华为技术有限公司资深架构师 马全一 - 《使用 Golang 实现 DevOps Orchestration》

2020-02-27 136浏览

  • 1.ContainerOps – DevOps Orchestration
  • 2.Quanyi Ma DevOps & Open Source Expert Senior Architect & Full Stack DeveloperEmail:maquanyi@huawei.comTwitter:@genednaGithub:https://github.com/genedna
  • 3.Agenda 1. The story of DevOps 2. What’s the DevOps 3. The ContainerOps Concept – Defining/Drawing/Running 4. What’s the DevOps Component 5. The ContainerOps core – DevOps Workflow Engine 6. A user case of ContainerOps
  • 4.DevOps Story •2007 - While consulting on a data center migration for the Belgium government, system administrator Patrick Debois becomes frustrated by conflicts between developers and system admins. He ponders solutions. •Agile Conference 2008 in Toronto - Andrew Clay Shafer’s “birds of a feather” ad hoc session called Agile Infrastructure. The only person who showed up was Patrick Debois. Shafer and Debois started a Google group called “Agile System Administration” •O’Reilly Velocity 2009 Conference - Presentation at Velocity of 10+ Deploys perDay:Dev and Ops Cooperation at Flickr by John Allspaw and Paul Hammond – Debois watched by streaming video, tweeted. •October 2009 - Organized through Twitter. Conversation continued on Twitter and the #DevOps hashtag was born, dropping “Days” for brevity. •2010 Mountain View, CA - DevOpsDays •Mar. 2011 - Gartner’s first notes about DevOps •April 2012 - In an InfoQ video interview, Debois admitted that naming the movement was not as intentional as it mightseem:“I picked ‘DevOpsDays’ as Dev and Ops working together because ‘Agile System Administration’ was too long,” he said. “There never was a grand plan for DevOps as a word.”
  • 5.What’s the DevOps? DevOps is "a portmanteau of 'development' and 'operations'" and is "a software development method that stresses communications, collaboration, integration, automation and measurement of cooperation between software developers and other IT professionals". -From Wikipedia DevOps is an operational philosophy that promotes better communication between development and operations as more elements of operations become programmable.
  • 6.What’s the DevOps ultimate AIM? Goal -> The ultimate is break down barriers between developer, QAs and operators. How -> • Define the operation environment at development stage. • Define the process from development to the production. • Automate everything.
  • 7.Why improve so hard? • Don’t break the original DevOps workflow. • Add DevOps orchestration tool adaptive the workflow. • Improve the process with customize DevOps task. • Add DevOps service like Travis CI. • Everyone is happy!!!
  • 8.ContainerOps - DevOps Orchestration Defining -> Component Drawing -> Workflow Running -> Container Orchestration Open Source @ ->https://github.com/Huawei/containerops
  • 9.Defining Component Container Image For DevOps 1. Encapsulating your DevOps task in a container image. 2. Defining the input & output data. a. Data type is Key/Value b. Input data with environment variables. Reserved environment variable name is [CO_DATA]. c. Output data in the stdout/stderr, the reserved environment variable name is [CO_RESULT]. d. Output data format [COUT] CO_RESULT = true/false . e. Base image is phusion/baseimage . 3. Management the component lifecycle in the ContainerOps system.
  • 10.Why Use DevOps Component? Search Download Environment Run Plugin Jenkins Wiki Wgethttps://...Component Component Repository Copy URL Follow README, prepare virtual machine and initialize the environments Scheduling the VM to run the plugin N/A N/A
  • 11.Why A Component Not A Pod? 1. Adaptive other container orchestration system like Docker Swarm. 2. Only one task in the component to easily maintain and share.
  • 12.Component Namespace Namespace Organization OR User The Library Component maintain by ContainerOps team and community partner together. Library
  • 13.Component Repository UI Component Repository Container Registry [https://github.com/Huawei/dockyard]
  • 14.Component Architecture Antilog Docker Collect log using “docker logs -f -t --details ContainerID“ Reserved environment variable [CO_DATA] for transfer data into container, user could define more variables and we suggest all variable name prefix with [CO_]. CO_DATA CO_XXXX CO_YYYY … DevOps Process STDOUT FORMAT [COUT] CO_RESULT = ... stdout stderr return code [Base Image]https://hub.docker.com/r/phusion/baseimageCO_RESULT CO_AAAA CO_BBBB …
  • 15.Logarithm Architecture Kubernetes Master Node Node Node Node … Dock er Rkt Dock er Rkt Dock er AntiLo g Log s Log s Logarithmd Logarithm Storage Driver TiDB/TiKV Driver Log Process Flow Flow Engine Webhook Notification Database TiDB TiKV
  • 16.Workflow Engine – Stage/Action/Job Start Stage All actions run by sequence. All actions run by parallel. Pause Stage Timeout Active Label Version Environment Variables Resource Trigger End Stage
  • 17.Workflow Engine - Job Container Registry The Other Service Pre Lua Interpreter Execute Lua Scripts Component Endpoint 1. Run with Kubernetes 2. Pull From Registry Kubernetes Cluster Input OR 1. Call third service. 2. Call the callback. Service API Service Post Lua Interpreter Execute Lua Scripts output Label : Value Label : Value Label : Value Label : Value Label : Value Label : Value Label : Value
  • 18.
  • 19.
  • 20.TiDB/TiKV/PB Case 3 DevOps Stage with differentsystems:1. Test case Within Travis CI 2. MergeStage:'>Stage: