controlplanesdesigninginfrastructureforrapiditeration

2020-02-27 59浏览

  • 1.@mohitgupta Designing Infrastructure for Rapid Iteration
  • 2.Help schools get the most out of learning software Single sign-on Gives instant access to any app, on any browser, and any device 3 million students log in daily to the Clever Portal
  • 3.Infrastructure Engineering 1. Improve resiliency 2. Increase engineering happiness 3. Unblock complex features >120K Containers per week >300 Applications ~40 Engineers >500 Deployments per week
  • 4.credits:Datadog, QCon New York
  • 5.credits:xkcd
  • 6.credits:xkcd, 20th Century Fox
  • 7.Stumbling into Containers The Containerization Rush Moar Engineers, Moar Problems Building using Control Planes
  • 8.Stumbling through Containers* * what are these containers you speak of?
  • 9.10,000 ?????? Schools use Clever
  • 10.app app app config
 management Zero-Downtime Rollovers Blue Green Deployments Immutable Deploys with History
  • 11.
  • 12.docker-1 docker-2 docker-3 docker-4 my-local-changes ❯❯❯ docker run -H docker-1.internal.clever.com app
  • 13.docker-1 docker-2 docker-3 docker-4 ?????? Zero-Downtime Rollovers ?????? Blue Green Deployments ?????? Immutable Deploys with History
  • 14.“Postpone evaluation of new tools that might fit better. Modify the current toolchain to move us closer to the ideal setup.”
  • 15.app app app config
 management
  • 16.app app app } app definition
  • 17.run:type:docker app app app config
 management
  • 18.Move fast and make time to think
  • 19.The Containerization Rush * here comes everybody
  • 20.
  • 21.credits:20th Century Fox
  • 22.Twitter used it and they were neighbors Open Source meant we could change things AWS was supported and we could run it
  • 23.The cluster is having issues again! Twitter had a massive Mesos team Nobody was an expert in the Mesos codebase Zookeeper!
  • 24.There was progress in important areas Fast deployments Improved instance utilization UI and API
  • 25.Move fast and test a leap of faith.
  • 26.More engineers, moar problems * onboarding forever!
  • 27."If my code runs in production then I shouldn't have to change it for you to run it locally"
  • 28.“more important to have dev stability than production velocity” “well thought out systems don’t have such problems” “resiliency, stability, safety !!!!”
  • 29.DockerCompose:how to run everything on my laptop? Configuration 
 Updated values for development in my environment Dependencies Current dependencies for walking through a feature Debugging Logs and metrics for testing outside production
  • 30.ARK a tool to supercharge development that can eventually be used in production
  • 31.On-Demand Development Environments eng1-env schema-change-replays ux-change-review
  • 32.
  • 33.ark start —environment my-env my-app
  • 34.
  • 35.
  • 36.engineers are users too.
  • 37.Building Control Planes
  • 38.start with thin wrappers
  • 39.find your own defaults
  • 40.highlight your quirkscredits:xkcd, dreamworks interactive
  • 41.outsource complexity when possible Owning Interactions.
  • 42.invest in training and documentation
  • 43.Changing all the time * how we learnt to stop worrying and embrace new tools
  • 44.
  • 45.Integration with existing tools we used Completely managed by AWS Met all our requirements!
  • 46.Canary Deployments Daemon Containers Cross-cluster Environments ARK Cluster Autoscaling
  • 47.Asynchronous Workflows at Clever ● REST API and Web UI for managing workflows ● More than a million executions a week ● Over 100 asynchronous workers
  • 48.Workflows
  • 49.Built using AWS Batch
  • 50.Built using AWS Step Functions
  • 51.Control planes allow for rapid infrastructure experimentation
  • 52.Questions @mohitgupta github.com/Clever