Episode Details

Back to Episodes
The High-level WTF on "Scheduling"

The High-level WTF on "Scheduling"

Episode 65 Published 10 years, 3 months ago
Description
SPONSOR Show notes

Father’s Day

MesosCon

  • Platform Infrastructure at Twitter: The Past, Present and - Future - Chris Pinkham, VP of Engineering, Twitter
  • Forgot to talk about this, but here are my notes from the MesosCon presentation by Twitter
  • Former Nimbula founder (Oracle acquisition), early AWS founder.
  • Twitter's kinda big deal, maybe you've heard of them. Over 1000 services manage Twitter, over 1,000,000 cores.
  • http://twitter.github.io
  • Heron is a newly open-sourced replacement for Storm. Supporting all of our own code isn't sustainable, need an open source community.
  • The Ellen Degeneres photo tweet from the 2015 Academy Awards knocked a couple of services over. 25% traffic spike, hit 255k/tweets per second. 2016 Academy Awards had 2x the traffic, no failures.
  • 30,000 node Mesos cluster (probably largest). "We don't like being the biggest of anything, we find the edge cases." 130,000,000 containers launched daily.
  • Some of their acquisitions were in public cloud, they don't move them in-house. They're actually pushing new services out to AWS where they can. Vine, TellApart, Crashlytics, MoPub, BlueFin, etc. Ad-serving is mostly in AWS. Users: Time Warner, Twitter (30,000 host deployment), Apple Siri.

What exactly is scheduling?

  • BMC CONTROL-M
  • Coté gets Matt to "checks out" his crudes understanding. (Spoiler: Checks out.)

Serverless, what’s the deal?

Mid-roll

Listen Now