Spinnaker Year in Review: 2018

2018 has been a huge year of change for me, and for Spinnaker it’s no different! I’d like to recap a few of the highlights for me as it relates to Spinnaker both internally at Netflix as well as the community at large.

Governance

The most exciting change for me is OSS Governance. We’re not a part of a foundation but governance is an important first step to that possibility, establishing the framework for how Spinnaker will be managed in a more open manner and provides both a sense of security for corporations to adopt and invest, as well as a clear path towards greater roles within the project.

We currently have two SIGs, but as time goes on, more will be created as they’re proposed!

Spinnaker Summit

We hosted our second annual Spinnaker Summit in Seattle! We took a lot of lessons away and we’ll be having it again in 2019. I’m hoping to see everyone again as well as welcome newcomers.

Didn’t make it? All of the talks are on YouTube.

Spinnaker Book

The Netflix and Google teams collaborated to create an O’Reilly handbook on continuous delivery and Spinnaker. You can download it for free or get in contact with the Netflix team in-person for a hardcopy.

Major OSS Contributions

I’m going to miss some major things and I’m sorry, but these stick out to me:

1. Orca SQL Backend. SQL will likely be seeing an ever-increasing role in the greater ecosystem as a more stable out-of-the-box persistence layer than what we currently provide.
2. Amazon ECS Support. Initially contributed by the fine folks at Lookout, Amazon added some new functionality to the provider, including Fargate support.
3. Amazon Lambda Support. This just landed a couple days ago and isn’t ready for prime time, but FaaS support is on the way!
4. Kayenta. The collaboration of Netflix and Google to bring Automated Canary Analysis to the open source world came out.
5. Swabbie. The successor to Janitor Monkey, Swabbie is now a first-class Spinnaker service dedicated to cleaning up resources. It currently only supports AMI cleanup, but is extensible to cleanup any cloud or non-cloud resource.
6. Titus Cloud Provider. Titus was open sourced this year and along with, the Spinnaker Titus cloud provider.
7. Amazon Pubsub Triggers. Pipelines can now be triggered off of SNS/SQS messages.
8. Custom Docker Stage. You can now (as of last week!) create pre-configured stages that execute an arbitrary Docker image and consume outputs into a Pipeline’s context. It’s only been tested on Titus, but should work fine for Kubernetes, ECS, or even be adapted for Lambda.

Netflix Advancements

While this doesn’t really help the OSS community at this point, I do want to call things out because we do a lot of work outside the view of the open source community that may inspire future open source work or potentially be candidates for promotion into upstream.

1. OpenConnect support. OpenConnect is Netflix’s global CDN for video and metadata, this team now uses Spinnaker to orchestrate deployments of new OpenConnect Appliance firmware.
2. Teams Page. An internal replacement for the Projects page, which provides a higher-level view of the world, including a nice view of build provenance and delivery lifecycle. Furthermore, we also now have a cost center view allowing teams to track how much each application’s server and infrastructure is costing Netflix month-over-month and calls out how much is spent on inactive server groups compared to the total.
3. Library support. Applications aren’t the only thing that Spinnaker facilitates delivering at Netflix, library owners can now manage library versions, monitor usage and create deprecation campaign cycles.
4. Data delivery support. Last year we introduced pipeline rollout support for Fast Properties, our key/value dynamic config service. This year we’ve expanded this data delivery capability to “medium” data; delivering MB to GB of data to different systems through custom pipelines and stages.

Growing Netflix Spinnaker Team

Netflix continues to grow and invest in Spinnaker. Shocking, I know, but we’ve grown quite a lot and I want to take the time to welcome our new members from 2018.

• Greg Comstock. Our new designer!
• Alan Quach. UI Engineer.
• Erik Munson. UI Engineer.
• Eric Chiang. Engineering Manager.
• Michael Galloway. Engineering Manager.
• Michael Graff. Backend Engineer.
• Daniel Reynaud. Backend Engineer and Doctor.
• Cheryl Potter. Technical Writer.
• Mark Vu. Backend Engineer. Actually doesn’t start until the new year, but welcome anyway!

We’re like, kinda big now. 3 full teams! Welcome all of you new blood, I’m elated to work with such a stunning group of people.

Armory Series A

Armory met a big milestone this year in successfully raising a Series A. This is really important for the community, as many large companies that are interested in Spinnaker want to have a corporate entity that will offer support and help them establish best practices through the organization. I’m looking forward to seeing more contributions from them and their bringing more companies into the Spinnaker community.

Looking Forward

2019 is going to be a great year for Spinnaker. Internally at Netflix, we’re currently looking to prioritize the following areas of improvement:

1. Reliability and performance. We’re actively working on clouddriver, but we’ll be going through all services and evaluating our architecture to ensure we’re setup for continued success into the future.
2. Declarative delivery. We now have an internal team dedicated to the effort, which is neat-o.
3. Spinnaker as a Platform. We want to make it easier to extend and modify Spinnaker without having to actually make changes to Spinnaker itself.