Pipelines: breaking the wall between Dev and Ops

Posted by Joep Weijers on October 11, 2017

At TOPdesk our Development department is working closely together with our Operations department. This collaboration started off a bit rough, but through several initiatives this was smoothed out. In this post I’d like to show how we used Deployment Pipelines to break down the wall between Development and Operations.

Three DevOps initiatives

Let’s start with a little background. Our SaaS offering started a long long time ago when someone from Sales came up to Operations and said: “I just sold SaaS, what is it and can we do it?” Operations responded: “Sure, we can host the TOPdesk installation for customers on our own premises. But we will have no fancy SaaS features like multi-tenancy or high availability.”

In this setup, it was really easy for Development to deliver a new version. We just threw our On-Premises monolith over the wall to Ops.

Lego figure depicting Development throwing software over the wall to Lego figure depicting Operations hitting it in the head

We figured we could do better than that, so we started three initiatives to improve the collaboration between Dev and Ops:

– One scrum team removed all hacks that Operations made to install our own software. Nowadays, our On-Premises installation can be put on SaaS as-is. This team sat next to Operations, to make it really easy to walk over and ask: “How do you do this? How should that work?”

– We made sure our On-Premises monolith can be build, released and deployed every day instead of four times a year. Several steps of this process needed to be automated. Not only on the Dev and Ops side, but also in between them.

– Our current project is to Continuously Deploy many multi-tenant microservices. Since microservices sound cool and everybody seems to be doing it, we figured we should try it out ourselves.

Three Chasms

According to Brian Dawson, DevOps evangelist at Cloudbees, there are three chasms you must cross to get to Continuous Deployment. The first one is People and Culture. Delivering every day sounds scary to a lot of people. They will say: “You are going to ship more bugs to customers!”. But those people should realize that you can also get your bug fixes out to customers more often.

The second chasm is Process and Practices. You can no longer do a big regression test if you deliver software multiple times a day.

The third chasm is Tooling and Technology. This chasm is the easiest to cross because it is the easiest to influence.

Around the time TOPdesk was working on Continuous Deployment the new Jenkins Pipelines were introduced. I realized that I should use these pipelines to demonstrate how easy Continuous Deployment can be. This nice, visual overview would show people the benefits of Continuous Deployment. So, by crossing the Tooling chasm I could influence the People and Culture.

I began spreading the knowledge about the Jenkins Pipelines. I demoed what you can do with them and how to build them yourself to both Development and Operations.

Lego figure depicting Development pushing software through a pipeline a through the wall to Lego figure depicting Operations

The challenge was to prevent Pipelines from just becoming an effective way of throwing stuff over the wall to Ops. I had good hopes that making the flow from Dev to Ops more visual with a Pipeline, would show developers that their product is actually going into production.

Pay-off

The first time I saw my promotion effort of Pipelines had paid off, was when one of the Ops guys came to me and asked: “I have put this service infrastructure online in production, but I have no idea how to let Dev teams get their services on there. I heard rumors that you know something about Pipelines?” We hooked up the credentials into the pipeline and in a couple of minutes we had solved his problem. By changing the Tooling, I had changed what People did.

Multiple Lego figures depicting Development pushing software through multiple pipelines through the wall to Lego figure depicting Operations

We built more Pipelines and extended them all the way to production. The Dev teams slowly realized that when they would push a button, something would happen in the production environment. Previously, this was known as ‘the realm of Ops’, but now the Dev teams realized that maybe they should talk to those Ops guys before pushing that button.

Dev and Ops started talking to each other. Nowadays they are working together to pump more and more services through the Pipelines. Although we sometimes still notice the wall between them, I think that in due time the Pipelines will grow so big that this wall will crumble down.

Lego figures depicting Development and Operations happy next to a broken down wall