A DevOps Primer
My goal is to start a series of blogposts explaining how to implement a true DevOps cycle with Plastic SCM and the main CI systems we integrate with — Bamboo, Jenkins and TeamCity.
But, prior to jumping into the specifics of each tool, I'd like to describe better what DevOps is, and how we envision the actual cycle every task must go through.
DevOps is all about breaking the production and development silos by delivering a continuous flow of stable changes to production.
Achieving stability means every task must go through enough checks to ensure nothing is broken before entering the main branch. Thus, every change merged-to-main is a potential new release.
DevOps distilled: breaking silos with frequent stable releases
Before getting into the tech details, I think it is worthwhile to briefly explain what is DevOps. To do that, it is key to understand what motivated DevOps in the first place – the colliding worlds of production and development.
The IT teams in charge of production prioritize stability over change. Breaking things is not good for their business. On the other hand, development is all about frequent changes. Worlds collide.
Traditionally, the definition of done in development was feature finished, instead of delivered to production, which didn't really add any business value. The reason was that stability was king (with a reason), so batches of finished features could be delayed for ages.
DevOps breaks the silos, introduces collaboration and radically modifies the definition of done. Done means delivered to production. Period.
But, frequent releases to production doesn't mean breaking stability. (Production owners wouldn't sign the agreement if that were the case.) Stability is still king.
How is it possible to solve this dichotomy? Frequent releases must be stable. And stability must be built-in, and not as a result of heroics, last minute fixes and fingers crossed. Code review and automated testing are two of the must haves for every team embracing DevOps.
If you need to get deeper into DevOps, I strongly recommend you read "The DevOps Handbook". Easy to read, straight to the point and an eye opener for me.
What "deploy to production" means for your team?
- For web/cloud apps: Certainly, updating the web server(s) with a new version. Web teams are the ones with a more natural view of this deployment vision, because most of the DevOps culture were developed in their ecosystem. "The DevOps Handbook" mentions how lead practitioners these days deploy even thousands of times a day (it means huge teams and microservices).
- For mobile apps: Publishing the app to one or more app stores.
- For traditional on-premise software (both servers and desktop apps, which is our case in Plastic SCM): Publish a new version on your website and make it available for users to download.
- For a game team: It can be a publish action on an app store or a digital distribution service.
- For embedded devices: It means putting your software on an actual device, which happens normally just at the end of the project, although upgrade strategies are more and more familiar than it used to be.
- For in-house business software: It can be very close to web apps (they'll be web apps anyway).
Can your team really afford deploying to production that often?
I will distinguish between 3 groups depending on business restrictions: teams that can deploy continuously to production (if tech permits), teams who can deploy once a day or even once a week, but not several times a day due to business restrictions, and finally teams who can only release once in a while (typically bound to some hardware or physical distribution restrictions).
- Can afford non-stop deployment: For web apps, the world’s largest players (Amazon, Facebook, Twitter and many others) already answered the question with a big yes. So, not many questions here. If you are developing cloud software, release as frequent as you can, after every single task gets integrated if possible. It if is several times a day, perfect, no worries. In-house business software and anyone sticking to this "transparent" release cycle can really stick to this pattern of non-stop deployments.
- Can deploy between once a day to once a week: For mobile apps, including many games and even games deployed to digital distribution services, publishing is an event. It is not like they can be publishing every few hours, specially the first time. Of course, once the software is published, receiving upgrades is something users are used to, but again, several upgrades a day seems to be out of the picture these days. Same applies to on-premise software, where several releases a day can be confusing for customers, but once a week shouldn't be a big deal.
- Can only deploy once: Here comes the case of embedded software and everything hardware related, and games delivered physically to stores or so-called package software in general.
The definition of "done" can be slightly modified for the last 2 groups. See the following example:
Here, after every task gets merged to main, a new "version" is created (tested, built and then labelled) and it is ready to be deployed. But, in the diagram, I only marked BL100 and BL103 in green, which are, in my example, the ones that were really published. As soon as each new version in main has everything needed to be published, it would still match with the concept of "done", because it would just be waiting for someone to hit the publish button (which of course should also be fully automated).
What I mean here is – in our case we don't publish every single new release, but still every single release is stable enough to be published if we have a business need for it.
Games, mobile apps, even embedded-software, can stick to this pattern. Always ready to release, just waiting on someone to hit the "publish" button on any available release.
Hands-on: how to really implement DevOps with Plastic SCM
Here are the key principles we recommend implementing DevOps:
- Everything is a task: Everything you do in your software is a task controlled in an issue tracker or project management system. It might be Jira, Polarion, or your own home-made one.
- A branch for each task: Enter task branches, not to be confused with feature branches. Use a branch to implement every task. Tasks must be short, no longer than 16 hours (according to scrum and agile in general), so your task branches must be really short lived. More about task branches here.
- Code review each branch: This is quite common now with the whole "pull request" terminology popularized by GitHub. We strongly recommend code reviews.
- Explore-test each branch: Probably not everyone needs this, but it is very good to have someone else in the team making sure the task makes sense for the user. This is not a replacement for automated testing, this is just a very short exploratory test. It can take 5 minutes if the new feature or change is short. We call this "validation".
- Test each branch merged with main prior to checkin: Here comes the magic that the CI system in combination with Plastic can do. The task is finished, reviewed and even "validated". Now it has to pass automated tests prior to be merged to main. But, you need to merge it before testing it to make sure the merge doesn't break anything. That's what our CI plugins do: merge the branch to main, run tests, and if everything goes well, checkin the result. If not, the result will be discarded.
- If tests pass – a new version is ready to release. And it can be labelled accordingly.
- The CI system deploys the new version: Automate the last mile. Publishing the new version must be as simple as hitting a button somewhere. Manual actions are not DevOps.
What we'll show in the next blogposts are how to implement this "merge bot" with Bamboo, TeamCity and Jenkins, all of them fully supported by its Plastic plugin.
How we implement DevOps ourselves
Since "eating or own's dog food" is essential for us, we thought it would be very useful to share how we do DevOps (and trunk-based development), with a focus on the importance of automation.
Here is a list of 5 blogposts explaining how we do it. I hope it will help you find your own way:
- How we do trunk-based development with Plastic SCM.
- To deploy versus to release.
- Trunk-based development blends well with task branches.
- How we do trunk-based development: answering frequent questions.
- Plastic SCM development cycle - key practices described.
Just remember, when we say "HAL" in the scenarios, you should replace it with Bamboo, Jenkins, TeamCity, and so on.
Key concepts
Just in case someone got lost, here are the concepts I consider key:
- Definition of "done": This is key in agile. When is a task really "done"? The team has to choose whether it is when the task is code complete, reviewed, tested, merged or delivered to production. These days, "delivered to production" seems to be the right choice, and the key to DevOps.
- Release vs deploy: I wrote a short blogpost about the difference.
- CI: Just in case it is not clear, I mean Continuous Integration :-)
Conclusion
DevOps is here to stay, and while the tools to implement it might not be new, the combination and last-mile automation is a real game changer.
Plastic SCM is a key part of the DevOps engines for teams in a wide variety of industries, including games, automobile, insurance, health care and others. The task-centric cycle is now boosted to the next level with Plastic and CI cooperating closer than ever before.
0 comentarios: