DevOps with Bamboo and Plastic
UPDATE 2018/03/22: Added section "Update branch attributes".
DevOps is all about breaking silos and making production and development cooperate to deploy a continuous flow of stable changes to production.
In the previous blogpost, DevOps Primer, I explained the basics of what DevOps is and our vision on how to implement it with Plastic SCM.
This blogpost will start where the last one ended and will cover how to implement a full DevOps cycle using both Atlassian's Bamboo and Plastic SCM.
I will delve into the details of how to handle task branches with Plastic and then run tests, merge and deploy using Bamboo with the Plastic Plugin. The entire cycle is based on the Bamboo Gatekeeper feature using Plastic attributes that are set to branches to decide when a branch is ready to be merged.
Task branch lifecycle
Let's first go through the lifecycle of a task in this process. While it was depicted already in the figure above, it is worth going into more details to ensure that we are on the same page.
plan – A task is created in the issue tracker (Jira, others), described, planned, etc.
dev – The developer creates the branch for the task, works on it, makes multiple checkins, marks it as finished with an attribute, typically status = "solved".
rev – The code review. Someone else in the team reviews the code, checks the style is fine, ensures there are no design issues, and even tries to find obvious bugs. The attribute in the branch changes to status = "reviewed".
val – Validation (optional). Someone ensures that the task does what it should. The attribute is set to "validated".
merge – Bamboo monitors Plastic branches and creates a plan for them only when they match some conditions and merges the task branch into the trunk/master/main branch. This is what is known as the Gatekeeper feature in Bamboo. This was the big change introduced in BL1995: the Plastic Bamboo Plugin can filter branches, so not every new branch that shows up in the repo is a candidate to be merged. Bamboo considers the branch only if status = "validated" (or any other condition you set). Then it tries to merge the branch to main, and if it succeeds, prior to checkin, it runs the tests.
build/test – Bamboo builds the code (the result of the merge done in the previous step) and then run the tests if the build succeeds.
checkin - If the tests pass, the checkin happens. This way we can guarantee that every new changeset in main is ready to be released.
label – Every task merged to main passed the tests, so it is a potential release candidate. It is a good practice to label it. Later, as discussed in DevOps Primer the team can decide whether the new version is ready to make public or not.
deploy – In the case where every new build is deployed, Bamboo can take care of it and simply push it to production, publish it on the website, or publish it to a app store, depending on the project.
Some exceptions to consider:
- If the merge can't be fully automated, the branch is rejected. Its status attribute will be modified to reflect the change.
- If the build fails, the branch is rejected. Again the status will reflect the new situation.
- If tests fail, the branch is also rejected and it will go back to the development phase.
Zoom into the branch
The previous topic described the task cycle in big detail. But, let's look at a possible Branch Explorer diagram to better understand the cycle with a real example. I mean, I'm sure the previous explanation was more than enough, but I just want to make it crystal clear, so saving words is not an option :-)
Mary starts working on task 1213 taken from the team's Jira. She creates a branch to start working on the branch. And she names it accordingly, with the convention "task" as prefix and then "number of the Jira issue". She can do this from Plastic by using the create branch from task dialog as explained here.
She works on the branch, makes a few checkins, then she marks the branch status attribute as resolved. (And, she'll close the task in Jira too).
Now Daniel will review the task. He can use the built-in code review in Plastic, or he can even just go and diff it, check if everything is ok, and note any issues in Jira. (This is a very manual process, but it can work too.)
In this case, Daniel decided to reopen the task because he thought a method was longer than needed. Mary works again in the branch, checkins again and submits the branch to review. This time Daniel accepts the task and sets its attribute to reviewed.
As I explained above, we strongly recommend someone looking at the task, from a user's perspective. We call it validation, but it is indeed a short exploratory test. Suppose everything goes fine (otherwise the task would be reopened). Manu, the validator, will set the attribute to validated as follows:
Now Bamboo enters the scene. It is continuously monitoring Plastic SCM to find new branches suitable to be merged and tested. It does that by finding branches where the status attribute is set to "validated". (Of course, both the attribute and the status can be easily configured. I'll cover that later.)
task1213 is detected as "validated" and then a new plan is created for it. Bamboo will try to merge the branch, and if there are no conflicts to be manually resolved, then it will build and run tests.
Please note that the workspace that Bamboo uses underneath to build and test has the merge in "checkout" status. I mean, as I said above, the build and tests happen prior to checkin. This means that if something fails, a new changeset was not created on main yet; it is temporary, and it can be easily uncheckedout and removed.
If the build and tests go fine, Bamboo will checkin the result of the merge into main.
What if the task1213 couldn't be merged automatically? Mary will have to "merge down" (we call it rebase, although it is a different operation than a Git rebase) from main as follows:
Then later, Bamboo will be able to merge it correctly:
But you are using only one branch to release and we use many
Yes, we stick to a single "release branch" and we use main for that. Not long ago, we used to have something like this internally:
But, we simplified the whole thing with more automation and sticking to "trunk-based development". A "release branch" was good for our release manager (or integrator) because it gave him freedom to handle the merges and keep the main stable while the new release was in progress. But all this work was done manually. Now, with full automation of merges, there is no need for extra artifacts.
I mean, yes, in Plastic you can create as many branches as you need, and as complex hierarchies as needed but, it is much more efficient to keep things simple. (You know, like in code, the simpler, the better.)
If you want to read more about trunk-based development and how to implement it with Plastic, check our series on How we do trunk based development.
How to install the Plastic Plugin for Bamboo
This is just an easy and fast task. Follow the steps in this article to install and configure the Bamboo Plastic SCM plugin.
How to configure Bamboo
Start by creating a new plan like the following figure shows:
- I'm using blackmore:8087 but you should be pointing to your real server location.
- Plans require you to enter a branch, although later the plan will be monitoring other branches too. This is because of historical reasons, although in our case it will be the branch used to merge the other branches to.
- This is the new part: a way to specify a filter for branches. We are using status=validated which means it will look for branches with an attribute named status and value "validated". The value can be different based on the specifics of your workflow.
Then, click Configure plan.
The next image shows the key points to configure. It is super simple but I'm trying to explain it as much as I can.
- Go to the Branches tab in the plan configuration.
- Select plan creation when "new branch in the repository is created". Actually, it will happen when a new branch matches the filter set in step 3, but this is the way Bamboo explains it.
- Delete plans early. This is useful if you have limited plans in your Bamboo. Since task branches are meant to be merged quickly, typically their plans can be deleted once they are inactive, the sooner the better.
- Under Merging, enable Branch merging enabled and select Gatekeeper. Do not forget to check the Push on check box, otherwise it won't do the checkin after merge.
Finally, you need to configure a "trigger" so that the new plan launches without manual intervention.
- Go to the Triggers tab.
- Configure a new trigger to launch the plan Bamboo will monitor for new branches every 60 seconds.
With this configuration, Bamboo will start working as a "merge bot" capable of removing the burden of the "integrator". Now, the continuous flow of merged tasks ready to deploy will happen without any manual intervention, in true DevOps fashion.
Update branch attributes
ADDED 2018/03/22: Available since 7.0.16.2047.
Wouldn't it be interesting to update the branch status attribute when it enters testing? Or reflect if the branch was correctly tested or rejected?
This way, when Bamboo starts testing the branch, its attribute will be upated to testing as follows:
If everything goes well and the branch is finally merged, the final status will be merged:
All this, and even more, can be configured from Bamboo as follows:
As you can see, it is possible to finely tune the status updates. As a remark, it is not mandatory to stick to the status attribute, you can use any attribute you want. And it is not mandatory to fill in all the transitions either, if you leave one black, no attribute update will be triggered.
Availability
This new functionality in the Bamboo Plugin became available in BL1995.
Updates to support "updating branch attributes" added later to 7.0.16.2047.
Conclusion
DevOps is all about fully automating tasks that were previously manual, to speed up the cycle, increase the stability and remove small issues due to manual intervention. It is about letting software do what it does best, and allowing team members to focus on the more creative tasks.
In our case, by fully automating the merge cycle, our former "integrator" was able to focus on the greatly speed up tests, and later was able to return to development part time, increasing our development power :-).
0 comentarios: