PSPlasticBisect - a PowerShell module to bisect your repository

PSPlasticBisect - a PowerShell module to bisect your repository

Tuesday, June 02, 2020 Sergio L. best practices 0 Comments

Somebody introduced a bug in the codebase, as a failing test demonstrates. But the test is the symptom, and the cause might be too deep to know exactly where it is.

It would help a lot if you were able to pinpoint the changeset where the bug was introduced, but that's manual labor. You must first switch your workspace to a changeset, then build the necessary assemblies, then pass the specific test, and based on the result, decide which is the next changeset to test. That's an awful lot of steps!

That's where bisect is useful, and that's what I bring you today.

Let's bisect a Plastic SCM repository using PowerShell!

Get PSPlasticBisect now

First thing first, if you want to entirely skip this blogpost and just get the PowerShell module I'm going to talk about, you can. Just go to the PlasticSCM/PSPlasticBisect repository on GitHub, follow the instructions in the README, and that's it.

What is a bisect

When talking VCS, a bisect is finding the changeset where a bug was introduced, using binary search.

You can do so by hand. But as developers, we tend to ask computers to do the heavy lifting. It is both easier and less error prone!

How to use Invoke-PlasticBisect

To use Invoke-PlasticBisect (and in general, to bisect any repository regardless of what VCS you might be using), you only need to know two things:

1. a changeset where there the bug is not present,
2. and a changeset where it is.

We will work under two minor but important assumptions:

• Time only goes forward.

• We want to know the changeset where a bug was introduced, and not where it was fixed.

Under these two assumptions, for a range of N changesets, the changesets in the range [0-M) will be correct, and the changesets in the range [M-N] will be incorrect (for any M / 0 <= M <= N).

And thus, the oldest changeset will be the one where everything worked OK (the correct changeset), whilst the newest one will be the one where something fails (the incorrect changeset).

Take a look at this repository I prepared. In the example, the correct changeset id will be 0, and the incorrect changeset id will be 10:

Because the bisect operation implements a binary search, these are the changesets that will get tested:

We will test each changeset using a ScriptBlock. This ScriptBlock must receive a parameter of type PSCustomObject, and we will set the result of testing the changeset in its Result member. This Result must be a member of the TestResult enum defined in the module ([TestResult]::Correct, [TestResult]::Incorrect or [TestResult]::Fatal).

We do NOT need to worry about setting the workspace before running the test — the bisect implementation takes care of that.

So take this as an example. In each one of the tests, I will run dotnet test and set the result depending on its exit code:

{
    param([PSCustomObject]$ResultCapture)
    cd ".\src\test"
    dotnet test
    if ($LASTEXITCODE -eq 0) {
        $ResultCapture.Result = [TestResult]::Correct;
    } else {
        $ResultCapture.Result = [TestResult]::Incorrect;
    }
}

You have a helper higher-ish order function named Invoke-InDirectory. Also, if you are using the latest PowerShell 7, you can use the ternary operator. So we can nicely wrap it like this:

{
    param([PSCustomObject]$ResultCapture)
    Invoke-InDirectory ".\src\test" {
        dotnet test
        $ResultCapture.Result = ($LASTEXITCODE -eq 0) `
            ? [TestResult]::Correct `
            : [TestResult]::Incorrect;
    }
}

Putting all of the pieces together:

PS> Invoke-PlasticBisect `
    -WorkspaceRoot "C:\Users\sergi\wkspaces\bisect-example" `
    -BranchName 'main' `
    -MininumCsetId 0 `
    -MaximumCsetId 10 `
    -TestScript {
        param([PSCustomObject]$ResultCapture)
        Invoke-InDirectory ".\src\test" {
            dotnet test
            $ResultCapture.Result = ($LASTEXITCODE -eq 0) `
                ? [TestResult]::Correct `
                : [TestResult]::Incorrect;
        }
    }

If you specify the -Verbose flag, you will see information about what's happening behind the scenes!

Caveats of PSPlasticBisect

• The module does not ensure a clean workspace (this is, without private files). If you want a clean workspace before each test run, it is your responsibility to clean it up at the beginning of the TestScript.
• This module does not have any test! Pull requests implementing tests are more than welcome.
• The bisect works in a single branch - it does not follow mergelinks. It would be really nice if it did, though!
  • You start testing branch main.
  • The bisect detects that the changeset where the bug appeared has an incoming mergelink from main/scm25600.
  • The bisect follows the mergelink to the source branch main/scm25600 and starts testing it.
    • It would be even nicer if the bisect was able to follow parent branches too, in case the bug was not introduced in a branch, but in its parent.

Wrapping up

Bisecting a repository consists in making a binary search looking for the changeset where someone introduced a specific change.

I presented you PSPlasticBisect, a PowerShell module that implements a bisect algorithm and some nice functions to test each changeset and determine whether it is correct or incorrect.

You do so specifying the changeset ids for a correct changeset and for an incorrect one. You test each changeset with the code specified in a ScriptBlock, and you set the result through a parameter passed down to said ScriptBlock.

You can find the PSPlasticBisect repository on GitHub, and your pull requests are more than welcome!

As always, if you have any issue, please DO contact us. You can ping us on Twitter at @plasticscm, or you can open an issue in the repository.

Stay safe!

Sergio Luis

After an intense internship at Codice during spring and part of summer 2015, I joined the ranks of Plastic SCM as junior developer.
I already contributed code to the Plastic REST API, the HAL Slack bot that controls our CI system, migrated our internal main server to "new" hardware, coded an Android repo browser and hacked wifi-direct for the upcoming Plastic version.
You can reach me at @_sluis_.