Concerto, a cross-platform .net profiler

Concerto, a cross-platform .net profiler

Friday, November 12, 2010 Pablo Santos dotnet , mono 4 Comments

I'm going to introduce the basics behind Concerto, the cross-platform profiler we've developed to help us optimizing Plastic SCM.

The problem

The problem we face is simple: we need to optimize our code in Mono/Linux, Mono/Solaris, Mono/Mac and .NET/Windows. The best way to find issues is to run exactly the same test on the same hardware but with different OS (ok, Mac and Solaris are slightly more complex, right?, but for the main ones, Linux/Windows, the same iron will work), and then check the results.

We've been doing that for months but we always missed good comparable profiling data. You know, we do use things like AQTime on Windows and then the Mono Profiler on Linux but the generated data is not easy to compare.

We needed a different tool.

Let's start the music

We had the following idea: let's instrument the code to add some instructions to measure the time spent on each method, this way we can use the same instrumented assemblies to run with .NET and Mono and then we can compare.

Cecil was there to help doing the tough instrumentation part, so I emailed our resident Mono hacker: Dick Porter, and told him the idea. It took him a few hours to come up with a prototype, and has been refining it for the last week or so...

And, since it was all about instrumenting code... Dick named it Concerto.

Welcome Concerto

Here is how the toy works:

$ mono Instrument.exe --help
Usage: Instrument [OPTIONS] assemblies
Instrument one or more assemblies to
 record profiling data.
If only one assembly is specified, 
 instrumented assembly output name and
 data filename can be set.  Otherwise
 defaults are chosen for each assembly.

Options:
  -v, --verbose              Increase verbosity
  -h, -?, --help             Show this message and exit
  -f, --filename=VALUE       The filename where profiling data is written to
                               at runtime
  -o, --out=VALUE            The filename where the instrumented assembly is
                               written
  -p, --private              Include private types
  -c, --class=VALUE          The specific class to instrument (can be given
                               more than once)

Usage is pretty simple:

1) run Instrument.exe on an assembly of your choice
2) copy the output assembly back to your application, along with the generated Concerto-blah.dll
3) run your application
4) Look at the output file with mprof-decoder (or equivalent)

For example:

$ ls -l hello.* Concerto*dll *.mprof
-rwxr-xr-x. 1 dick dick 13312 2010-11-08 14:06 Concerto.dll
-rw-r--r--. 1 dick dick   384 2010-11-04 20:10 hello.cs
-rwxr-xr-x. 1 dick dick  3072 2010-11-05 16:51 hello.exe

$ mono Instrument.exe -vvv hello.exe
Instrumenting hello.exe, creating hello.exe.ins
The helper assembly is Concerto-hello_exe.dll
Data shall be written to concerto-hello_exe.mprof
Instrumenting class hello
Instrumenting method System.Void hello::.ctor()
Instrumenting method System.Int32 hello::DoStuff(System.Int32)
Instrumenting method System.Void hello::DoMoreStuff()
Instrumenting method System.Void hello::Main()
Done.

$ ls -l hello.* Concerto*dll
-rw-r--r--. 1 dick dick 13824 2010-11-12 11:53 Concerto-hello_exe.dll
-rwxr-xr-x. 1 dick dick 13312 2010-11-08 14:06 Concerto.dll
-rw-r--r--. 1 dick dick   384 2010-11-04 20:10 hello.cs
-rwxr-xr-x. 1 dick dick  3072 2010-11-05 16:51 hello.exe
-rw-r--r--. 1 dick dick  3584 2010-11-12 11:53 hello.exe.ins

$ mono hello.exe.ins
Hello, world!
1
1
2

$ ls -l hello.* Concerto*dll *.mprof
-rw-r--r--. 1 dick dick 13824 2010-11-12 11:53 Concerto-hello_exe.dll
-rwxr-xr-x. 1 dick dick 13312 2010-11-08 14:06 Concerto.dll
-rw-r--r--. 1 dick dick   365 2010-11-12 11:54 concerto-hello_exe.mprof
-rw-r--r--. 1 dick dick   384 2010-11-04 20:10 hello.cs
-rwxr-xr-x. 1 dick dick  3072 2010-11-05 16:51 hello.exe
-rw-r--r--. 1 dick dick  3584 2010-11-12 11:53 hello.exe.ins

Now let's inspect the output with the mprof-decoder

$ mono mprof-decoder.exe concerto-hello_exe.mprof

------------------------------------------------
Reporting execution time (on 4 methods)
97.19% (0.007282s) hello.System.Void hello::Main()
 2.48% (0.000186s) hello.System.Int32 hello::DoStuff(System.Int32)
        1 calls from hello.System.Void hello::Main()
        1 calls from hello.System.Void hello::DoMoreStuff()
 0.33% (0.000025s) hello.System.Void hello::DoMoreStuff()
        1 calls from hello.System.Void hello::Main()


------------------------------------------------
Reporting execution time by stack frame
97.19% (0.007282s, 1 calls) hello.System.Void hello::Main()
     2.33% (0.000169s, 1 calls) hello.System.Int32 hello::DoStuff(System.Int32)
     0.34% (0.000025s, 1 calls) hello.System.Void hello::DoMoreStuff()
        66.67% (0.000017s, 1 calls) hello.System.Int32 hello::DoStuff(System.Int32)

Finally an example of picking classes to instrument would look like this:

Get the list of classes with verbosity level 2:

$ mono Instrument.exe -vv plastictcpchannel.dll
Instrumenting plastictcpchannel.dll, creating plastictcpchannel.dll.ins
The helper assembly is Concerto-plastictcpchannel_dll.dll
Data shall be written to concerto-plastictcpchannel_dll.mprof
Instrumenting class Codice.Channels.ClientSinkProvider
Instrumenting class Codice.Channels.ClientSink
Instrumenting class Codice.Channels.PlasticBinaryServerFormatterSink
Instrumenting class Codice.Channels.PlasticBinaryServerFormatterSinkProvider
Instrumenting class Codice.Channels.PlasticTcpChannel
Done.

Pick a couple of classes to instrument:

$ mono Instrument.exe -vv -c
  Codice.Channels.PlasticBinaryServerFormatterSinkProvider -c
  Codice.Channels.ClientSinkProvider plastictcpchannel.dll
Instrumenting plastictcpchannel.dll, creating plastictcpchannel.dll.ins
The helper assembly is Concerto-plastictcpchannel_dll.dll
Data shall be written to concerto-plastictcpchannel_dll.mprof
Instrumenting class Codice.Channels.ClientSinkProvider
Instrumenting class Codice.Channels.PlasticBinaryServerFormatterSinkProvider
Done.

Enjoy!

Pablo Santos

I'm the CTO and Founder at Códice.
I've been leading Plastic SCM since 2005. My passion is helping teams work better through version control.
I had the opportunity to see teams from many different industries at work while I helped them improving their version control practices.
I really enjoy teaching (I've been a University professor for 6+ years) and sharing my experience in talks and articles.
And I love simple code. You can reach me at @psluaces.