NOTE: this blog post was written for version 0.8 of the Karma test runner. An updated blog post for the new Karma 0.10 can be found here.

For my current project we are using Maven to build our AngularJS application. Furthermore we use Sonar (recently renamed to SonarCube) to monitor our code standards / best practices and unit test coverage. In this blog post we describe how to integrate the Karma (Testacular) test runner with Maven and how to add your AngularJS (or any JavaScript) application to SonarQube.

Basic integration of the Karma test runner in Maven

When I initially integrated the Karma test runner with Maven I used the Maven AntRun plugin because there was no Karma plugin for Maven at that moment. Luckily integrating Karma into Maven becomes much easier using the official maven-karma-plugin:

 com.kelveden
        maven-karma-plugin
        1.3

        start 
        PhantomJS
          dots
          false
          false

In the section we overrule the following settings from the "karma.conf.js" Karma configuration file:

  1. Instead of using the browsers specified in the Karma configuration file we want to use the "headless" browser "PhantomJS" in order not to spawn any browser windows during our Maven build.
  2. To allow the Karma progress to be properly send its output to a log file we use "dots" for progress reporting instead of the standard percentage that's being updated.
  3. Since colors are also not possible in a log file we also disable the use of colors.
  4. Finally we explicitly turn off "autowatch" because we are executing a "single run" in Karma (which is the default behavior when using the Karma Maven plugin) and want to prevent the following unwanted message: config - autoWatch set to false, because of singleRun

Generating HTML unit test coverage reports

To generate HTML unit test coverage reports while running the Karma test runner we will have to modify the "karma.conf.js" Karma configuration file:

  1. To pre-process the source files to allow test coverage reporting;
    this process of altering the sources is also known as instrumentation.
  2. To contain an additional "coverageReporter" section to specify that we want HTML reporting and the location of the generated reports
preprocessors = {
  // ...
  'js/*.js': 'coverage'         // (1)
};

coverageReporter = {            // (2)
  type : 'html',
  dir : 'target/karma-coverage'
}

NOTE: this blog post uses the (old) pre-0.10 configuration file format of the Karma test runner. However when using version 0.10 all snippets (and sample application) should still be working. In order to generate the unit test coverage you will have to append ",coverage" to the <reporters> in the Maven POM:

 com.kelveden
  maven-karma-plugin
  

  dots,coverage

The pre-processing of JavaScript source files dramatically alters the code, causing the line / column numbers of the altered code to no longer match the original ones. Therefore the stack traces of exceptions that occur while testing will also use those altered line / column numbers. Instead of always pre-processing the sources we could disable it at default by commenting it out:

preprocessors = {
  // ...

  // WARNING: every "coverage" preprocessor line MUST be commented-out
  //          and will instead be "uncommented" while generating the
  //          "target/karma-cobertura.conf.js" file in the Maven POM
//  'js/*.js': 'coverage'
};

Instead we use the Maven AntRun Plugin to generate a additional file "target/karma-coverage-html.conf.js" with those lines uncommented:

 org.apache.maven.plugins
  maven-antrun-plugin
  1.7

  karma-coverage-html.conf.js
      initialize
      run

After running (at least) a mvn initialize a separate "target/karma-coverage-html.conf.js" file is generated to be used for HTML test coverage reporting while using Karma (outside the Maven build) on the command-line or from within Webstorm / IntelliJ IDEA. The generated Karma configuration file differs from the original Karma configuration file in that:

  • the "basePath = '';" line is altered to "basePath = '..';" to let Karma know that all paths are relative to ".." (since the generated "karma-coverage-html.conf.js" is located in "target" whereas all paths in the Karma configuration file are still relative to the directory in which "karma.conf.js" is located)
  • each earlier commented-out " // '...': 'coverage'" line is stripped from its comment (therefore enabling the pre-processing of the sources)

In order to use the generated "target/karma-coverage-html.conf.js" Karma configuration file you will need to add a <configFile> element to the configuration of the Maven Karma Plugin:

 com.kelveden
  maven-karma-plugin
  

  ${basedir}/target/karma-coverage-html.conf.js

NOTE: Later in this blog post we will modify our Maven POM to generate an additional Karma configuration file for LCOV test coverage reporting and configure the <configFile> element of the configuration of the Maven Karma Plugin to use it.

Basic integration with SonarCube (no unit test coverage yet)

For a basic integration of your JavaScript code in SonarCube you will have to specify:

  1. That this particular Maven module is using the JavaScript language (instead of the default Java)
  2. That all sources in lib/** (containing JavaScript libraries) should be ignored.
  3. That SonarCube should reuse existing reports for dynamic analysis
  4. That information about the executed unit tests can found in "target/surefire-reports"
  5. The source directory containing our JavaScript source-code.
  6. The "test" source directory containing our JavaScript (i.e. Jasmine) unit tests.
  7. That the Karma test runner should create a JUnit style XML report (containing information about the executed JavaScript unit tests)
 js                        
    lib/**                
    reuseReports
    
    target/surefire-reports 

  ${basedir}/js           
    ${basedir}/test 

    com.kelveden
        maven-karma-plugin
        

        dots,junit,coverage

Additionally, in order for Karma "junit" reporter to work we will have to specify in the "karma.conf.js" file where the JUnit XML report should be generated:

// ...
junitReporter = {
  outputFile: 'target/surefire-reports/TEST-karma.xml'
};
// ...

NOTE: as it turns out the JavaScript plugin of SonarCube (as of version 1.5) no longer supports importing "unit test execution reports". More information can be found here (see answer of "Ronald") and here.

Integrating unit test coverage with SonarCube

In order to integrate unit test coverage with SonarCube we will need to used LCOV reporting (instead of HTML). For this we need to add an additional <execution> section to our Maven AntRun Plugin configuration to generate an additional file "target/karma-coverage-lcov.conf.js" in which we also modify the Karma configuration to (only) output LCOV test coverage reporting:

 org.apache.maven.plugins
  maven-antrun-plugin
  1.7

  karma-coverage-lcov.conf.js
      initialize
      run

The generated "target/karma-coverage-lcov.conf.js" file differs from the original Karma configuration file in that:

  • the "basePath = '';" line is altered to "basePath = '..';"
  • each earlier commented-out " // '...': 'coverage'" line is stripped from its comment
  • the "type : 'html'," line is altered to "type : 'lcovonly'," to let Karma generate LCOV coverage reporting (for consumption by SonarCube) instead of HTML coverage reporting.

In order to use the generated Karma configuration file you will need to modify the (earlier added) <configFile> element in the configuration of the Maven Karma Plugin:

 com.kelveden
  maven-karma-plugin
  

  ${basedir}/target/karma-coverage-lcov.conf.js

Then we need to tell SonarCube where to find the LCOV unit test coverage report:

 target/karma-coverage/PhantomJS 1.8 (Windows)/lcov.info

NOTE: later in this blog post we present a more future proof to specify the location of the LCOV file that doesn't depend on the browser version in its path (like in this case "1.8" of "PhantomJS" on "Windows"). Configuring <sonar.javascript.lcov.reportPath> isn't enough to make SonarCube properly process the LCOV file. To successfully let SonarCube process the generated LCOV file we will add an additional <execution> section to the configuration of the Maven AntRun plugin that:

  1. Copies the generated "lcov.info" file from the "target/karma-coverage/PhantomJS*/" directory to the "target" directory; this allows us to change the <sonar.javascript.lcov.reportPath> property to target/lcov.info

NOTE: for correct LCOV processing, ensure the latest SonarCube JavaScript plugin (1.4) is installed.

 org.apache.maven.plugins
  maven-antrun-plugin
  1.7

  copy-lcov-file-to-target
      test
      run 

      ${skipTests}

Sample application

To demonstrate the integration of the Karma test runner in Maven and SonarCube I forked AngularJS sample application called "ng-directive-testing" in GitHub. After running a "mvn clean install sonar:sonar" (either from command-line or through Jenkins) you will see the following dashboard in SonarCube: sonar-dashboard

References

shadow-left