NOTE: this post updates an earlier blog post written for version 0.8 of the Karma test runner.

For my current project we are using Maven to build our AngularJS application. Furthermore we use Sonar (recently renamed to SonarCube) to monitor our code standards / best practices and unit test coverage. In this blog post we describe how to integrate version 0.10 of the the Karma test runner with Maven and how to add your AngularJS (or any JavaScript) application to SonarQube.

Basic integration of the Karma test runner in Maven

When I initially integrated the Karma test runner with Maven I used the Maven AntRun plugin because there was no Karma plugin for Maven at that moment. Luckily integrating Karma into Maven becomes much easier using the official maven-karma-plugin.

Unfortunately we still need the Maven AntRun Plugin to do a local installation of Karma;
a local installation of Karma 0.10 is preferred since it nowadays depends heavily on plugins. To install it (using npm install) we will first need to create a "package.json" file:

{
  "devDependencies": {
    "karma": "~0.10.8",
    "karma-jasmine": "~0.1.3",
    "karma-chrome-launcher": "~0.1.1",
    "karma-phantomjs-launcher": "~0.1.1",
    "karma-coverage": "~0.1.4",
    "karma-junit-reporter": "~0.2.1"
  }
}

We can now automatically install Karma locally (including its plugins) and then execute it:

 com.kelveden
    maven-karma-plugin
    1.5

    start 
    ${basedir}/node\_modules/.bin/karma
      PhantomJS
      dots
      false
      false 

  org.apache.maven.plugins
    maven-antrun-plugin
    1.7

    npm-install
        initialize
        run

In the section of the "karma-maven-plugin" we overrule the following settings from the "karma.conf.js" Karma configuration file:

  1. To use the locally installed Karma test runner.
  2. Instead of using the browsers specified in the Karma configuration file we want to use the "headless" browser "PhantomJS" in order not to spawn any browser windows during our Maven build.
  3. To allow the Karma progress to be properly send its output to a log file we use "dots" for progress reporting instead of the standard percentage that's being updated.
  4. Since colors are also not possible in a log file we also disable the use of colors.
  5. Finally we explicitly turn off "autowatch" because we are executing a "single run" in Karma (which is the default behavior when using the Karma Maven plugin) and want to prevent the following unwanted message: config - autoWatch set to false, because of singleRun

Generating HTML unit test coverage reports

To generate HTML unit test coverage reports while running the Karma test runner we will have to modify the "karma.conf.js" Karma configuration file:

  1. To pre-process the source files to allow test coverage reporting;
    this process of altering the sources is also known as instrumentation.
  2. To contain an additional "coverageReporter" section to specify that we want HTML reporting and the location of the generated reports
preprocessors: {
  // ..
  , 'js/\*.js': 'coverage'         // (1)
},

coverageReporter: {
  type : 'html',                // (2)
  dir : 'target/karma-coverage'
},

In order to generate the unit test coverage you will have to append ",coverage" to the <reporters> in the Maven POM:

 com.kelveden
  maven-karma-plugin
  

  dots,coverage

The pre-processing of JavaScript source files dramatically alters the code, causing the line / column numbers of the altered code to no longer match the original ones. Therefore the stack traces of exceptions that occur while testing will also use those altered line / column numbers. Instead of always pre-processing the sources we could disable it at default by commenting it out:

    preprocessors: {
      // ...

      // WARNING: every "coverage" preprocessor line MUST be commented-out
      //          and will be automatically be "uncommented" when then
      //          "target/karma-coverage-....conf.js" files are generated
      //          by the Maven POM of this module
//      , 'js/\*.js': 'coverage'
    },

Instead we use the Maven AntRun Plugin to generate a additional file "target/karma-coverage-html.conf.js" with those lines uncommented:

 org.apache.maven.plugins
  maven-antrun-plugin
  1.7

  karma-coverage-html.conf.js
      initialize
      run

After running (at least) a mvn initialize a separate "target/karma-coverage-html.conf.js" file is generated to be used for HTML test coverage reporting while using Karma (outside the Maven build) on the command-line or from within Webstorm 6 / IntelliJ IDEA 12 (using its NodeJS support). The generated Karma configuration file differs from the original Karma configuration file in that:

  • the "basePath: ''," line is altered to "basePath: '..'," to let Karma know that all paths are relative to ".." (since the generated "karma-coverage-html.conf.js" is located in "target" whereas all paths in the Karma configuration file are still relative to the directory in which "karma.conf.js" is located)
  • each earlier commented-out " // '...': 'coverage'" (or " //, '...': 'coverage'") line is stripped from its comment (therefore enabling the pre-processing of the sources)

In order to use the generated "target/karma-coverage-html.conf.js" Karma configuration file you will need to add a <configFile> element to the configuration of the Maven Karma Plugin:

 com.kelveden
  maven-karma-plugin
  

  ${basedir}/target/karma-coverage-html.conf.js

NOTE: Later in this blog post we will modify our Maven POM to generate an additional Karma configuration file for LCOV test coverage reporting and configure the <configFile> element of the configuration of the Maven Karma Plugin to use it.

Basic integration with SonarCube (no unit test coverage yet)

For a basic integration of your JavaScript code in SonarCube you will have to specify:

  1. That this particular Maven module is using the JavaScript language (instead of the default Java)
  2. That all sources in "lib/**" (containing JavaScript libraries) should be ignored.
  3. That SonarCube should reuse existing reports for dynamic analysis
  4. That information about the executed unit tests can found in "target/surefire-reports"
  5. The source directory containing our JavaScript source-code.
  6. The "test" source directory containing our JavaScript (i.e. Jasmine) unit tests.
  7. That the Karma test runner should create a JUnit style XML report (containing information about the executed JavaScript unit tests)
 js                        
    lib/\*\*                
    reuseReports
    
    target/surefire-reports 

  ${basedir}/js           
    ${basedir}/test 

    com.kelveden
        maven-karma-plugin
        

        dots,junit,coverage

Additionally, in order for Karma "junit" reporter to work we will have to specify in the "karma.conf.js" file where the JUnit XML report should be generated:

    // ...
    junitReporter: {
      outputFile: 'target/surefire-reports/TEST-karma.xml'
    },
    // ...

NOTE: as it turns out the JavaScript plugin of SonarCube (as of version 1.5) no longer supports importing "unit test execution reports". More information can be found here (see answer of "Ronald") and here.

Integrating unit test coverage with SonarCube

In order to integrate unit test coverage with SonarCube we will need to used LCOV reporting (instead of HTML). For this we need to add an additional <execution> section to our Maven AntRun Plugin configuration to generate an additional file "target/karma-coverage-lcov.conf.js" in which we also modify the Karma configuration to (only) output LCOV test coverage reporting:

 org.apache.maven.plugins
  maven-antrun-plugin
  1.7

  karma-coverage-lcov.conf.js
      initialize
      run

The generated "target/karma-coverage-lcov.conf.js" file differs from the original Karma configuration file in that:

  • the "basePath: ''," line is altered to "basePath: '..',"
  • the "type : 'html'," line is altered to "type : 'lcovonly'," to let Karma generate LCOV coverage reporting (for consumption by SonarCube) instead of HTML coverage reporting.
  • each earlier commented-out "// '...': 'coverage'" (or "//, '...': 'coverage'") line is stripped from its comment

In order to use the generated Karma configuration file you will need to modify the (earlier added) <configFile> element in the configuration of the Maven Karma Plugin:

 com.kelveden
  maven-karma-plugin
  

  ${basedir}/target/karma-coverage-lcov.conf.js

Then we need to tell SonarCube where to find the LCOV unit test coverage report:

 target/karma-coverage/PhantomJS 1.8 (Windows)/lcov.info

NOTE: later in this blog post we present a more future proof to specify the location of the LCOV file that doesn't depend on the browser version in its path (like in this case "1.8" of "PhantomJS" on "Windows"). Configuring <sonar.javascript.lcov.reportPath> isn't enough to make SonarCube properly process the LCOV file. To successfully let SonarCube process the generated LCOV file we will add an additional <execution> section to the configuration of the Maven AntRun plugin that:

  1. Copies the generated "lcov.info" file from the "target/karma-coverage/PhantomJS*/" directory to the "target" directory; this allows us to change the <sonar.javascript.lcov.reportPath> property to target/lcov.info

NOTE: for correct LCOV processing, ensure the latest SonarCube JavaScript plugin (1.4) is installed.

 org.apache.maven.plugins
  maven-antrun-plugin
  1.7

  copy-lcov-file-to-target
      test
      run 

      ${skipTests}

Coverage support using the JetBrains Karma plugin of IntelliJ IDEA 13

To support coverage measurements while using the JetBrains Karma plugin of IntelliJ IDEA 13
we explicitly need to add a "coverage" reporter to the Karma configuration file.
Since we like the disable coverage reporting in the "karma.conf.js" file we comment it out:

    // will be automatically uncommented when the
    // "target/karma-coverage-idea13.conf.js" file is generated;
    // required to make the coverage support of the
    // Jetbrains Karma plugin (of IntelliJ IDEA 13) working
//    ,reporters: \['coverage'\]

Finally we can need to add the following the Maven AntRun Plugin configuration to generate a separate "target/karma-coverage-idea13.conf.js" file:

 org.apache.maven.plugins
  maven-antrun-plugin
  1.7

  karma-coverage-idea13.conf.js
      initialize
      run

NOTE: for coverage reporting in IntelliJ IDEA make sure that you are using IDEA 13.0.1 or higher.
IntelliJ IDEA 13.0(.0) only supports an buggy version (133.287) of the JetBrains Karma plugin.
At the time of writing a "Check for Update" in IntelliJ still reports that no new IDEA version is available;
therefor you will need to do a manual installation of 13.0.1 using the download from here.

Sample application

To demonstrate the integration of the Karma test runner in Maven and SonarCube I forked AngularJS sample application called "mavenized-ng-directive-testing" in GitHub. After running a "mvn clean install sonar:sonar" (either from command-line or through Jenkins) you will see the following dashboard in SonarCube: sonar-dashboard

References

shadow-left