Running tests using IntelliJ test runner instead of Gradle

If you have a Gradle project and you run the tests from IntelliJ, by default it will use gradle test to run the tests. If you do not like this behaviour you can change it by doing the following:

  • Go to "File → Settings → Build, Execution, Deployment → Build Tools → Gradle"

  • Set "build and run using" and "test using" to "IntelliJ IDEA"

Also, the out directory is not automatically set:

  • In module settings → Project

  • Set the "Project compiler output" field to the value: out

Now you are able to run your Gradle tests using the IntelliJ test runner.

Running GradleRunner tests in IntelliJ

If you are using GradleRunner you might have a bit of code that looks like this:

GradleRunner.create()
  .withProjectDir(myTempDirectory) //Temp directory, could be created by @TempDir for example
  .withDebug(true)
  .withArguments("my-task")
  .forwardOutput()
  .withPluginClasspath()
  .build()

If you run the tests using the Gradle test runner things will run fine. If you however use the IntelliJ test runner you might run into one of the following problems:

  • org.Gradle.testkit.runner.InvalidPluginMetadataException: Test runtime classpath does not contain plugin metadata file 'plugin-under-test-metadata.properties'

  • The code you’re testing might not be the latest version of your code

Both of these issues have the same root cause.

So let’s see what happens here. In order to test the plugin using GradleRunner the plugin you are building needs to be on the classpath of GradleRunner. The plugin is being run in myTempDirectory…​ so this is a problem. GradleRunner fixes this with the .withPluginClasspath() option which adds your current project to the classpath. To figure out what the classpath is, Gradle looks for the {Gradlebuilddir}\pluginUnderTestMetadata\plugin-under-test-metadata.properties file. In this file there is information on where the .jar dependencies are, and also where the compiled classes of your project are. By default it will look for your classes in the Gradle build directory e.g. {Gradlebuilddir}\classes\java\main.

And here lies the problem. The classes in your tests will be those from the Gradle build directory, and not from the IntelliJ out directory. This means that if you change any code, IntelliJ will update the out directory, but the Gradle build directory will only get updated once you do a gradle build

Solution

There is not a great solution for this. Basically there are 3 options:

  • Choose to run tests that use GradleRunner using Gradle instead of using IntelliJ

  • Make sure a gradle build is run everytime IntelliJ compiles your code (very slow)

  • Use this ugly hack (code is in Kotlin, should work for Java too with a few minor tweaks):

private fun runGradle(vararg task: String): BuildResult {
  return GradleRunner.create()
    .withProjectDir(myTempDirectory)
    .withDebug(true)
    .withArguments(task.toList())
    .forwardOutput()
    .withPluginClasspath(getPluginClasspath()) //Create custom plugin path
    .build()
}

/**
 * So this hack is here to make GradleRunner look at the .class files generated from IntelliJ instead
 * To make sure this does not interfere with `gradle test` it is enabled with the -DIntelliJClasspath=true flag
 */
private fun getPluginClasspath(): List<File> {
  //Get the default classpath
  val defaultClasspath = DefaultGradleRunner().withProjectDir(myTempDirectory).withPluginClasspath().pluginClasspath

  //Replace the Gradle classpath with the IntelliJ one
  if (System.getProperty("IntelliJClasspath") != null) {
    return defaultClasspath
      .filterNot { it.absolutePath.contains("classes") }
      .plus(Paths.get("./out/production/classes").toFile())
  }
  return defaultClasspath
}

This only works if "Project compiler output" field has the value out, see top of this blog for instructions. If you want to run the test from IntelliJ you only need to add the -DIntelliJClasspath=true flag to the vm options in the run configuration.

Tested with Gradle 5.6.4.

shadow-left