Recently, in one of my pet projects, I decided to try writing tests using Spock framework. It was a great experience, I enjoyed it a lot. Except for the part of setting it up, which was somewhat annoying.

In addition to Spock, the project used Gradle and Spring Boot 2.3.1 which, unfortunately, by default doesn’t cooperate with Spock very well.

The problem

When I tried to execute my first test (after enabling in IntelliJ groovy framework), as simple as:

import spock.lang.Specification

class TestSpec extends Specification {

    def test() {
        expect:
        2 + 2 == 4
    }
}

The IDE was complaining that > No tests found for given includes: [com.example.demo.TestSpec](filter.includeTestsMatching).

The Gradle build was passing, even when I experimentally added a failing test. Gradle ignored Spock tests and was not complaining that no tests were run during the build… It looks like I’m not the only one bothered by it, so maybe in the future, there will appear a new configuration option in Gradle.

JUnit Vintage

After some quick googling it turned out that the stable version of Spock (1.3) uses JUnit 4 under the hood, but new Spring Boot comes with JUnit 5.

To keep things backwards compatible JUnit team has provided JUnit Vintage engine which allows running JUnit 4 tests on JUnit 5 platform and it looked like it should do the job. By default, it’s excluded from the project initialized by Spring Initializr, so it’s enough to remove it from the exclusions.

We also need to configure Gradle to work with JUnit 5 (it’s enabled by default in projects generated by Spring Initializr):

test {
    useJUnitPlatform()
}

Now, both IntelliJ and Gradle can run the tests. Unfortunately, Gradle doesn’t inform how many tests were run, so I figured out that it worked only by having a failing test in the suite.

I found a nifty snippet to improve it and make Gradle reporting number of executed tests:

test {
    afterSuite { desc, result ->
        if (!desc.parent) { // will match the outermost suite
            println "Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} successes, ${result.failedTestCount} failures, ${result.skippedTestCount} skipped)"
        }
    }
}

and another useful one, to log tests results while they are executed:

test {
    testLogging {
        showStandardStreams = true
        events = ["passed", "skipped", "failed"]
    }
}

The full test runner configuration for that case looked like this:

test {
    useJUnitPlatform()

    testLogging {
        showStandardStreams = true
        events = ["passed", "skipped", "failed"]
    }

    afterSuite { desc, result ->
        if (!desc.parent) { // will match the outermost suite
            println "Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} successes, ${result.failedTestCount} failures, ${result.skippedTestCount} skipped)"
        }
    }
}

Now, everything worked:

This approach is pretty much straightforward and it works for me now, while I’m preparing the blog post. Unfortunately, a few days ago, I wasn’t that successful… Either something was fixed in the meantime or (and this is more probable) I didn’t notice that the tests were executed, because Gradle didn’t report the number of passed tests.

Spock 2.x

I didn’t give up, though, and I gave Spock (or rather Spring Boot, Spock and Gradle combination) one more attempt. After a little bit of googling it turned out that there is a new version of Spock (2.x), which was implemented on top of JUnit 5. It’s still not stable (at the time of writing this it’s in the version M2), but for my purposes, it was more than enough.

A word of warning here, though. Spring Boot 2.3.1 comes with Groovy 2.5.x dependency, which at first I didn’t know about. When I tried to use Spock based on Groovy 3 (testCompile group: 'org.spockframework', name: 'spock-core', version: '2.0-M2-groovy-3.0') it threw org.spockframework.util.IncompatibleGroovyVersionException. I didn’t want to mess up with Spring and changing Groovy versions defined by them, so I used Spock based on Groovy 2.5 (testCompile group: 'org.spockframework', name: 'spock-core', version: '2.0-M2-groovy-2.5') and the exception was gone.

With new Spock everything worked, even with JUnit Vintage left excluded (and with the useJUnitPlatform and the Gradle test logs as described above):

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter'
    testImplementation('org.springframework.boot:spring-boot-starter-test') {
        exclude group: 'org.junit.vintage', module: 'junit-vintage-engine'
    }
    testCompile group: 'org.spockframework', name: 'spock-core', version: '2.0-M2-groovy-2.5'
}

Summary

Spock is pretty cool and I plan to use it in more projects, where I have a chance for that. If you haven’t tried it I suggest to give it a go.

There might be some issues with setting it up, but it’s not something that can’t be solved (and this post should help with that).

And if you need more details please take a look on this very informative post by Marcin Zajączkowski.