Gradle Goodness: Enable Task Based On Offline Command Line Argument

One of the command line options of Gradle is --offline. With this option we run Gradle in offline mode to indicate we are not connected to network resources like the internet. This could be useful for example if we have defined dependencies in our build script that come from a remote repository, but we cannot access the remote repository, and we still want to run the build. Gradle will use the locally cached dependencies, without checking the remote repository. New dependencies, not already cached, cannot be downloaded of course, so in that scenario we still need a network connection.

We can check in our build script if the --offline command line argument is used. We can use this to disable tasks that depend on network resources so the build will not fail. To see if we invoked our build with the --offline option we can access the property gradle.startParameter.offline. The value is true if the command line argument --offline is used and false if the command line argument is not used.

Continue reading

Gradle Goodness: Command Line Options For Custom Tasks

Gradle added an incubation feature to Gradle 4.6 to add command line options for custom tasks. This means we can run a task using Gradle and add command line options to pass information to the task. Without this feature we would have to use project properties passed via the -P or --project-property. The good thing about the new feature is that the Gradle help task displays extra information about the command line options supported by a custom task.

Continue reading

Microservice architecture diagram composition

A low code approach to composing microservice architecture diagrams from per service context diagrams.

Background

On a recent assignment I was one of multiple new engineers joining a start-up transitioning into a scale-up. The past two years had been spent rapidly developing to prove the concept, and while at the outset some diagrams were drawn up, at the time I joined these were outdated, lacking or even misleading. To help get ourselves and other new developers up to speed, Andrew Morgan and I set out to create architecture diagrams for the system as a whole.

Approach

In typical fashion these days, the system consists of microservices communicating with one-another in a mix of direct HTTP invocation and by publishing and subscribing to Kafka topics. In the absence of tracing information or tools to deduce the diagrams, we settled on a simple approach:

  1. Create context diagrams per service
    Using PlantUML we created diagrams for each service, in the root of the project repository. The diagrams contain the topics published/subscribed to, and direct HTTP invocations going out from the service, as well as any data stores and interactions with external services.
  2. Compose overall diagrams
    Using the GitLab API we gather all projects with a particular tag, and for those projects extract the PlantUML diagrams from the repository. These are then combined into a single overall diagram by stripping out the @start/enduml tags and simply appending them one after the other. By using distinct tags we’re able to create diagrams for services in active development, in production or zoom in on a particular aspect such as data processing. New projects are automatically picked up, and warnings are sent out when diagrams are missing.
  3. Expose diagrams as images
    A nightly pipeline creates images for each service individually, as well as a number of composite diagram images. The individual service diagrams are visualized in the GitLab repository README of each project, giving passing developers a quick glance of the context of a single service. The composed service diagrams are visualized in the README of the composition project, for easy reference elsewhere.

Compose individual diagrams

Implementation

The code consists of a Gradle task, a Gradle build file and a GitLab CI file. We place the task in the buildSrc directory, so we can easily use it in our project’s build.gradle file.

Gradle Task

buildSrc build.gradle

Project build.gradle

.gitlab-ci.yml

Strengths

We chose this approach as a simple way to document the overall architecture, in a landscape that’s still undergoing changes and needs to be kept up to date. Having the individual diagrams close to the code ensures the individual diagrams are easily maintained, while the overall diagrams always reflect the latest changes. These qualities are not as easily achieved when the diagram is further removed from the code. The low-tech approach allows anyone to quickly contribute diagrams, with minimal tooling.

Weaknesses

As the overall diagrams are composed from individual diagrams, a common set of guidelines should be enforced for a consistent overall view. Inconsistent use of arrow directionality for instance, can have big consequences for the overall diagram layout. Also, layout of the overall diagram is not stable with regards to previous versions in light of recent changes. Introduction of a new inter-service-dependency for example, can rearrange large parts of the diagram, which can confuse those familiar with previous versions.

Future extensions

GitLab PlantUML integration

Above we use separate files for the diagrams for simplicity, and render the diagrams using the PlantUML jar. However, GitLab can integrate with PlantUML directly to visualize diagrams present in any AsciiDoc and Markdown documents in a repository. This would allow one to make the diagrams part of the README for instance, with only slight changes to the diagram composition.

Other platforms

While the code above uses the GitLab API the same approach can just as easily be applied to different platforms such as GitHub.

Alternative diagrams

As briefly mentioned above, there are alternatives to generate overall system architecture diagrams. One such approach is Structurizer, which inspects the code to generate diagrams at multiple levels. Another, more deductive approach is to use the operational platform and available tracing information to compose diagrams based on runtime service dependencies. These and more are explored to extend and supplement the current approach.

Run one or Exclude one test with Gradle

From time to time you only want to run one test, one test method, one class or one package from the command-line.

Or on the contrary: you want to exclude / ignore one specific test or group of tests during the build cycle.

Excluding tests from the build cycle by the command line usually occurs when the following scenarios meet:

  • A test requires significant amount of resources (time, memory, disk space, etc.)
  • The run needs to be independent from the IDE (to reenact the Continuous Integration / Continuous Delivery pipeline) as some IDEs load test-dependencies on the compile-time class-path.
  • You have no or limited ability to change the code-base

Continue reading

Run one or Exclude one test with Maven

From time to time you only want to run one test, one test method, one class or one package from the command-line.

Or on the contrary: you want to exclude / ignore one specific test or group of tests during the build cycle.

Excluding tests from the build cycle by the command line usually occurs when the following scenarios meet:

  • A test requires significant amount of resources (time, memory, disk space, etc.)
  • The run needs to be independent from the IDE (reenact the Continuous Integration / Continuous Delivery pipeline) as some IDEs load test-dependencies on the compile-time class-path.
  • You have no or limited ability to change the code-base
    Continue reading

Implementing architectural fitness functions using Gradle, JUnit and code-assert

Architectural fitness functions

Inspired by Neal Ford’s presentation at our Change is the Only constant event I started experimenting with architectural fitness functions. An architectural fitness function provides an objective integrity assessment of some architectural characteristic(s).

If you want to take a deeper dive into evolutionary architectures including fitness functions take look at Neals book: Building Evolutionary Architectures: Support Constant Change.

Neal’s slides contained an example of verifying package dependencies from a Unit Test using JDepend.

 

Verifying code modularity

In this blog post we’ll elaborate on that approach and create a Unit Test that verifies that our code complies to the chosen packaging strategies using an alternative to JDepend named code-assert.

Continue reading

Gradle Goodness: Using Incremental Task Action

Gradle has incremental build support to speed up our builds. This means Gradle checks input and output for a task and if something changed the task is executed, otherwise the task is skipped. In previous posts we learned how to add incremental build support to our tasks with annotations and inputs and outputs property of a task. When we have a task that has an output file for an input file, like with transformations, we can have a more efficient task using an incremental task action. With an incremental task action we have extra information on the files that are handled by the task. We can have different actions based on if an input file is out of date or removed. This way we can handle only the input files that have changed or removed with incremental builds, instead of all the input files.

To create an incremental task action we must have a task action method (annotated with @TaskAction) that has a single argument of type IncrementalTaskInputs. The IncrementalTaskInputs class has the method outOfDate and removed. These methods take an action, that can be implemented with a closure, with an instance of InputFileDetails as argument. We can get to the input file via this instance and use that for our task logic. When an input file is out of date, because the file contents has changed or the output file has been removed, the action we defined for the outOfDate method is invoked. If the input file is removed the action for the method removed is invoked.

Continue reading

Gradle Goodness: Change Local Build Cache Directory

Gradle 3.5 introduced the build cache. With the build cache we can reuse task output from builds that can come from different computers. We can also use the build cache feature for our local builds. By default the directory to store the cache is located in the Gradle user home directory on our computer (USER_HOME/.gradle/caches/build-cache-1). We can change the directory for the local cache via settings.gradle of our Gradle project. For example we could configure a directory in our project file structure to be the build cache directory. Then it is easy to clean the cache, because it is a directory not shared by other Gradle projects. With the default directory location in the Gradle user home directory the caches of all Gradle projects we run on our computer are stored in a single directory. And the cache doesn’t shrink and will only grow we might want to have more control of where the cache of a single Gradle project is stored. This way we can easily clean the cache, because all files of a project are stored in the directory for that project.

In the following example settings.gradle file we configure our build cache directory to be the directory build-cache in the root directory of our project where we store our settings.gradle file:

Written with Gradle 3.5.

Original post

Gradle Goodness: Enable Build Cache For All Builds

Gradle 3.5 introduced the build cache. With the build cache we can share task output between builds on different computers. For example the build output from a continuous integration server can be used on a developer’s computer. To use the build cache feature we use the command-line option --build-cache. Instead of using the command-line option --build-cache we can set the Gradle property org.gradle.caching with the value true in the file gradle.properties of our project. To set this property for all our projects we set the property in the gradle.properties file in the Gradle home directory, which is usually at USER_HOME/.gradle/gradle.properties.

In the following example we set the property org.gradle.caching in ~/.gradle/gradle.properties:

If we want to disable the build cache feature set via the global property we can use the command-line option --no-build-cache to disable the build cache for a particular build.

Written with Gradle 3.5.

Original post

Gradle Goodness: Create Shortcut Key To Refresh Gradle Projects In IntellIJ IDEA

We can open a Gradle project in IntelliJ IDEA and get support for Gradle inside IntelliJ. Sometimes we need to refresh the project in IntelliJ IDEA, for example when we add a new dependency or plugin in our Gradle build file. We need to refresh the Gradle project so IntelliJ IDEA can work with the changes. The Gradle tool window has an icon to Refresh all Gradle projects. But this means a mouse action and we want to have a shortcut key so we can leave our hands on the keyboard.

The action Refresh all Gradle projects is actually the action Refresh all external projects. We can add keyboard shortcut key via Preferences | Keymap. We use the search box to search for Refresh all external projects.

We can right click on the found action and select Add Keyboard Shortcut to define a new shortcut key:

Now we simply use the shortcut to refresh our Gradle project when we have made a change in the Gradle build file that IntelliJ IDEA should know about.

Besides have the action in the Gradle tool window we can also add it to the main toolbar. We right click on the main toolbar and select the option Customize Menus and Toolbars…. We can add the action Refresh all external projects here to the toolbar:

Written with Gradle 3.4.1 and IntelliJ IDEA 2016.3.4.

Original blog post