Some choices about technologies

Relying on Spring Boot is now a standard for java applications strongly focused on open source technologies.
But to take advantages of microservices, we cannot not only rely on that because microservices require communication between services but also registry, health-checks, unregistry, discovery, and some other things.
There are really multiple ways to address the microservices management in Java. We have Java-centric solutions such as the Netflix stack (Eureka/Zuul/Ribbon) packaged by Spring Cloud, we have agnostic-language solutions (relying on HTTP) such as Consul and we also have full-features solutions such as Kubernetes. While some of these technologies may also be combined.
Here I have chosen Consul combined to docker-compose.
For production, Consul + Kubernetes would make more sense but for local/dev environment that looks a lightweight and straight approach.
We will also use Spring Cloud to benefit from the consul discovery and the spring cloud loadbalancer (replacement for Ribbon) with a really concise and neat configuration for that.

The use case

We have two microservices : weather-forecast-service and what-to-do-service.
weather-forecast-service provides services that what-to-do-service uses to perform its logical.
We need to run multiple weather-forecast-service instances. But here we don’t look for auto-scalability (that would require to use Docker Swarm or Kubernetes), we want just to be able to run multiple instances of weather-forecast-service and to allow what-to-do-service. to use a client loadbalancer to request its services.

Docker-compose

When I design a set of services run by docker-compose, I like to start with the docker-compose template (and refine it later) because that structure things at a very high level.

version: '3.5'
 
services:
 
  consul:
    image: consul:1.6.3
    command: agent -server -ui -node=server1 -bootstrap-expect=1 -client=0.0.0.0
    ports:
      - "8500:8500"
      - "8600:8600/udp"
    networks:
      - consul-net
 
  weather-forecast-service:
    restart: on-failure
    build:
      context: weather-forecast-service
      dockerfile: docker/DockerFile
    ports:
      - "7000-7005:7000"
    networks:
      - consul-net
 
  what-to-do-service:
    restart: on-failure
    build:
      context: what-to-do-service
      dockerfile: docker/DockerFile
    ports:
      - "7010:7000"
    networks:
      - consul-net
 
networks:
  consul-net:
    driver: bridge

Comments about consul declaration

– we run a consul server agent with bootstrap-expect to 1 to fast up the election of the cluster leader (that would be that agent)
– we specify an client address bound to 0.0.0.0 to means any IP. That allows the docker host or any machine on the network to connect to the agent
– we publish ports for the API/UI(8500) and the DNS(8600) to allow their access from the docker host. For dev that is fine but should not be used in production

Comments about microservices declaration

– while rare, the spring boot applications may start during the initialization phase of the consul agent. And according to my tries, when it happens, the registering task to Consul triggered by the spring boot application may fail in a critical way (no retry after that).
That’s why I specify restart: on-failure for each microservice. While that solves the quoted inconsistency issue, that brings also some drawbacks : it pollutes containers logs and it may shadow other issues since the container restart will set the container state to up and then to be set still to down. So containers state displayed by docker-compose ps may be misleading.
– Similarly to the consul container, we publish microservices to the host to allow us to test the applications outside the containers.
– I use a range for published ports of weather-forecast-service: - "7000-7005:7000" because I will run multiple instances of that. Using the same published port would cause a startup error since the port of the host cannot be bound more than one time.

Comments about network declaration

– we don’t define a user-defined bridge network (we use the default created by docker-compose).
We could have adjusted that to reduce the communication between the services to the minimum but that would make things more complex because adding more than one docker network to the consul service would require to specify the cluster address in order to remove the ambiguity about the IP address to communicate to the cluster agents. Indeed, each docker network associated to a service creates a new network interface on the container machine.

The pom.xml files

Common things for weather-forecast-service and what-to-do-service

We really need to declare few things.
Some interesting things :
– the parent pom is Spring Boot Starter Parent. That structures our pom.
– whereas we only import dependencies of the spring cloud bom as dependencies management.
– the spring cloud version and the spring boot version (second digit) have to design to work together. We don’t choose it randomly.
– only HORTON version and above provides the spring cloud load balancer.
– Spring Boot provides several starters for consul (bus, discovery, configuration and all). Here we declare only what we need : the discovery starter. If you need more, add them.
– We need to include spring-boot-starter-actuator because by default Consul uses the URI actuator/health as service health check.
– We remove Hystrix and Ribbon because we don’t need that. That prevent from using that by mistake and it also makes the final jar smaller.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
 
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.4.RELEASE</version>
        <relativePath></relativePath>
    </parent>
 
    <groupId>davidxxx</groupId>
    <artifactId>weather-forecast-service</artifactId>
    <version>0.0.1-SNAPSHOT</version>
 
 
    <properties>
        <java.version>11</java.version>
        <spring-cloud.version>Hoxton.RELEASE</spring-cloud.version>
    </properties>
 
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
 
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-consul-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-consul-discovery</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>org.springframework.cloud</groupId>
                    <artifactId>spring-cloud-netflix-hystrix</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.springframework.cloud</groupId>
                    <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>
 
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
 
</project>

A small add for what-to-do-service

To use the client load balancer from what-to-do-service, we need to include that dependency :

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        </dependency>

bootstrap.yml

we specify here things properties for the boostrap process of Spring Cloud.
All these properties have default values (thanks to Spring Boot context and consul starters) but I want to override that with custom values because it matters. During the service registry, the service name (type) in consul is equal to the spring.application.name and that is also from the service name that we query a service. So we want that that name be specified explicitly and not computed.
About cloud.consul.host, the default is localhost. It cannot work because the consul docker service is not located on each service container. That is located in a specific container that we named consul in the docker-compose template. what-to-do-service bootstrap.yml :

spring:
  application:
    name: what-to-do
  cloud:
    consul:
      host: consul
      port: 8500

weather-forecast-service bootstrap.yml :

spring:
  application:
    name: weather-forecast-service
  cloud:
    consul:
      host: consul
      port: 8500

An application.yml file identical for the two applications

– In theory only the health check endpoint exposure is required (for the actuator requirement). Here I add more by convenience.
– The consul logs set to debug may sometimes help you to understand an issue (at runtime or later). – the instanceId used during the service registry is the id of the service instance on consul.
By default the id is equal to its Spring Application Context ID : ${spring.application.name}:comma,separated,profiles:${server.port}.
It is not suitable for weather-forecast-service because Consul considers the service id as the unique way to identify an instance. If we register twice a service with the same id, Consul will just overwrite that.
In our case, we will create multiple instances of weather-forecast-service but with that default setting all instances will have the same id : weather-forecast-service:7000.
That’s why we use the ${random.value} placeholder provided by Spring boot to get unique ids.

server:
  port: 7000
 
spring:
  cloud:
    consul:
      discovery:
        instanceId: ${spring.application.name}:${server.port}:${random.value}
 
management:
  endpoints:
    web:
      exposure:
        include: "*"
 
logging.level:
    org.springframework.cloud:
      consul: DEBUG

The « forecast » endpoint

That is a rest controller that produces as text plain type a very basic information about the forecast weather : GOOD or BAD (random results).
To trace the effective invoked service instance, we log its instance id.

package davidxxx.microservicewithconsul;
 
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;
 
import java.util.Random;
 
@RestController
@RequestMapping(value = "/forecast", produces = MediaType.TEXT_PLAIN_VALUE)
public class WeatherForecastController {
 
  private static final Logger logger = LoggerFactory.getLogger(WeatherForecastController.class);
  private String instanceId;
 
  public WeatherForecastController(@Value("${spring.cloud.consul.discovery.instanceId}") String instanceId){
    this.instanceId = instanceId;
  }
 
  @RequestMapping(method = RequestMethod.GET)
  public ResponseEntity<String> get() {
    logger.info("get() from the " + instanceId + " instance");
    int randomInt = new Random().nextInt(2);
    if (randomInt == 0) {
      return ResponseEntity.ok("GOOD");
    } else {
      return ResponseEntity.ok("BAD");
    }
 
  }
}

The « what-to-do » endpoint

That is a rest controller that produces as text plain type the activity that we should do according to the forecast weather.

package davidxxx.microservicewithconsul;
 
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;
 
@RestController
@RequestMapping(value = "/what-to-do", produces = MediaType.TEXT_PLAIN_VALUE)
public class WhatToDoController {
 
  private RestTemplate restTemplate;
 
  public WhatToDoController(RestTemplate restTemplate) {
    this.restTemplate = restTemplate;
  }
 
  @RequestMapping(method = RequestMethod.GET)
  public ResponseEntity<String> get() {
    ResponseEntity<String> responseEntity =
        restTemplate.getForEntity("http://weather-forecast/forecast", String.class);
    String weather = responseEntity.getBody();
    if ("GOOD".equals(weather)){
      return ResponseEntity.ok("Go out!");
    }
    else{
      return ResponseEntity.ok("Stay at home!");
    }
 
  }
}

In that rest controller we use a RestTemplate. So we need to declare that. But by default RestTemplates don’t use the client loadbalancer even if exists. We need to explicitly set that :

@LoadBalanced
@Bean
RestTemplate template() {
	return new RestTemplate();
}

The docker-compose up with scale

We will in detached mode : build the images, create the containers, start them (which 3 instances for weather-forecast-service) and we chain that command to a following log command : docker-compose up --build --force-recreate -d --scale weather-forecast-service=3 && docker-compose logs -f

The test

After docker-compose up is returned, docker-compose ps shows us the running containers.
Now as test we request the what-to-do service several times .
The below image illustrates that (zoom on that to see it bigger) :

If we look in the weather-forecast-service logs (with for example docker-compose logs -f weather-forecast-service that logs all instances of that), we can see that the Spring Cloud client-loadbalancer does your job : the instance returned by the loadbalanced RestTemplate are effectively loadbalanced :

weather-forecast-service_3  | 2020-02-19 20:23:46.162  INFO 6 --- [nio-7000-exec-7] d.m.WeatherForecastController : get() from the weather-forecast:7000:cd7558c7869ec8eaec42b96928f2e3af instance
weather-forecast-service_2  | 2020-02-19 20:23:58.302  INFO 7 --- [nio-7000-exec-8] d.m.WeatherForecastController : get() from the weather-forecast:7000:b9d1af1f1182108172edd3f1a10da665 instance
weather-forecast-service_1  | 2020-02-19 20:23:58.686  INFO 6 --- [nio-7000-exec-9] d.m.WeatherForecastController : get() from the weather-forecast:7000:761e564bb3f89a1222b52801db705d3c instance
weather-forecast-service_3  | 2020-02-19 20:23:59.110  INFO 6 --- [nio-7000-exec-9] d.m.WeatherForecastController : get() from the weather-forecast:7000:cd7558c7869ec8eaec42b96928f2e3af instance
weather-forecast-service_2  | 2020-02-19 20:23:59.500  INFO 7 --- [nio-7000-exec-7] d.m.WeatherForecastController : get() from the weather-forecast:7000:b9d1af1f1182108172edd3f1a10da665 instance

Source code

Clone the git repository https://github.com/ebundy/microservice-with-docker-compose-and-consul/ and follow instructions in the README or above

Microservices with Spring Boot , Spring Cloud, Consul and Docker