Setup Spring RestTemplate to accept Self Signed Cert

This is strictly for testing only but may be useful if you need to perform integration tests. For example, the system you develop needs to access another internal or 3rd party test server via https where the server’s certificate is not signed.

PKIX path building failed

By default, if you try to access a server via https with a self signed certificate, for example with the following codes

RestTemplate template = new TestRestTemplate();
template.getForObject(https://<some server>/, String.class);

you will get the following exception:

org.springframework.web.client.ResourceAccessException: I/O error on GET request for "https://<some server>":
sun.security.validator.ValidatorException: PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target;

HttpClient

To fix the above, update the RestTemplate with a custom HttpClient that accepts self-signed certificate:

      SSLConnectionSocketFactory socketFactory = new SSLConnectionSocketFactory(new SSLContextBuilder().loadTrustMaterial(null, new TrustSelfSignedStrategy()).build());

      HttpClient httpClient = HttpClients.custom().setSSLSocketFactory(socketFactory).build();

      RestTemplate template = new TestRestTemplate();
      ((HttpComponentsClientHttpRequestFactory) template.getRequestFactory()).setHttpClient(httpClient);

and add the following dependency if needed

      <dependency>
           <groupId>org.apache.httpcomponents</groupId>
           <artifactId>httpclient</artifactId>
           <scope>test</scope>
      </dependency>

Again this should only be used for testing purpose only.

Configuring Multiple JPA Entity Managers In Spring Boot

This blog will demonstrate how to setup multiple entity managers in Spring to connect to different data sources. The solution here also supports Spring Data.

Update Maven Pom file

Include Spring Boot dependency for Spring Data:

 <dependency>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-data-jpa</artifactId>
 </dependency>

Disable DataSourceAutoConfiguration

Since we are setting up the data sources, disable the auto configuration in Spring Boot

@Configuration
@ComponentScan
@EnableAutoConfiguration (exclude = {  DataSourceAutoConfiguration.class })
public class Application {
   ...

Configure Primary Entity Manager

Below is the Java configuration for the primary entity manager

@Profile("!test")          // 1
@Configuration
@EnableTransactionManagement
@EnableJpaRepositories(basePackages = "au.com.myblog.dao", entityManagerFactoryRef = "entityManager", transactionManagerRef = "transactionManager")        // 2
public class PrimaryMysqlDBConfiguration {
     @Bean(name = "dataSource")      // 3
     @Primary
     @ConfigurationProperties(prefix = "primary.datasource.mysql")
     public DataSource mysqlDataSource() {
          return DataSourceBuilder.create().build();
     }
    @PersistenceContext(unitName = "primary")   // 4
     @Primary
     @Bean(name = "entityManager")
     public LocalContainerEntityManagerFactoryBean mySqlEntityManagerFactory(EntityManagerFactoryBuilder builder) {
          return builder.dataSource(mysqlDataSource()).persistenceUnit("primary").properties(jpaProperties())
                    .packages("au.com.myblog.domain").build();
      }
     private Map<String, Object> jpaProperties() {
          Map<String, Object> props = new HashMap<>();
          props.put("hibernate.ejb.naming_strategy", new SpringNamingStrategy());
          return props;
      }
}

Note:

  1. @Profile annotation to use this configuration only for non test profile. This allows me to set a different datasource, e.g. H2, when running tests.
  2. @EnableJpaRepositories is used for Spring Data. Note we are using the default transaction manager setup in Spring
  3. Bean for primary datasource. The @ConfigurationProperties annotation specifies the prefix of the properties to use by this datasource. For the example here, the application.properties file should include properties like below:
    # DB Connection
    primary.datasource.mysql.url=jdbc:log4jdbc:mysql://localhost/bpmn_service
    primary.datasource.mysql.username=root
    primary.datasource.mysql.password=root
    primary.datasource.mysql.driver-class-name=net.sf.log4jdbc.DriverSpy
  4. Persistence unit name should be setup in the EntityManager bean as shown here.

Configure Secondary Entity Manager

Configuration of the secondary entity manager is very similar to that of the primary.The only thing is we have to define a new transaction manager. Make sure a different prefix is set in the configuration properties of the data source. Also, a different persistent unit name should be used.

@Profile("!test")
@Configuration
@EnableTransactionManagement
@EnableJpaRepositories(basePackages = "au.com.myblog.dao", entityManagerFactoryRef = "secondaryMySqlEntityManager", transactionManagerRef = "secondaryTransactionManager")
public class SecondaryMysqlDBConfiguration {
      @Bean
      @ConfigurationProperties(prefix = "secondary.datasource.mysql")
       public DataSource mysqlDataSource() {
            return DataSourceBuilder.create().build();
       }
      @PersistenceContext(unitName = "secondary")
      @Bean(name = "secondaryMySqlEntityManager")
      public LocalContainerEntityManagerFactoryBean mySqlEntityManagerFactory(EntityManagerFactoryBuilder builder) {
           return  builder.dataSource(mysqlDataSource()).properties(jpaProperties()).persistenceUnit("secondary").packages("au.com.myblog.domain").build();
       }
      @Bean(name = "secondaryTransactionManager")
       public PlatformTransactionManager transactionManager(EntityManagerFactoryBuilder builder) {
             JpaTransactionManager tm = new JpaTransactionManager();
             tm.setEntityManagerFactory(mySqlEntityManagerFactory(builder).getObject());
             tm.setDataSource(mysqlDataSource());
             return tm;
       }
       private Map<String, Object> jpaProperties() {
             Map<String, Object> props = new HashMap<>();
            // naming strategy to put underscores instead of camel case
            // as per auto JPA Configuration
            props.put("hibernate.ejb.naming_strategy", new SpringNamingStrategy());
            props.put("hibernate.hbm2ddl.auto", "update");
            return props;
       }

Note the setup above assumes the primary and secondary data sources are not used together in a single transaction. Hence we can use 2 independent transaction managers.

Specify which Entity Manager to use

Finally, make sure that you specify the name of the secondary transaction in the spring @Transactional annotation:

@Transactional(value = "secondaryTransactionManager")

Also, add the @PersistenceContext annotation with the unit name as defined in the configuration when injecting entity managers;

 @PersistenceContext(unitName = "secondary")
 private EntityManager entityManager;

That’s it!

Validating Spring MVC Request Mapping Method parameters

This short post demonstrates how to set up and use JSR-303 Validation for the arguments in Spring MVC request mapping methods for path variable and request parameter arguments. I am using Spring Boot v1.2.5 with Maven 3.

I. MethodValidationPostProcessor

The only configuration needed is adding the MethodValidationPostProcessor (javadoc) bean to the Spring configuration, e.g.

 @Bean
 public MethodValidationPostProcessor methodValidationPostProcessor() {
      return new MethodValidationPostProcessor();
 }

II Add validation to controller request mapping method

First, add the @Validate annotation to the controller class as follows:

@RestController
@Validated
public class HelloController {
     ...

Then add any JSR-303 validation annotation to a request mapping method arguments:

 @RequestMapping("/hi/{name}")
 public String sayHi(@Size(max = 10, min = 3, message = "name should have between 3 and 10 characters") @PathVariable("name") String name) {
      return "Hi " + name;
 }

The example codes above shows how to validate a value in the request path marked by @PathVariable. You can do the same with @RequestParam

III Validation exception handling

A ConstraintViolationException will be thrown if the size of the path variable is not within 3 to 10 characters. You may need to catch this exception and process it using a Spring MVC exception handler, for example to return the error messages in the response:

 @ExceptionHandler(value = { ConstraintViolationException.class })
 @ResponseStatus(value = HttpStatus.BAD_REQUEST)
 public String handleResourceNotFoundException(ConstraintViolationException e) {
      Set<ConstraintViolation<?>> violations = e.getConstraintViolations();
      StringBuilder strBuilder = new StringBuilder();
      for (ConstraintViolation<?> violation : violations ) {
           strBuilder.append(violation.getMessage() + "\n");
      }
      return strBuilder.toString();
 }

Using Netflix servo to monitor Java applications

Netflix servo is a lightweight API for exposing and publishing application metrics. This blog will demonstrate step by step how to use servo in a Java application.

Project set up

I use Maven so the first step is to include the following in the pom file:

 <dependency>
      <groupId>com.netflix.servo</groupId>
      <artifactId>servo-core</artifactId>
      <version>0.8.0</version>
 </dependency>

Adding Metrics to JMX

Adding metrics with servo is simple. For example, the following codes add metrics to a Rest controller for a single end point (/sayhi):

@RestController("testController")
public class TestController {

 @Monitor(name = "requestCounter", type = DataSourceType.COUNTER, description = "Total number of requests", level = DataSourceLevel.INFO)
 private final AtomicInteger requestCounter = new AtomicInteger(0);
 
 @Monitor(name = "aGauge", type = DataSourceType.GAUGE, description = "A random gauge", level = DataSourceLevel.CRITICAL)
 private final AtomicInteger aGauge = new AtomicInteger(0);
 
 @MonitorTags
 private final TagList tags = BasicTagList.of("id", "testController", "class", "au.com.dac.controller.TestController");
 
 @PostConstruct
 public void init() {
      Monitors.registerObject("testController", this);
 }
 
 @RequestMapping(value = "/sayhi", method = RequestMethod.GET )
 public String sayHi(@RequestParam String to) {
      requestCounter.incrementAndGet(); // increment counter
      aGauge.set(RandomUtils.nextInt(0, 100)); // set some random value
      return "hi " + to;
 }
 
}

Two metrics are defined in this class using the @Monitor annotation. There are 3 types of monitors: counter, gauge and informational. Note also the user of @MonitorTags annotation. This is used to add a set of tags as key-value pairs to all the annotated fields in the class. In the example above, two key-value pairs with key “id” and “class” and values “testController” and “au.com.dac.controller.TestController” respectively.

The object needs to be registered with a monitor registry for it to be monitored. This is done with the following line:

Monitors.registerObject("testController", this);

The registerObject method of the class Monitors use the default monitor registry class DefaultMonitorRegistery to register an object. By default, a JMXMonitorRegistry is used.

To update the monitors, use their corresponding API methods:

 requestCounter.incrementAndGet(); // increment counter
 aGauge.set(RandomUtils.nextInt(0, 100)); // set some random value

Now the metrics are available in JMX and can be viewed via JConsole or VisualVM.

Publishing to other sources

Servo provides a simple and yet powerful API for collecting and publishing metrics to other sources. This makes it easy to add monitors to a Java application and expose them to external monitoring system such as Stackdriver and AWS CloudWatch. The following method setup a metrics poller to collect all metrics regularly (once a minute) and publish to the included metric observers.

 private void initMetricsPublishing() {
      PollScheduler scheduler = PollScheduler.getInstance();
      scheduler.start();
      MetricObserver logObserver = new LoggerMetricObserver("logger-observer"); 
      MetricObserver logObserverRate = new LoggerMetricObserver("logger-observer-rate");
      MetricObserver transform = new CounterToRateMetricTransform(logObserverRate, 1, TimeUnit.MINUTES);
      PollRunnable task = new PollRunnable(
           new MonitorRegistryMetricPoller(),
           BasicMetricFilter.MATCH_ALL,
           logObserver, transform);
      scheduler.addPoller(task, 1, TimeUnit.MINUTES);
 }

Note the poller is associated with two MetricObserver instances logObserver and logObserverRate. The later is wrapped by a CounterToRateMetricTransformer which is used to transform any counter values in the observer into rates.

MetricObserver

This is the interface servo uses to publish metrics to. Servo provides a few example implementations, include one for AWS CloudWatch. The following implements a metric observer that logs some metric information

public class LoggerMetricObserver extends BaseMetricObserver {
 private Logger logger = LoggerFactory.getLogger(getClass());

 public LoggerMetricObserver(String name) {
      super(name);
 }
 @Override
 public void updateImpl(List<Metric> metrics) {
      Preconditions.checkNotNull(metrics, "metrics");
      try {
         for (Metric m : metrics) {
              logger.info("{}: name[{}] tags[{}] value[{}]", new LocalDateTime(m.getTimestamp()), m.getConfig()
 .getName(), m.getConfig().getTags(), m.getValue());
         }
       } catch (Throwable t) {
              incrementFailedCount();
              throw Throwables.propagate(t);
       }
 }
}

Running the Application

The complete source codes can be found in here. Running the web application (embedded Tomcat) and sending requests to http://localhost:8080/sayhi?to=Raymond with the browsers should display on the console something like the below every minute:

2015-08-04 23:48:10.119 INFO 952 --- [PollScheduler-3] a.c.m.b.servo.LoggerMetricObserver : 2015-08-04T23:48:10.119: name[requestCounter] tags[class=TestController,COUNTER,id=testController,INFO] value[3]
2015-08-04 23:48:10.119 INFO 952 --- [PollScheduler-3] a.c.m.b.servo.LoggerMetricObserver : 2015-08-04T23:48:10.119: name[aGauge] tags[class=TestController,GAUGE,id=testController,CRITICAL] value[51]
2015-08-04 23:48:10.120 INFO 952 --- [PollScheduler-3] a.c.m.b.servo.LoggerMetricObserver : 2015-08-04T23:48:10.119: name[requestCounter] tags[class=TestController,RATE,id=testController,INFO] value[0.04987697014032054]
2015-08-04 23:48:10.120 INFO 952 --- [PollScheduler-3] a.c.m.b.servo.LoggerMetricObserver : 2015-08-04T23:48:10.119: name[aGauge] tags[class=TestController,GAUGE,id=testController,CRITICAL] value[51]

Note the log messages for the 2 metric observers.

Using Spring 4 Condition To Control Bean Registration

Spring 4 introduces the Condition interface which supports a condition to checked and registers a bean only when the condition is matched. In this blog, I will demonstrate how to implement a custom condition interface which will register beans when the active profiles match a regex defined in the condition. This would be useful, for example, to inject mock beans when testing while using the actual implementations at deployment. The classes implemented here is a slight modification of the standard @Profile annotation and supporting classes.

@ProfileRegex

First we define the interface @ProfileRegex as follows

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.METHOD})
@Conditional(ProfileRegexCondition.class)
public @interface ProfileRegex {
      /**
      * The profile regex for which the annotated component should be registered.
      */
      String value();
}

The interface requires a String value attribute which define the regex to use to match the spring context’s active profiles. Note the use of @Condition annotation to define the class that implements the Condition interface, i.e. ProfileRegexCondition.

ProfileRegexCondition

The class ProfileRegexCondition implements the Condition interface

class ProfileRegexCondition implements Condition {

 @Override
 public boolean matches(ConditionContext context, AnnotatedTypeMetadata metadata) {
       Environment environment = context.getEnvironment();
       if (environment != null) {
            String[] profiles = environment.getActiveProfiles();
            MultiValueMap<String, Object> attrs = metadata.getAllAnnotationAttributes(ProfileRegex.class.getName());
           if (attrs != null) {
                for (Object value : attrs.get("value")) {
                     for (String profile : profiles) {
                          boolean matches = Pattern.matches((String) value, profile);
                          if (matches) {
                             return true;
                          }
                     }
                }
                return false;
          }
      }
      return true;
 }
}

The Condition interface has only one method matches() which determines whether the condition is matched. In our implementation, it iterates all the active profiles and return true if the profile matches the regex as defined in the value of the @ProfileRegex annotation.

Example

Suppose we want to register a mock bean for unit testing. We could define the following bean configuration class

@Configuration
@EnableAutoConfiguration
@ComponentScan
@ProfileRegex("test")
public class TestConfiguration {
     @Bean
     MyService myServiceMockBean() {
         return mock(MyService.class);
     }
}

Note the use of @ProfileRegex annotation to only register the beans when the active profile matches the regex “test”.  Also, the @ProfileRegex annotation can also be applied to the individual @Bean method to provide a more fine control over bean registration. The following configuration class does the opposite of the above and registers whenever the active profiles does not match the regex “test”.

@Configuration
@ProfileRegex("((?!test).)*")
public class ServiceConfiguration {

    @Bean
    MyService myServiceBean() {
         return new MyServiceImpl();
    }
}

Below is the unit test to verify the correct (mock) bean is registered

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = TestConfiguration.class)
@ActiveProfiles({"test"})
public class ServiceConfigTest {
 
    @Autowired
    private ApplicationContext context;

    @Test
    public void testProfileRegexCondition() {
         MyService service = context.getBean(MyService.class);
         assertThat(service, CoreMatchers.notNullValue());
         assertThat(new MockUtil().isMock(service), equalTo(true));
    }
}

Finally, Spring Boot provides a number of useful conditional annotations under the package org.springframework.boot.autoconfigure.condition. See the javadoc for more details.

Query MongoDB using the Java Driver

In this blog I will demonstrate how to get start with MongoDB Java Driver with examples on how to execute some common queries to the database.

Set up

I use Spring Boot (v1.2.1) with Maven for my project setup and include the following Maven dependency

 <dependency>
      <groupId>org.mongodb</groupId>
      <artifactId>mongo-java-driver</artifactId>
 </dependency>

My MongoDB server is on version 2.6

Example Document

For the rest of this blog, I will use the following example document

{
    "firstname" : "John",
    "lastname" : "Smith",
    "fullname" : "John N Smith"
    "age" : 25,
    "qualifications" : [
        "Master of Science",
        "B. Eng"
    ]
}

Exact Match

The Java driver API implements a query as a DBObject which is then used as an input to the Collection’s find method for executing a query. The query DBObject can be constructed using a BasicDBObjectBuilder. The following codes demonstrate how to find all documents within a collection with field name “firstname” with value “John”.

 BasicDBObjectBuilder builder = BasicDBObjectBuilder.start().add("firstname", "john");
 DBObject query = builder.get();
 DB db = mongoClient.getDB("mydb");
 DBCursor cursor = db.getCollection("myCollection").find(query);
 List<DBObject> dbObjects = cursor.skip(5).limit(100).toArray();

Note the 5 and 100 in the last line to indicate the first and the maximum number of records to return by the cursor.

Like

To perform like query as in relational database, you have to use regular expression when building the query DBObject using BasicDBObjectBuilder. For example,

BasicDBObjectBuilder builder = BasicDBObjectBuilder.start()
                              .add("fullname", Pattern.compile("john"));

will return documents with fullname “john smith” and “lee john” etc.

Greater Than &/or Less Than

To query with greater than or less than, create a DBObject using $gt(e) and $lt(e).

To get documents with age greater than 18:

BasicDBObjectBuilder builder = BasicDBObjectBuilder.start()
                               .add("age", new BasicDBObject("$gt", 18));

To get documents with age less than 50:

BasicDBObjectBuilder builder = BasicDBObjectBuilder.start()
                              .add("age", new BasicDBObject("$lt", 50));

To get documents with age between 18 and 50 (exclusive):

BasicDBObjectBuilder builder = BasicDBObjectBuilder.start()
                            .add("age", new BasicDBObject("$lt", 50).append("$gt", 18));

Match Value in Array

To query for all documents with matching element in array. For example to find all persons with qualification “B. Eng.”

 BasicDBObject in = new BasicDBObject("$elemMatch", new BasicDBObject("$eq", "B. Eng."));
 BasicDBObjectBuilder builder = BasicDBObjectBuilder.start().add("qualifications", in);

Combining query criteria

The BasicDBObjectBuilder can be used to combine multiple query criteria together, for example

 BasicDBObjectBuilder builder = BasicDBObjectBuilder.start()
                                .add("firstname", "john")
                                .add("age", new BasicObject("$gt", 18))