A configurable framework for implementing full text search using Hibernate Search

In this blog, I will describe a database driven framework that I implement to perform full text search of Hibernate entities using the Hibernate Search project.

Setup Hibernate Search

Including Hibernate Search in your project is straight forward. Follow the instruction in the documentation here. I add the following  Maven dependencies:

 <dependency>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate-search</artifactId>
      <version>4.1.1.Final</version>
 </dependency>
 <!-- Additional Analyzers: -->
 <dependency>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate-search-analyzers</artifactId>
      <version>4.1.1.Final</version>
 </dependency>
 <!-- Infinispan integration: -->
 <dependency>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate-search-infinispan</artifactId>
      <version>4.1.1.Final</version>
 </dependency>

The framework

Domain layer

The “configurable” part of the framework is built around a SearchPreference class as shown below

@Entity
@Indexed
public class SearchPreference extends AbstractEntity {

@Field(index=Index.YES, analyze=Analyze.NO, store=Store.NO)
private String entityName;

private String propertyName;

@Enumerated(EnumType.STRING)
@Field(index=Index.YES, analyze=Analyze.NO, store=Store.NO)
private SearchType searchType;

private BoolType boolType;

@Id
@GeneratedValue(strategy = GenerationType.AUTO)
private long id;

@Override
public long getId() {
return id;
}

// rest of codes omitted here

Note:

(1) The class SearchPreference is used to store the full text search config of an indexed property of the target Hibernate entity. It is itself a Hibernate entity for persistence to the database and has the following properties:

  • entityName – fully qualified class name of the target Hibernate entity
  • propertyName – name of an indexed property of the target Hibernate entity
  • searchType – an enum to define the type of search for this field, i.e. match, wildcard, phase
  • boolType – an enum to define the mode of aggregation of the subquery (must, should)

(2) The class AbstractEntity is an abstract entity class to be extended by all Hibernate entity classes.

(3)  A Hibernate entity is associated with multiple SearchPreference objects for full text search across multiple fields.

Repository layer

The repo layer consists of the following interfaces:

public interface ISearchPreferenceRepository extends IEntityRepository<SearchPreference> {

List<SearchPreference> getSearchFieldsFor(Class<? extends AbstractEntity> entityClass);
}

and

public interface IEntityRepository<T extends AbstractEntity> {

List<T> search(String keyword, int firstResult, int fetchSize, SearchPreference… fields);

}

The interface ISearchPreferenceRepository provides the method getSearchFieldsFor() to load the search field configs as a list of SearchPreference objects using HQL, with one for each field to be queried for. The implementation is shown below.

@Transactional
@Repository(“searchPreferenceRepository”)
public class SearchPreferenceRepository extends AbstractEntityRepository<SearchPreference> implements ISearchPreferenceRepository {

 private static final String hql = “FROM ” + SearchPreference.class.getName() + ” WHERE entityName=?”;

@Autowired(required=true)
public SearchPreferenceRepository(SessionFactory sessionFactory) {
super(sessionFactory);
}

public List<SearchPreference> getSearchFieldsFor(Class<? extends AbstractEntity> entityClass) {
Query hqlQuery = sessionFactory.getCurrentSession().createQuery(hql);
hqlQuery.setParameter(0, ClassUtils.getShortClassName(entityClass));
return hqlQuery.list();
}

}

The actual full text search is done via the search() method of the IEntityRepository interface. Below is the implementation of the method:

public List<ENTITY> search(String keyword, int firstResult, int fetchSize, SearchPreference… fields) {

Session session = sessionFactory.getCurrentSession();

FullTextSession fullTextSession = Search.getFullTextSession(session);

QueryBuilder qb = fullTextSession.getSearchFactory().buildQueryBuilder().forEntity(getType()).get(); // [1]

BooleanJunction boolJn = qb.bool();

List<Query> subqueries = new ArrayList<Query>();

for (int i = 0; i < fields.length; i++) {

SearchPreference field = fields[i];

Query fieldQuery = null;

if (SearchType.MATCH.equals(field.getSearchType())) { // [2]

fieldQuery = qb.keyword().onField(field.getPropertyName()).matching(keyword).createQuery();

} else if (SearchType.WILDCARD.equals(field.getSearchType())) {

fieldQuery = qb.keyword().wildcard().onField(field.getPropertyName()).matching(keyword).createQuery();

}

subqueries.add(fieldQuery);

if (BoolType.MUST == field.getBoolType()) { //[3]

boolJn = boolJn.must(fieldQuery);

}

if (BoolType.SHOULD == field.getBoolType()) {

boolJn = boolJn.should(fieldQuery);

}

}

Query query = boolJn.createQuery(); //[4]

// wrap Lucene query in a org.hibernate.Query

org.hibernate.Query hibQuery = fullTextSession.createFullTextQuery(query, getType()); // [5]

hibQuery.setFirstResult(firstResult);

hibQuery.setFetchSize(fetchSize);

// execute search

List result = hibQuery.list(); // [6]

return result;

}

Its a rather long method but basically what it does is to create a lucene search query via Hibernate Search API using the SearchPreference objects in the input arguments. A Hibernate query object is then created and executed to generate the search results.

Note:

[1] – Create a DSL query builder of type QueryBuilder for the entity type.

[2] – For each field of type SearchPreference, a sub query is created for matching the input search string keyword with the query type (i.e. match, wildcard) matching the searchType property of that in the SearchPreference field.

[3] – The subqueries created in [2] are aggregated here. Currently 2 aggregation operations are supported:

  • SHOULD: the query should contain the matching elements of the subquery
  • MUST: the query must contain the matching elements of the subquery

[4] – The final query object is created.

[5] – The query object in [4] of type org.apache.lucene.search.Query is converted into a core Hibernate query object by the full text session.

[6] – Finally, the query is executed to return search results.

Service layer

The 2 repositories ISearchPreferenceRepository and IEntityRepository are used together to implement a generic full text search service. The implementation of this layer is straight forward is not included here.

Example

Let say we have an entity Product:

@Entity
@Indexed
public class Product extends AbstractEntity {

@Id
@GeneratedValue(strategy=GenerationType.AUTO)
private long id;

@Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
private String sku;

@Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
private String name;

@Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
private String description;

… // getters and setters

Note the properties (sku, description, name) are indexed using Hibernate Search annotations. They are required for the entity to be queried via full text search. Let say we have the following data in the product data (mapped to by Hibernate by default):

sku name description
sku_1 name_1 product desc 1
sku_2 name_2 product desc 2
sku_3 name_3 product desc 3

The following codelet of an unit test class demonstrates how the repository classes can be used to perform various full text search of the product entity:

public class ProductRepositoryTest {
@Autowired
 @Qualifier("productRepository")
 private IEntityRepository<Product> repository;
@Test
 public void testSearchAndMatch() throws Exception {
      SearchPreference[] fieldPref = new SearchPreference[2];
      fieldPref[0] = createPref(Product.class.getName(), "sku", SearchType.MATCH, BoolType.MUST);
      fieldPref[1] = createPref(Product.class.getName(), "description", SearchType.MATCH, BoolType.MUST);
      List<Product> results = repository.search("sku_1", 0, 100, fieldPref);
      assertEquals(0, results.size()); // search returns no result as description MUST match keyword "sku_1"
 }
@Test
 public void testSearchOrMatch() throws Exception {
      SearchPreference[] fieldPref = new SearchPreference[2];
      fieldPref[0] = createPref(Product.class.getName(), "sku", SearchType.MATCH, BoolType.SHOULD);
      fieldPref[1] = createPref(Product.class.getName(), "description", SearchType.MATCH, BoolType.SHOULD);
      List<Product> results = repository.search("sku_1*", 0, 100, fieldPref);
      assertTrue(results.size() > 0); // search returns results where sku matches keyword "sku_1", no result from matching description field
 }
@Test
 public void testSearchAndWildcard() throws Exception {
      SearchPreference[] fieldPref = new SearchPreference[2];
      fieldPref[0] = createPref(Product.class.getName(), "sku", SearchType.WILDCARD, BoolType.MUST);
      fieldPref[1] = createPref(Product.class.getName(), "description", SearchType.WILDCARD, BoolType.MUST);
      List<Product> results = repository.search("*1*", 0, 100, fieldPref);
      assertTrue(results.size() > 0); // search return results where both sku and description matches wildcard "*1*", i.e. "sku_1" and "product desc 1"
 }
@Test
 public void testSearchOrWildcard() throws Exception {
      SearchPreference[] fieldPref = new SearchPreference[2];
      fieldPref[0] = createPref(Product.class.getName(), "sku", SearchType.WILDCARD, BoolType.SHOULD);
      fieldPref[1] = createPref(Product.class.getName(), "description", SearchType.WILDCARD, BoolType.SHOULD);
      List<Product> results = repository.search("sku_*", 0, 100, fieldPref);
      assertTrue(results.size() > 0); // search return results where sku matches wildcard "sku_*", no result from matching description field
 }
         // ...

Note the method createPref() returns an instance of SearchPreference object constructed using the method’s input argument. The codes are omitted above. Refer to the highlighted comments of each test for explanation.

That’s it for now.