“Searching and Sorting”, made simpler with ElasticSearch and RESTful APIs

source: pixabay, author: Free-Photos

In comes Elasticsearch, can it solve our problem?

Elasticsearch is a distributed, free and open search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured.

From elastic.co, “What is Elasticseach?”

Ironically, I came to know of Elasticsearch when I was instructed to use the ELK stack (Elasticsearch, Logstash, and Kibana) as the logging solution by my past employer. It’s a powerful solution, able to retrieve search results from both structured and unstructured data.

Other Benefits

Separately, as an Open Source Software (OSS), Elasticsearch is free to use. This provides additional benefits, such as wider support from the community that is already using it (and contributing to fixes along the way). Having the capability to host the service yourself can also be a big plus, especially if you are a small company or don’t have the budget to work with third-party tools.

Integrating Elasticsearch

In the last chapter, I talked about how Elasticsearch can be viewed in terms of its “indices” and “documents”. In some ways, I would view Elasticsearch as a datastore, and instead of using SQL queries as with SQL datastores, we can use their APIs for reading and writing information.

Elasticsearch API, an API wrapper

Putting together the above observations, I came up with Elasticsearch Wrapper API, serving as a wrapper to do the following:

  • HTTP Endpoints to create and retrieve documents
  • An HTTP endpoint to patch (partially update) documents whenever data is updated on the originating datastore.
  • The endpoint to list documents must have a simple interface for searching and sorting
Elasticsearch Wrapper API — full documentation found here

Searching

For solving our problem to search for data, we have two endpoints.

Miscellaneous

This service was written in Golang, where I am using the go-kit framework. Using this framework allows me to easily extend the above in the future to using other communication protocols, such as gRPC. This creates the potential for an easy-to-integrate service, especially if you have many different teams that use different communication protocols.

TL;DR

By creating an API wrapper on Elasticsearch, we can use it to solve search problems in a more consistent way. This is especially powerful with integrating into service clusters, such as that implemented under SOA.

Staff Engineer, Hubble | Former Tech Lead, Rakuten Viki | bit.ly/weiyuan

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store