Getting random documents from ElasticSearch in a native way without code



I am using elastic search (ES) as my backend no-sql DB.
One of the tasks I had today is to fetch random amount of documents from specific index/type,
The initial thought was to fetch the number of documents matching my criteria, and than fetch random page within in - but this would require 2 round trips (although quick and under 50 ms) and some code

It felt weird, I wanted a way that will use the native capabilities of ES, After 15 min of search and reading I found a way to do it!

The process is based on the fact that in the query syntax I can define to ES on how to calculate the score per each document (this is done using function_score)
The nice thing is that by default, the query is sorted by the score desc and when asking for X amount of documents, I get actually X amount of random documents :)

Sample query is below


The "random_score" function actually compute a random number between 0-1 --> the score will be 0-1

Be aware that using this feature will load field data for _uid, which can be a memory intensive operation since the values are unique.


  • In my scenario i can predict the size of max _uid and can hold this.
Thanks wikimedia for the image



Getting random documents from ElasticSearch in a native way without code Getting random documents from ElasticSearch in a native way without code Reviewed by Ran Davidovitz on 9:59 AM Rating: 5

No comments:

Powered by Blogger.