[SOLVED] can the number of null properties in elasticsearch document have some considerable impact when scaled?

Issue

This Content is from Stack Overflow. Question asked by Karthikeyan Amaresan

We have a cluster with an index of doc type that can have up-to a maximum of 120 properties, but most of the time most properties will have value as null.

Will it make a difference in elasticsearch if we choose to drop the properties having null, instead of being updated with null values.

For example, Consider there is a doc with 5 property

{
  "firstName" : "Guna", 
  "lastName" : "Sekar", 
  "phone" : null,
  "email" : "guna.sekar@alkasdf.com",
  "country" : null
}

Will it make a difference, if we choose to drop the properties from being stored if the value is null

{
  "firstName" : "Guna", 
  "lastName" : "Sekar", 
  "email" : "guna.sekar@alkasdf.com"
}

is dropping the properties with null value from being stored will help improve something, for example

  1. storage space
  2. Overall cluster query performance etc..

Thanks in Advance.



Solution

you can drop the fields that don’t have values and it will save some storage and may make queries a little more performant

the flip side is how do users that interact with this index know that a field that is missing implies a null value?

overall there’s probably not a super high value in doing this, but you should test to see how it applies to your data and mappings and see if it’s deemed worthwhile


This Question was asked in StackOverflow by Karthikeyan Amaresan and Answered by Mark Walkom It is licensed under the terms of CC BY-SA 2.5. - CC BY-SA 3.0. - CC BY-SA 4.0.

people found this article helpful. What about you?