Hello Readers!! We are again back with a new interesting topic with this blog. While using kibana some of you may have faced an issue of unavailable values in Kibana for index fields with dot notations. So In this blog, we will see why we face this problem and what we can do to resolve this issue.
Why do we not get data in Fields with Dot Notations?
If you are facing the issue of empty values in Kibana for index field names containing dots, this issue is caused by Kibana’s treatment of dots in field names. In Elasticsearch, we get the data but not in kibana. In Kibana, dots are used as separators in the field names, which can result in empty values in the visualization if the dots are not properly escaped. This issue may occur because Elasticsearch and Kibana handle field names with dots differently.
What are scripted fields in Kibana?
Scripted fields in Kibana are calculated fields that are generated using a script. They are used to derive new values based on existing data and to manipulate the data before it is displayed in visualizations and dashboards. Scripted fields can be created using either Painless or Lucene expressions, and they can be used in conjunction with other fields to provide additional insights into your data.
Some common use cases for scripted fields include:
- Deriving new values based on existing data, such as calculating the difference between two fields.
- Formatting data, such as converting timestamps into human-readable dates.
- Aggregating data, such as counting the number of unique values in a field.
- Transforming data
How to fix this using scripted fields in Kibana?
As this issue occurs because Elasticsearch and Kibana handle field names with dots differently. There can be a number of ways to solve this issue. So here we will use Kibana scripted field scripts. Follow the following steps in Kibana:
Step 1: Go to the “Management” section in Kibana and select “Index Patterns.” Find the index pattern that contains the fields with dots and click on it.
This is my index pattern.
You can see here the index_field containing dot notations:
Step 2: Move to the “Scripted Fields” tab and after this click on “Add Scripted Field.”
Give the scripted field a name without dots, for example, “field_without_dots.” Also select the type as per your respective fields.
Step 3: In the script field, enter the following code:
return doc['field_with_dots'].value
Replace “field_with_dots” with the actual name of the field that contains dots.
And now click “Create field” to save the scripted field. As you can see below my scripted field is created successfully.
Now, this scripted field can now be used in discover, visualizations, and dashboards, just like all other index fields.
Yes, we are all done now!! I hope this will help you somewhere.
Conclusion:
Thank you for sticking to the end. In this blog, we have learned how we can fix the issue of unavailable values in Kibana for index fields with dot notations. This is really very useful. I hope this blog helped you somewhere. Please share if you liked this blog. Kindly reach out to me for any related queries.
HAPPY LEARNING!
