02 Elasticsearch FAQ

NTOPNG - Elasticsearch - Kibana FAQ

---

Q: Do I need Logstash to insert ntopng data in Elasticsearch?

A: No, ntopng is able to directly and properly feed data to Elasticsearch Bulk API

Example:
ntopng -F 'es;ntopng;ntopng-%Y.%m.%d;http://elasticsearch:9200/_bulk;'

Format:
         es;<idx type>;<idx name>;<es URL>;<http:auth>
Usage:
         es;flows;ntopng-%Y.%m.%d;http://localhost:9200/_bulk;
         Note: the <idx name> accepts the strftime() format.

---

Q: How often will ntopng push data to Elasticsearch?

A: Once started, ntopng will push ES flows that are expired or periodically send (every 5 mins) partial flows for long lasting flows.

---

Q: IP Location coordinates are not recognized as "geo_point"

A: Create a Template with the correct mapping for ntopng location fields, ie:

curl -XPUT localhost:9200/_template/ntopng_template -d '
{
    "template" : "ntopng-*",
    "mappings" : {
        "ntopng" : {
                "dynamic_templates": [
                { "ntopng_geopoint": { 
                    "match": "*_IP_LOCATION",
                    "mapping": {
                      "type": "geo_point"
                    }
                }}
             ]
        }
    }
}
'

ref: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html

---

Q: How can I have ES recognize correct data types from ntopng?

A: Create a Template with the correct mapping for ntopng location fields, ie:

curl -XPUT localhost:9200/_template/ntopng_template -d '
{
    "template" : "ntopng-*",
    "mappings" : {
        "ntopng" : {
                "dynamic_templates": [
		        { "ntopng_geopoint": { 
		            "match": "*_IP_LOCATION",
		            "mapping": {
		              "type": "geo_point"
		            }
		        }},
		        { "ntopng_ip": { 
		            "match": "IPV4_*",
		            "mapping": {
		              "type": "ip"
		            }
		        }},
		        { "ntopng_bytes": { 
		            "match": "*_BYTES",
		            "mapping": {
		              "type": "byte"
		            }
		        }}
     		 ]
        }
    }
}
'

---