Logstash整合Kafka( 三 )

这里Kibana全文搜索使用的是query_string语法 , 下面是常用的参数

  • query:可以使用简单的Lucene语法
  • default_field:指定默认查询哪些字段 , 默认值是_all
  • analyze_wildcard:默认情况下 , 通配符查询是不会被分词的 , 如果该属性设置为true , 将尽力去分词 。 (原文:By default, wildcards terms in a query string are not analyzed. By setting this value to true, a best effort will be made to analyze those as well.)
下面是ES官方文档的相关说明
WildcardsWildcard searches can be run on individual terms, using ? to replace a single character, and * to replace zero or more characters:qu?ck bro*Be aware that wildcard queries can use an enormous amount of memory and perform very badly?—?just think how many terms need to be queried to match the query string "a* b* c*".WarningAllowing a wildcard at the beginning of a word (eg "*ing") is particularly heavy, because all terms in the index need to be examined, just in case they match. Leading wildcards can be disabled by setting allow_leading_wildcard to false.Wildcarded terms are not analyzed by default?—?they are lowercased (lowercase_expanded_terms defaults to true) but no further analysis is done, mainly because it is impossible to accurately analyze a word that is missing some of its letters. However, by setting analyze_wildcard to true, an attempt will be made to analyze wildcarded words before searching the term list for matching terms.遇到的问题和解决方法Q : 公司之前的架构是Flume + KafKa + Logstash + ES , 但是使用Flume作为Shipper端添加相关的type、host、path等Header字段会按照StringSerializer序列化到Kafka中 , 但是Logstash无法解析Flume序列化后的Header字段A : 将Shipper端换成Logstash , 保证Shipper和Indexer用同样的序列化和反序列化方式 。
【Logstash整合Kafka】Q : 最近部署了线上的logstash , 发现一个问题ES的host字段为0.0.0.0 , 这个host是Logstash Shipper端自动添加的Header字段 。 A : 后来发现是因为/etc/hosts的IP、主机名和hostname不一致导致的, 只要设置成一致就可以解决这个问题了 。