> 主要为一个博客网站搭建站内搜索，让用户更容易查询到自己感兴趣的内容，主要包含功能：模糊查询、分页、关键词高亮显示、排序。 ### 一、使用工具 1. elasticsearch（6.8.0）：开源分布式搜索引擎，是整个ELK架构的核心 2. logstash（6.8.0）：对数据进行收集、过滤，存储到ELasticsearch中 3. kibana（6.8.0）：可以为 Logstash 和 ElasticSearch 提供的日志分析友好的 Web 界面，可以帮助您汇总、分析和搜索重要数据日志 4. 操作系统：Linux（Windows下安装较Linux比较简单，Linux下安装会出现一些问题） ### 二、安装教程 #### 2.1 elasticsearch安装 elasticsearch安装： 1. 下载地址：[elasticsearch下载地址](https://www.elastic.co/cn/downloads/past-releases#elasticsearch) 2. 将下载的文件通过 ftp 上传到服务器的 /usr/local/es 目录下 3. 解压文件，执行命令 ```bash tar -zxvf elasticsearch-6.8.0.tar.gz ``` 4. 删除压缩包，执行命令 ```bash rm -rf elasticsearch-6.8.0.tar.gz ``` 5. 修改配置文件 /usr/local/es/elasticsearch-6.8.0/config/elasticsearch.yml，将注释去掉 ```bash cluster.name: my-application ``` 6. 进入到 /usr/local/es/elasticsearch-6.8.0/bin 目录下，执行以下命令启动 ```bash ./elasticsearch -d ``` 启动过程中，会遇到一些问题，问题解决方案在文章最后都有整理。 ik分词器安装： 1. 下载地址：[ik分词器下载地址](https://github.com/medcl/elasticsearch-analysis-ik/releases/tag/v6.8.0) 2. 将下载的文件通过 ftp 上传到服务器的 /usr/local/es/elasticsearch-6.8.0/plugins/ik 目录下 3. 安装zip命令 ```bash yum install -y unzip ``` 4. 解压文件，执行命令 ```bash unzip elasticsearch-analysis-ik-6.8.0.zip ``` 5. 删除压缩包，执行命令 ```bash rm -rf elasticsearch-analysis-ik-6.8.0.zip ``` #### 2.2 kibana安装 1. 下载地址：[kibana下载地址](https://www.elastic.co/cn/downloads/past-releases/kibana-6-8-0) 2. 将下载的文件通过 ftp 上传到服务器的 /usr/local/es 目录下 3. 解压文件，执行命令 ```bash tar -zxvf kibana-6.8.0-linux-x86_64.tar.gz ``` 4. 删除压缩包，执行命令 ```bash rm -rf kibana-6.8.0-linux-x86_64.tar.gz ``` 5. 修改配置文件 /usr/local/es/kibana-6.8.0-linux-x86_64/config/kibana.yml，将注释去掉 ```bash server.port: 5601 server.host: "0.0.0.0" elasticsearch.hosts: ["http://127.0.0.1:9200"] ``` 6. 进入到 /usr/local/es/kibana-6.8.0-linux-x86_64/bin 目录下，执行以下命令启动 ```bash ./kibana ``` 7. 创建mapping映射，运行以下三条命令即可 ```bash GET /article/_mapping DELETE /article PUT /article { "mappings": { "_doc": { "properties": { "@timestamp": { "type": "date" }, "@version": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "description": { "type": "text", "analyzer": "ik_smart", "search_analyzer": "ik_smart", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "id": { "type": "long" }, "modified_time": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "title": { "type": "text", "analyzer": "ik_smart", "search_analyzer": "ik_smart", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } } } } ``` #### 2.3 logstash安装 1. 下载地址：[logstash下载地址](https://www.elastic.co/cn/downloads/past-releases/logstash-6-8-0) 2. 将下载的文件通过 ftp 上传到服务器的 /usr/local/es 目录下 3. 解压文件，执行命令 ```bash tar -zxvf logstash-6.8.0.tar.gz ``` 4. 删除压缩包，执行命令 ```bash rm -rf logstash-6.8.0.tar.gz ``` 5. 上传Mysql驱动包到logstash根目录下（maven仓库可以下载jar包：[Mysql驱动包下载地址](https://mvnrepository.com/artifact/mysql/mysql-connector-java/5.1.45) 6. 在 /usr/local/es/logstash-6.8.0/bin 目录下新建文件logstash-mysql-es.conf，文件内容为 ```bash input { jdbc { jdbc_driver_library => "/usr/local/es/logstash-6.8.0/mysql-connector-java-5.1.45.jar" jdbc_driver_class => "com.mysql.jdbc.Driver" jdbc_connection_string => "jdbc:mysql://xxxx:3306/xxxx" jdbc_user => "xxxx" jdbc_password => "xxxx" schedule => "* * * * *" statement => "SELECT id,title,description,DATE_FORMAT(modified_time, '%Y-%m-%d %H:%i:%s') AS modified_time FROM blog WHERE modified_time >= :sql_last_value" use_column_value => true tracking_column_type => "timestamp" tracking_column => "modified_time" last_run_metadata_path => "syncpoint_table" } } output { elasticsearch { hosts => ["127.0.0.1:9200"] index => "article" document_id => "%{id}" document_type => "_doc" } stdout { codec => json_lines } } ``` 7. 进入到 /usr/local/es/logstash-6.8.0/bin 目录下，执行以下命令启动 ```bash ./logstash -f logstash-mysql-es.conf & ``` ### 三、Java集成elasticsearch - pom.xml ```xml 4.0.0 com.xxx.xxx xxx-xxx-parent 1.0-SNAPSHOT org.springframework.boot spring-boot-starter-parent 2.1.1.RELEASE UTF-8 UTF-8 1.8 org.springframework.boot spring-boot-starter org.springframework.boot spring-boot-starter-web org.springframework.boot spring-boot-starter-test org.springframework.boot spring-boot-starter-aop org.projectlombok lombok ${lombok.version} org.apache.commons commons-lang3 org.springframework.boot spring-boot-starter-data-elasticsearch ``` - 实体类 ```java @Getter @Setter @Document(indexName="article",type="_doc") public class ESArticleEntity { // 文章标识 @Id private Long id; // 标题 private String title; // 描述 private String description; // 更新时间 private Date modified_time; } ``` - 搜索函数 ```java Integer pageIndex = 1; Integer pageSize = 10; String keyword = "测试"; // 查询条件 BoolQueryBuilder boolQueryBuilder = null; if(StringUtils.isNotBlank(keyword)){ boolQueryBuilder = QueryBuilders.boolQuery() .should(QueryBuilders.multiMatchQuery(keyword, "title","description")); }else{ boolQueryBuilder = QueryBuilders.boolQuery(); } // 高亮设置 HighlightBuilder.Field titleHighLight = new HighlightBuilder.Field("title") .preTags("") .postTags("") //注意加上高亮片数即可 .fragmentSize(800000) .numOfFragments(0) .requireFieldMatch(false); HighlightBuilder.Field descriptionHighLight = new HighlightBuilder.Field("description") .preTags("") .postTags("") //注意加上高亮片数即可 .fragmentSize(800000) .numOfFragments(0) .requireFieldMatch(false); // 排序 FieldSortBuilder sort = SortBuilders.fieldSort("id").order(SortOrder.DESC); SearchQuery query = new NativeSearchQueryBuilder() .withQuery(boolQueryBuilder) .withSort(sort) .withHighlightFields(descriptionHighLight,titleHighLight) .withPageable(PageRequest.of( pageIndex - 1, pageSize)) .build(); AggregatedPage page = elasticsearchTemplate.queryForPage(query, ESArticleEntity.class, new SearchResultMapper() { @Override public AggregatedPage mapResults(SearchResponse response, Class clazz, Pageable pageable) { ArrayList articleList = new ArrayList(); SearchHits hits = response.getHits(); for (SearchHit searchHit : hits) { if (hits.getHits().length <= 0) { return new AggregatedPageImpl((List) articleList); } Map sourceAsMap = searchHit.getSourceAsMap(); String title = (String) sourceAsMap.get("title"); String description = (String) sourceAsMap.get("description"); Long id = Long.valueOf(sourceAsMap.get("id").toString()); Date modifiedTime = TimeUtils.parseTime(sourceAsMap.get("modified_time").toString()); ESArticleEntity article = new ESArticleEntity(); article.setId(id); article.setModified_time(modifiedTime); HighlightField titleHighlightField = searchHit.getHighlightFields().get("title"); if (titleHighlightField == null) { article.setTitle(title); } else { article.setTitle(titleHighlightField.fragments()[0].toString()); } HighlightField descriptionHighlightField = searchHit.getHighlightFields().get("description"); if (descriptionHighlightField == null) { article.setDescription(description); } else { article.setDescription(descriptionHighlightField.fragments()[0].toString()); } articleList.add(article); } return new AggregatedPageImpl((List) articleList); } }); // 5、查询总记录数 long totalElements = elasticsearchTemplate.count(query); long totalPages = (totalElements + pageSize - 1)/pageSize; // 6、数据返回 PageResponse response = new PageResponse(); response.setPageIndex(pageIndex); response.setTotol(totalElements); response.setPages((int)totalPages); response.setData(page.getContent()); return response; ``` ### 四、实现效果 ![实现效果](https://www.88-bit.com/images/blog/fdd54f69ace04487bbfeffe095b980da.png) ### 五、启动问题 - can not run elasticsearch as root（root账号下不允许启动）添加用户组es,及用户es 密码password 给用户 es:es 添加指定目录拥有权限 ```bash groupadd es useradd es -g es -p password #-g 指定组 -p 指定密码 chown -R es:es elasticsearch-6.8.0/ #-R : 处理指定目录下的所有文件 ``` 切换到用户,并执行elasticsearch ```bash su es cd elasticsearch-6.8.0/bin ./elasticsearch ./elasticsearch -d #后台方式启动 ```

他的文章