小工具      在线工具  汉语词典  css  js  c++  java

springboot在es中的两种连接方式

elasticsearch,elasticsearch,transport 额外说明

收录于:40天前

使用客户端调用es服务的两种方式

有两种方式,一种 是9200端口或叫rest 接口,
另一种是用 节点的9300端口或叫 TransportClient

说明

es的官网中https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-api.html
We plan on deprecating the TransportClient in Elasticsearch 7.0 and removing it completely in 8.0. Instead, you should be using the Java High Level REST Client, which executes HTTP requests rather than serialized Java requests. The migration guidedescribes all the steps needed to migrate.
The Java High Level REST Client currently has support for the more commonly used APIs, but there are a lot more that still need to be added. You can help us prioritise by telling us which missing APIs you need for your application by adding a comment to this issue: Java high-level REST client completeness.
Any missing APIs can always be implemented today by using the low level Java REST Client with JSON request and response bodies.

简单点说就是 TransportClient 在es7后不支持了,所以你要用 java high level rest client,这个是用http的请求)
所以 ,后面都 是采用9200端口的方式!!!

官网会提到low-level REST client和High Level REST Client。这两个是令人震惊的rest方法,取代了transportclient方法!

aws 连elasticsearch 的 坑

项目 中是使用 transport来调用es的,但是在aws中,这种 方式是不支持的!!!,要用9200的方式
具体情况:
https://forums.aws.amazon.com/thread.jspa?messageID=683536
在这里插入图片描述

不过这个还好,官网也说了,之后高版本会放弃这个,所以早做改变也是OK的.
org.springframework.data.elasticsearch.core中的ElasticsearchOperations 是一个接口,可以多种 实现!

在这里插入图片描述

使用springframework.data的情况(Transport), 不建议使用,请查看下一章.

怎么连es,使用第一种 方式

下面是Spring Boot的使用方法

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

spring对es地址及相关配置.
spring.data.elasticsearch.cluster-nodes=xxxx:9300,xxx:9301
spring.data.elasticsearch.cluster-name=b_cluster

怎么使用

直接引用即可

聚合,
//聚合操作
TermsAggregationBuilder tbuild = AggregationBuilders.terms("sa_groupid").field("said");
AggregationBuilder sab = AggregationBuilders.stats("a_stat").field("player_count");
tbuild.subAggregation(sab);
tbuild.size(saidlist.size());
SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withQuery(boolQueryBuilder).withIndices("aaa_online_log").withTypes("doc").withSearchType(SearchType.DEFAULT)
        .addAggregation(tbuild)
        .build();

Aggregations aggregations = elasticsearchTemplate.query(searchQuery, new ResultsExtractor<Aggregations>() {
    @Override
    public Aggregations extract(SearchResponse response) {
        return response.getAggregations();
    }
});

//聚合的内容获取!
MultiBucketsAggregation targetAggregation = aggregations.get("sa_groupid");
Map<Integer, DAggOnlineStat> sa_stat = new HashMap<>();
targetAggregation.getBuckets().forEach(tmpbuck ->{
    DAggOnlineStat dAggOnlineStat = new DAggOnlineStat();
    dAggOnlineStat.setApp_id(((Long)tmpbuck.getKey()).intValue());
    dAggOnlineStat.setDoc_count((int) tmpbuck.getDocCount());

    Aggregation tmpa = tmpbuck.getAggregations().getAsMap().get("a_stat");
    Stats tmpstats = (Stats) tmpa;

    dAggOnlineStat.setAvg(tmpstats.getAvg());
    dAggOnlineStat.setMax(((Double)tmpstats.getMax()).intValue());
    dAggOnlineStat.setMin(((Double)tmpstats.getMin()).intValue());
    sa_stat.put(dAggOnlineStat.getApp_id(), dAggOnlineStat);
});
return sa_stat;
查询单表
        String indexName = String.format("a_ooo_price_%s_log", currency_flag);

        BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
        boolQueryBuilder.must(new TermQueryBuilder("appid", appid));

        //按时间从早到晚排序
        SortBuilder sortBuilder = new FieldSortBuilder("current_time").order(SortOrder.ASC);

        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQueryBuilder).withIndices(indexName).withTypes("doc").withSort(sortBuilder)
                .build();

        SearchHits searchHits = elasticsearchTemplate.query(searchQuery, new ResultsExtractor<SearchHits>() {
            @Override
            public SearchHits extract(SearchResponse response) {
                return response.getHits();
            }
        });

        List<sfoogamePriceLog> sfoogamePriceLogs = new ArrayList<>();
        for (SearchHit searchHit : searchHits) {
            Map<String, Object> hitMap = searchHit.getSource();
            sfoogamePriceLog sfoogamePriceLog = new sfoogamePriceLog();
            sfoogamePriceLog.setId(searchHit.getId());
            sfoogamePriceLog.setDiscount_percent((Integer) hitMap.get("discount_percent"));
            sfoogamePriceLog.setUpdated((Long) hitMap.get("updated"));
            sfoogamePriceLog.setCdest((Integer) hitMap.get("cdest"));
            sfoogamePriceLog.setInitial((Integer) hitMap.get("initial"));
            sfoogamePriceLog.setCurrent_time((Long) hitMap.get("current_time"));
            sfoogamePriceLog.setCdate((String) hitMap.get("cdate"));
            sfoogamePriceLog.setAppid((Integer) hitMap.get("appid"));
            sfoogamePriceLog.setCurrency_flag((String) hitMap.get("currency_flag"));
            sfoogamePriceLogs.add(sfoogamePriceLog);
        }
        return sfoogamePriceLogs;
查询percent
PercentilesAggregationBuilder tbuild = AggregationBuilders.percentiles("b_sta").field("score");

BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
boolQueryBuilder.must(new TermQueryBuilder("language", "english"));

SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withQuery(boolQueryBuilder).withIndices("sfoogame_detail").withTypes("doc").withSearchType(SearchType.DEFAULT)
        .addAggregation(tbuild)
        .build();

Aggregations aggregations = elasticsearchTemplate.query(searchQuery, new ResultsExtractor<Aggregations>() {
    @Override
    public Aggregations extract(SearchResponse response) {
        return response.getAggregations();
    }
});

InternalTDigestPercentiles tmp = aggregations.get("b_sta");
Double percentile_val = tmp.percentile(50);
return percentile_val;

怎么连jest与使用,即9200端口.

官网:
https://github.com/VanRoy/spring-data-jest

com.github.vanroy spring-boot-starter-data-jest 3.1.5.RELEASE

官网使用xml配置

    <!-- ElasticSearch Jest Client -->
    <bean id="jestClient" factory-bean="jestClientFactory" factory-method="getObject" destroy-method="shutdownClient" />

    <bean id="jestClientConfigBuilder" class="io.searchbox.client.config.HttpClientConfig.Builder">
        <constructor-arg type="java.lang.String" value="http://localhost:9200"/>
    </bean>

    <bean id="jestClientConfig" factory-bean="jestClientConfigBuilder" factory-method="build"/>

    <bean id="jestClientFactory" class="io.searchbox.client.JestClientFactory">
        <property name="httpClientConfig" ref="jestClientConfig"/>
    </bean>

    <!-- Elasticsearch Jest Template -->
    <bean id="jestElasticsearchTemplate" class="com.github.vanroy.springdata.jest.JestElasticsearchTemplate">
        <constructor-arg ref="jestClient"/>
    </bean>

由于我们是使用springboot的,上面的配置可以不用,
我们使用注解的方式进行配置

    @Value("${project.es.serveruri}")
    public String serverUri ;

    @Bean
    public io.searchbox.client.config.HttpClientConfig.Builder jestClientConfigBuilder() {
        io.searchbox.client.config.HttpClientConfig.Builder builder = new io.searchbox.client.config.HttpClientConfig.Builder(serverUri);
        return builder;
    }
    @Bean
    public HttpClientConfig jestClientConfig(@Qualifier("jestClientConfigBuilder") io.searchbox.client.config.HttpClientConfig.Builder builder) {
        return builder.build();
    }

    @Bean
    public io.searchbox.client.JestClientFactory jestClientFactory(@Qualifier("jestClientConfig") HttpClientConfig httpClientConfig){
        io.searchbox.client.JestClientFactory factory = new io.searchbox.client.JestClientFactory();
        factory.setHttpClientConfig(httpClientConfig);
        return factory;
    }

    @Bean
    public JestClient jestClient(@Qualifier("jestClientFactory")JestClientFactory jestClientFactory) {
        return jestClientFactory.getObject();
    }


    @Bean
    public com.github.vanroy.springdata.jest.JestElasticsearchTemplate jestElasticsearchTemplate(@Qualifier("jestClient")JestClient jestClient){
        JestElasticsearchTemplate jestElasticsearchTemplate = new JestElasticsearchTemplate(jestClient);
        return jestElasticsearchTemplate;
    }

jest怎么使用?

@Autowired
private JestElasticsearchTemplate elasticsearchTemplate;

直接使用就可以了,但是我找了很久,没有找到靠谱的文档。关于使用模板的资料很少,只能自己摸索了。

使用template查单表数据
        String indexName = "aaa_index"

        BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
        boolQueryBuilder.must(new TermQueryBuilder("aid", aid));

        //按时间从早到晚排序
        SortBuilder sortBuilder = new FieldSortBuilder("current_time").order(SortOrder.ASC);

        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQueryBuilder).withIndices(indexName).withTypes("doc").withSort(sortBuilder)
                .build();

        List<SearchResult.Hit<AAA, Void>> searchHits = elasticsearchTemplate.query(searchQuery, new JestResultsExtractor<List<SearchResult.Hit<AAA, Void>>>() {
            @Override
            public List<SearchResult.Hit<AAA, Void>> extract(SearchResult searchResult) {
                return  searchResult.getHits(AAA.class);
            }
        });

        List<AAA> aaas = new ArrayList<>();
        for (SearchResult.Hit<AAA, Void> searchHit : searchHits) {
            AAA aaa = searchHit.source;

        }

AAA.class 是一个自定义的 pojo 类,用于保存内容。

查max
        BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
        TermsQueryBuilder queryBuilder1 = new TermsQueryBuilder("aid",String.valueOf(aid));

        boolQueryBuilder.must(queryBuilder1);

        MaxAggregationBuilder tbuild = AggregationBuilders.max("count_max").field("player_count");


        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQueryBuilder).withIndices("project_online_peak").withTypes("doc").withSearchType(SearchType.DEFAULT)
                .addAggregation(tbuild)
                .build();

        MetricAggregation aggregations = elasticsearchTemplate.query(searchQuery, new JestResultsExtractor<MetricAggregation>() {
            @Override
            public MetricAggregation extract(SearchResult searchResult) {
                return  searchResult.getAggregations();
            }
        });


        Double count_max_double = null;
        try {
//            count_max_double = ((InternalMax)aggregations.get("count_max")).value();
            count_max_double = aggregations.getMaxAggregation("count_max").getMax();
        } catch (Exception e) {
            log.error("count_max获取失败", e);
        }
        return  count_max_double.intValue();
查percentiles数据
PercentilesAggregationBuilder tbuild = AggregationBuilders.percentiles("b_sta").field("score");

BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
boolQueryBuilder.must(new TermQueryBuilder("language", "english"));

SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withQuery(boolQueryBuilder).withIndices("sfoogame_detail").withTypes("doc").withSearchType(SearchType.DEFAULT)
        .addAggregation(tbuild)
        .build();

MetricAggregation aggregations = elasticsearchTemplate.query(searchQuery, new JestResultsExtractor<MetricAggregation>() {
    @Override
    public MetricAggregation extract(SearchResult searchResult) {
        return  searchResult.getAggregations();
    }
});


//        InternalTDigestPercentiles tmp = aggregations.get("b_sta");
Map<String, Double> tmp = aggregations.getPercentilesAggregation("b_sta").getPercentiles();

Double targetVal = null;
for (String percentilestr : tmp.keySet()) {
    Double tdouble = new Double(percentilestr);
    if(75 == tdouble.intValue()){ // %50以上
        targetVal = tmp.get(percentilestr);
    }
}
terms聚合
QueryBuilder queryBuilder = QueryBuilders.boolQuery()
        .must(QueryBuilders.termQuery("cdatestr", "20170912"));
//聚合操作
TermsAggregationBuilder tbuild = AggregationBuilders.terms("sa_groupid").field("appid");
AggregationBuilder sab = AggregationBuilders.stats("a_stat").field("player_count");
tbuild.subAggregation(sab);
SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withQuery(queryBuilder).withIndices("sa_online_log").withTypes("doc").withSearchType(SearchType.DEFAULT)
        .addAggregation(tbuild)
        .build();



MetricAggregation aggregations = elasticsearchTemplate.query(searchQuery, new JestResultsExtractor<MetricAggregation>() {
    @Override
    public MetricAggregation extract(SearchResult response) {
        return  response.getAggregations();
    }
});

//聚合的内容获取!
List<TermsAggregation.Entry> tmpList = aggregations.getTermsAggregation("sa_groupid").getBuckets();
Map<Integer, DAggOnlineStat> sa_stat = new HashMap<>();
for (TermsAggregation.Entry entry : tmpList) {
    System.out.println(entry);
    DAggOnlineStat dAggOnStat = new DAggOnlineStat();
    dAggOnStat.setApp_id(Integer.valueOf(entry.getKey()));
    dAggOnStat.setDoc_count(entry.getCount().intValue());
    StatsAggregation statsAggregation = entry.getStatsAggregation("a_stat");
    dAggOnStat.setAvg(statsAggregation.getAvg());
    dAggOnStat.setMax(statsAggregation.getMax().intValue());
    dAggOnStat.setMin(statsAggregation.getMin().intValue());
    sa_stat.put(dAggOnStat.getApp_id(), dAggOnStat);
}
System.out.println("-----------");
System.out.println(sa_stat);

两种的一些使用差异.

在有些方法,会出现unsupportedOpertationException.对这些方法,我们要区别去改代码…【竟然不是在接口层去封装而是改业务代码,真是诡异】,
调这些方法会抛错

java.lang.UnsupportedOperationException
    at com.github.vanroy.springdata.jest.JestElasticsearchTemplate.query(JestElasticsearchTemplate.java:413)

现在看看,换个方法就可以了

参考:

https://help.aliyun.com/document_detail/69194.html?spm=a2c4g.11186623.6.552.45263075k2X9j5

如果要连aws,可参考:
https://blog.csdn.net/mingover/article/details/82984579

. . .

相关推荐

额外说明

SpringBoot+actuator和admin-UI实现监控中心

使用SpringBoot很久了,但是很少使用到SpringBoot的查看和监控,将来八成也不会用到,万一有机会用到呢?所以记录一下以前学习SpringBoot+actuator和adminUI实现监控中心的方式 Springboot的版本2.0.x <p

额外说明

DOM ------ 修改元素样式属性的两种方法

修改元素样式属性的两种方法 通过 this.style 修改 适用情况: 样式比较少 或 功能比较简单 //css代码 div{ width: 200px; height: 200px; backgr

额外说明

Direct3D 12——计算着色器——共享内存与线程同步

每个线程组都有一块称为共享内存(shared memory )或线程本地存储器(thread local storage )的内存空间。这种内存的访问速度很快,可认为与硬件高速缓存的速度不相上下。在我们的计算着色器的代 码中,共享内存的声明如下: gro

额外说明

Java基础 第五节 第八课

集合元素处理 传统方式 题目 解答 Stream 方式 题目 解答 传统方式 题目 现在有两个 ArrayList 集合存储队伍当中的多个成员姓名. 要求使用传统的 for 循环 (或增强 for 循环) 依次进行以下若干操作步骤: 第一个队伍只要名字为

额外说明

分布式高并发情况下全局Id生成策略

分布式高并发情况下全局Id生成策略 一、高并发情况下,生成分布式全局id策略需注意 注意幂等性且全局唯一性 注意安全性,不能被猜疑 趋势递增性 二、利用全球唯一UUID生成订单号 UUID基本概念: UUID是指在一台机器上生成的数字,它保证对在同一时空

额外说明

要想通过面试,MySQL的Limit子句底层原理你不可不知

文章目录 1. 老样子,建个表 2. 从sql执行计划看Limit的影响 3. 从server层和存储引擎层分析Limit执行过程 1. 老样子,建个表 还是这张表,表里我创建了近10W条数据 CREATE TABLE demo_info( id

额外说明

大数据学习笔记32:Hive - 下载、安装与配置

文章目录 一、初步了解Hive 1、Hadoop开发存在的问题 2、OLTP系统与OLAP系统 (1)OLTP - 在线联机事务处理系统 (2)OLAP - 在线联机分析处理系统 3、Hive概述 (1)Hive的SQL - HQL (2)数据库与数据仓

额外说明

Microsoft Dynamics 365 CE 扩展定制 - 3. SDK企业功能

在本章中,我们将介绍以下内容: 服务器端并发控制 客户端并发控制 在事务中执行请求 批处理请求 暂存数据导入 创建早期绑定的实体类 扩展CrmSvcUtil用Filter 扩展CrmSvcUtil以生成选项集枚举 使用CRM配置迁移工具跨实例迁移配置 简

额外说明

JS 双问号(??)和问号点(?.)的用法

1.双问号(??) 值1 ??值2 ??获取value1和value2之间的值,只有当value1为null或未定义时才取value2,否则取value1(0、false、“”认为有意义,所以仍然取value1) const obj = {} conso

ads via 小工具