MongoDB中文社区
java版本 : jdk 11
springboot版本 : 2.3.6.RELEASE
mongo驱动版本 : 4.0.5
mongo版本 ： 4.2.22（副本集）
出现问题的数据结构
图片上传不了，大致描述一下
{
"_id" : "A"
"aaa" : "aa" #类似字段有8个
"data" : {
"field1": [
"1",
"2"
],
"field2": "aaaa", # 类似字段有20个
"field3" : [
# 300个元素， 每个都有20个字段左右
]
}
}
查询时出现如下错误
mongoTemplate.findById(id, TestMongo.class);
2022-09-26 15:00:46.573 INFO 43720 --- [nio-8081-exec-1] org.mongodb.driver.connection : Opened connection [connectionId{localValue:19, serverValue:20}] to 192.168.10.61:27017
2022-09-26 15:00:47.620 INFO 43720 --- [nio-8081-exec-1] org.mongodb.driver.connection : Closed connection [connectionId{localValue:19, serverValue:20}] to 192.168.10.61:27017 because there was a socket exception raised by this connection.
2022-09-26 15:00:47.633 INFO 43720 --- [nio-8081-exec-1] org.mongodb.driver.connection : Opened connection [connectionId{localValue:20, serverValue:21}] to 192.168.10.61:27017
2022-09-26 15:00:47.641 WARN 43720 --- [nio-8081-exec-1] org.mongodb.driver.connection : Got socket exception on connection [connectionId{localValue:20, serverValue:21}] to 192.168.10.61:27017. All connections to 192.168.10.61:27017 will be closed.
2022-09-26 15:00:47.641 INFO 43720 --- [nio-8081-exec-1] org.mongodb.driver.connection : Closed connection [connectionId{localValue:20, serverValue:21}] to 192.168.10.61:27017 because there was a socket exception raised by this connection.
org.springframework.data.mongodb.UncategorizedMongoDbException: Exception receiving message; nested exception is com.mongodb.MongoSocketReadException: Exception receiving message
at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:133)
at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:2874)
at org.springframework.data.mongodb.core.MongoTemplate.executeFindOneInternal(MongoTemplate.java:2749)
at org.springframework.data.mongodb.core.MongoTemplate.doFindOne(MongoTemplate.java:2466)
at org.springframework.data.mongodb.core.MongoTemplate.doFindOne(MongoTemplate.java:2436)
at org.springframework.data.mongodb.core.MongoTemplate.findById(MongoTemplate.java:876)
at org.springframework.data.mongodb.core.MongoTemplate.findById(MongoTemplate.java:863)
at com.bowmann.ehs.activiti.web.controller.CustomController.ccc(CustomController.java:273)
at com.bowmann.ehs.activiti.web.controller.CustomController$$FastClassBySpringCGLIB$$617b82b1.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:62)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.aop.framework.adapter.AfterReturningAdviceInterceptor.invoke(AfterReturningAdviceInterceptor.java:55)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
at com.bowmann.ehs.activiti.web.controller.CustomController$$EnhancerBySpringCGLIB$$62b01b5e.ccc(<generated>)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:878)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:792)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:626)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.mongodb.MongoSocketReadException: Exception receiving message
at com.mongodb.internal.connection.InternalStreamConnection.translateReadException(InternalStreamConnection.java:568)
at com.mongodb.internal.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:447)
at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:298)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:258)
at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99)
at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:500)
at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:71)
at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:224)
at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:202)
at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:118)
at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:110)
at com.mongodb.internal.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:343)
at com.mongodb.internal.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:334)
at com.mongodb.internal.operation.CommandOperationHelper.access$000(CommandOperationHelper.java:67)
at com.mongodb.internal.operation.CommandOperationHelper$6.call(CommandOperationHelper.java:245)
at com.mongodb.internal.operation.OperationHelper.withReleasableConnection(OperationHelper.java:482)
at com.mongodb.internal.operation.CommandOperationHelper.executeCommandWithConnection(CommandOperationHelper.java:236)
at com.mongodb.internal.operation.FindOperation$1.call(FindOperation.java:631)
at com.mongodb.internal.operation.FindOperation$1.call(FindOperation.java:625)
at com.mongodb.internal.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:462)
at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:625)
at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:77)
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:190)
at com.mongodb.client.internal.FindIterableImpl.first(FindIterableImpl.java:189)
at org.springframework.data.mongodb.core.MongoTemplate$FindOneCallback.doInCollection(MongoTemplate.java:2917)
at org.springframework.data.mongodb.core.MongoTemplate$FindOneCallback.doInCollection(MongoTemplate.java:2888)
at org.springframework.data.mongodb.core.MongoTemplate.executeFindOneInternal(MongoTemplate.java:2746)
... 63 more
Caused by: java.net.SocketException: Connection reset
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
at com.mongodb.internal.connection.SocketStream.read(SocketStream.java:109)
at com.mongodb.internal.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:587)
at com.mongodb.internal.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:444)
... 88 more
2022-09-26 15:00:47.976 INFO 43720 --- [168.10.61:27017] org.mongodb.driver.cluster : Discovered replica set primary 192.168.10.61:27017
mongo的日志
2022-09-26T15:18:55.344+0800 I NETWORK [listener] connection accepted from 192.168.2.140:54167 #56 (24 connections now open)
2022-09-26T15:18:55.344+0800 I NETWORK [conn56] received client metadata from 192.168.2.140:54167 conn56: { driver: { name: "mongo-java-driver", version: "4.0.5" }, os: { type: "Windows", name: "Windows 10", architecture: "amd64", version: "10.0" }, platform: "Java/Oracle Corporation/11+28" }
2022-09-26T15:18:55.348+0800 I ACCESS [conn56] Successfully authenticated as principal fk on framework from client 192.168.2.140:54167
2022-09-26T15:18:55.362+0800 I NETWORK [conn56] Error sending response to client: HostUnreachable: Connection reset by peer. Ending connection from 192.168.2.140:54167 (connection id: 56)
2022-09-26T15:18:55.362+0800 I NETWORK [conn56] end connection 192.168.2.140:54167 (23 connections now open)
我的配置
spring:
data:
mongodb:
uri: mongodb://fk:Fk#123@192.168.10.61:27017,192.168.10.61:27018,192.168.10.61:27019/framework?authSource=framework&replicaSet=rs&ssl=false
option:
socket:
connectTimeout: 30000
readTimeout: 10000
pool:
maxPoolSize: 80
minPoolSize: 0
maxWaitTime: 5000
maxConnectionLifeTime: 600000
maxConnectionIdleTime: 30000
import com.mongodb.*;
import com.mongodb.client.MongoClient;
import com.mongodb.client.internal.MongoClientImpl;
import org.bson.UuidRepresentation;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.mongo.MongoProperties;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.mongodb.MongoDatabaseFactory;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.SimpleMongoClientDatabaseFactory;
import org.springframework.util.StringUtils;
import java.util.concurrent.TimeUnit;
/**
* @author jacob
* @date 2022/9/16 11:04
*/
@Configuration
@EnableConfigurationProperties({MongoOptionProperty.class, MongoProperties.class})
@ConditionalOnClass(MongoClient.class)
public class MongoConfiguration {
@Bean
public MongoClientSettings mongoClientSettings(MongoOptionProperty option, MongoProperties properties) {
if (null == properties || StringUtils.isEmpty(properties.getUri())) {
return MongoClientSettings.builder().build();
}
ConnectionString connectionString = new ConnectionString(properties.getUri());
option.setDatabase(connectionString.getDatabase());
return MongoClientSettings.builder()
.applyConnectionString(connectionString)
.applyToClusterSettings(settings -> {
settings.applyConnectionString(connectionString);
settings.requiredClusterType(option.getCluster().getClusterType());
settings.mode(option.getCluster().getClusterConnectionMode());
settings.serverSelectionTimeout(option.getCluster().getServerSelectionTimeout(), TimeUnit.MILLISECONDS);
})
.applyToSocketSettings(settings -> {
settings.applyConnectionString(connectionString);
settings.connectTimeout(option.getSocket().getConnectTimeout(), TimeUnit.MILLISECONDS);
settings.readTimeout(option.getSocket().getReadTimeout(), TimeUnit.MILLISECONDS);
settings.sendBufferSize(20480000);
settings.receiveBufferSize(20480000);
})
.applyToConnectionPoolSettings(settings -> {
settings.applyConnectionString(connectionString);
settings.maxSize(option.getPool().getMaxPoolSize());
settings.minSize(option.getPool().getMinPoolSize());
settings.maxWaitTime(option.getPool().getMaxWaitTime(), TimeUnit.MILLISECONDS);
settings.maxConnectionLifeTime(option.getPool().getMaxConnectionLifeTime(), TimeUnit.MILLISECONDS);
settings.maxConnectionIdleTime(option.getPool().getMaxConnectionIdleTime(), TimeUnit.MILLISECONDS);
// settings.maintenanceInitialDelay(200, TimeUnit.MILLISECONDS);
settings.maintenanceFrequency(60000, TimeUnit.MILLISECONDS);
})
.applyToServerSettings(settings -> {
settings.applyConnectionString(connectionString);
settings.heartbeatFrequency(option.getServer().getHeartbeatFrequency(), TimeUnit.MILLISECONDS);
settings.minHeartbeatFrequency(option.getServer().getMinHeartbeatFrequency(), TimeUnit.MILLISECONDS);
})
.applyToSslSettings(settings -> {
settings.applyConnectionString(connectionString);
settings.enabled(option.getSsl().isSslEnabled());
settings.invalidHostNameAllowed(option.getSsl().isInvalidHostNameAllowed());
})
.readPreference(ReadPreference.primary())
// 设置写关注
.writeConcern(WriteConcern.ACKNOWLEDGED)
.readConcern(ReadConcern.MAJORITY)
// 设置由于网络错误而失败时是否应重试写入
.retryWrites(true)
// 设置由于网络错误而失败时是否应重试读取
.retryReads(true)
.uuidRepresentation(UuidRepresentation.JAVA_LEGACY)
.build();
}
@Bean
public MongoClient mongoClient(MongoClientSettings settings){
return new MongoClientImpl(settings, null);
}
@Bean
public MongoDatabaseFactory mongoDatabaseFactory(MongoClient mongoClient, MongoOptionProperty property){
return new SimpleMongoClientDatabaseFactory(mongoClient, property.getDatabase());
}
@Bean
public MongoTemplate mongoTemplate(MongoDatabaseFactory factory){
MongoTemplate mongoTemplate = new MongoTemplate(factory);
mongoTemplate.setReadPreference(ReadPreference.primaryPreferred());
return mongoTemplate;
}
}
keepalive_time也已设置
net.ipv4.tcp_keepalive_time = 120
如果数据量小可以查询出来（如集合中只有10个元素），但是查询全部也会出错（每个文档的数据量都很小10个字段左右，数量大概是200条；查询170条会小概率报错，查询200条必报错，错误信息和上面一样）；
但是，有一种情况可以查询出来，比如说我随便插入一条数据，控制台打印
2022-09-26 15:22:56.044 INFO 43720 --- [nio-8081-exec-8] org.mongodb.driver.connection : Opened connection [connectionId{localValue:39, serverValue:59}] to 192.168.10.61:27017
连接成功后，然后我再进行查询，就可以正常查询出这条数据。
mongo的错误信息很模糊，我都不知道从哪个地方排错，只知道连接断开了，但是其它数据可以正常查询出来（就是上面说的数据很小，数据里面的集合只有10来个），太离谱了
xiaoxu 已回答的问题
空闲连接会断开的。另外tcp.keepalive官方是300s.你改试试。
另外链接池线先去掉看看，你多少应用，多少并发？
数据库日志里面有没有其他错误。
<span class="lit">2022</span><span class="pun">-</span><span class="lit">09</span><span class="pun">-</span><span class="lit">26T15</span><span class="pun">:</span><span class="lit">18</span><span class="pun">:</span><span class="lit">55.362</span><span class="pun">+</span><span class="lit">0800</span><span class="pln"> I NETWORK </span><span class="pun">[</span><span class="pln">conn56</span><span class="pun">]</span> <span class="typ">Error</span><span class="pln"> sending response to client</span><span class="pun">:</span> <span class="typ">HostUnreachable</span><span class="pun">:</span> <span class="typ">Connection</span><span class="pln"> reset </span><span class="kwd">by</span><span class="pln"> peer</span><span class="pun">.</span> <span class="typ">Ending</span><span class="pln"> connection </span><span class="kwd">from</span> <span class="lit">192.168</span><span class="pun">.</span><span class="lit">2.140</span><span class="pun">:</span><span class="lit">54167</span> <span class="pun">(</span><span class="pln">connection id</span><span class="pun">:</span> <span class="lit">56</span><span class="pun">)</span> 这个错误是客户端不可答。猜测tcp连接断开了。你先改keepalive看下
<span class="pln"> option</span><span class="pun">:</span><span class="pln">
socket</span><span class="pun">:</span><span class="pln">
connectTimeout</span><span class="pun">:</span> <span class="lit">30000</span><span class="pln">
readTimeout</span><span class="pun">:</span> <span class="lit">10000</span><span class="pln">
pool</span><span class="pun">:</span><span class="pln">
maxPoolSize</span><span class="pun">:</span> <span class="lit">80</span><span class="pln">
minPoolSize</span><span class="pun">:</span> <span class="lit">0</span><span class="pln">
maxWaitTime</span><span class="pun">:</span> <span class="lit">5000</span><span class="pln">
maxConnectionLifeTime</span><span class="pun">:</span> <span class="lit">600000</span><span class="pln">
maxConnectionIdleTime</span><span class="pun">:</span> <span class="lit">30000</span>
xiaoxu 编辑答案
