刚接触mongo,希望mongo大佬解惑,感谢。
需求
数据结构中有有个数组`malIps`,这个数组中有个字段`attack`,想对malIps这个数组进行分页,并且不返回`attack`这个数组对象。
mongo中存储的数据结构:
<code class="hljs language-javascript">{ <span class="hljs-string">"_id"</span> : ObjectId(<span class="hljs-string">"60ab1cd22119d970838a7db8"</span>), <span class="hljs-string">"attackTypeAmount"</span> : <span class="hljs-number">7</span>, <span class="hljs-string">"logTotal"</span> : <span class="hljs-number">48500</span>, <span class="hljs-string">"attackTotal"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"attackTypeTotal"</span> : { <span class="hljs-string">"scanrobots"</span> : <span class="hljs-number">330</span>, <span class="hljs-string">"FileInclude"</span> : <span class="hljs-number">4020</span>, <span class="hljs-string">"Javaouinstream"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Sensitivefile"</span> : <span class="hljs-number">4785</span>, <span class="hljs-string">"SQLInjection"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Webshell"</span> : <span class="hljs-number">12</span>, <span class="hljs-string">"XSS"</span> : <span class="hljs-number">599</span> }, <span class="hljs-string">"malIpTotal"</span> : <span class="hljs-number">3</span>, <span class="hljs-string">"malIps"</span> : [ { <span class="hljs-string">"attackCounts"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"telecomOperator"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"finishTime"</span> : <span class="hljs-string">"2020-12-03 16:05:46"</span>, <span class="hljs-string">"isMalIp"</span> : <span class="hljs-literal">false</span>, <span class="hljs-string">"attackTypeAmount"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"ip"</span> : <span class="hljs-string">"192.168.0.112"</span>, <span class="hljs-string">"latitude"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"location"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"startTime"</span> : <span class="hljs-string">"2020-12-02 13:55:59"</span>, <span class="hljs-string">"attackTypeTotal"</span> : { <span class="hljs-string">"scanrobots"</span> : <span class="hljs-number">330</span>, <span class="hljs-string">"FileInclude"</span> : <span class="hljs-number">4020</span>, <span class="hljs-string">"Javaouinstream"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Sensitivefile"</span> : <span class="hljs-number">4785</span>, <span class="hljs-string">"SQLInjection"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Webshell"</span> : <span class="hljs-number">12</span>, <span class="hljs-string">"XSS"</span> : <span class="hljs-number">599</span> }, <span class="hljs-string">"longitude"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"attack"</span> : [ { <span class="hljs-string">"dateTime"</span> : <span class="hljs-string">"2020-12-03 16:05:46"</span>, <span class="hljs-string">"lineNo"</span> : <span class="hljs-number">48224</span>, <span class="hljs-string">"log"</span> : <span class="hljs-string">"192.168.0.112 - - [03/Dec/2020:16:05:46 0800] \"GET /noindex/common/css//../../../WEB-INF/web.xml\u0000.jsp HTTP/1.1\" 404 242 \"-\" \"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21\""</span>, <span class="hljs-string">"filePath"</span> : <span class="hljs-string">"F:\\test\\logs\\apache\\access_log"</span>, <span class="hljs-string">"type"</span> : <span class="hljs-string">"Sensitivefile"</span>, <span class="hljs-string">"status"</span> : <span class="hljs-string">"404"</span> }, { <span class="hljs-string">"dateTime"</span> : <span class="hljs-string">"2020-12-03 16:05:46"</span>, <span class="hljs-string">"lineNo"</span> : <span class="hljs-number">48272</span>, <span class="hljs-string">"log"</span> : <span class="hljs-string">"192.168.0.112 - - [03/Dec/2020:16:05:46 0800] \"GET /noindex/common/fonts/documents HTTP/1.1\" 404 228 \"-\" \"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21\""</span>, <span class="hljs-string">"filePath"</span> : <span class="hljs-string">"F:\\test\\logs\\apache\\access_log"</span>, <span class="hljs-string">"type"</span> : <span class="hljs-string">"XSS"</span>, <span class="hljs-string">"status"</span> : <span class="hljs-string">"404"</span> } ] }, { <span class="hljs-string">"attackCounts"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"telecomOperator"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"finishTime"</span> : <span class="hljs-string">"2020-12-03 16:05:46"</span>, <span class="hljs-string">"isMalIp"</span> : <span class="hljs-literal">false</span>, <span class="hljs-string">"attackTypeAmount"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"ip"</span> : <span class="hljs-string">"192.168.0.113"</span>, <span class="hljs-string">"latitude"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"location"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"startTime"</span> : <span class="hljs-string">"2020-12-02 13:55:59"</span>, <span class="hljs-string">"attackTypeTotal"</span> : { <span class="hljs-string">"scanrobots"</span> : <span class="hljs-number">330</span>, <span class="hljs-string">"FileInclude"</span> : <span class="hljs-number">4020</span>, <span class="hljs-string">"Javaouinstream"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Sensitivefile"</span> : <span class="hljs-number">4785</span>, <span class="hljs-string">"SQLInjection"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Webshell"</span> : <span class="hljs-number">12</span>, <span class="hljs-string">"XSS"</span> : <span class="hljs-number">599</span> }, <span class="hljs-string">"longitude"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"attack"</span> : [ { <span class="hljs-string">"dateTime"</span> : <span class="hljs-string">"2020-12-03 16:05:46"</span>, <span class="hljs-string">"lineNo"</span> : <span class="hljs-number">48224</span>, <span class="hljs-string">"log"</span> : <span class="hljs-string">"192.168.0.113 - - [03/Dec/2020:16:05:46 0800] \"GET /noindex/common/css//../../../WEB-INF/web.xml\u0000.jsp HTTP/1.1\" 404 242 \"-\" \"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21\""</span>, <span class="hljs-string">"filePath"</span> : <span class="hljs-string">"F:\\test\\logs\\apache\\access_log"</span>, <span class="hljs-string">"type"</span> : <span class="hljs-string">"Sensitivefile"</span>, <span class="hljs-string">"status"</span> : <span class="hljs-string">"404"</span> }, { <span class="hljs-string">"dateTime"</span> : <span class="hljs-string">"2020-12-03 16:05:46"</span>, <span class="hljs-string">"lineNo"</span> : <span class="hljs-number">48300</span>, <span class="hljs-string">"log"</span> : <span class="hljs-string">"192.168.0.113 - - [03/Dec/2020:16:05:46 0800] \"GET /noindex/common/images/documents HTTP/1.1\" 404 229 \"-\" \"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21\""</span>, <span class="hljs-string">"filePath"</span> : <span class="hljs-string">"F:\\test\\logs\\apache\\access_log"</span>, <span class="hljs-string">"type"</span> : <span class="hljs-string">"XSS"</span>, <span class="hljs-string">"status"</span> : <span class="hljs-string">"404"</span> } ] }, { <span class="hljs-string">"attackCounts"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"telecomOperator"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"finishTime"</span> : <span class="hljs-string">"2020-12-03 16:05:46"</span>, <span class="hljs-string">"isMalIp"</span> : <span class="hljs-literal">false</span>, <span class="hljs-string">"attackTypeAmount"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"ip"</span> : <span class="hljs-string">"192.168.0.114"</span>, <span class="hljs-string">"latitude"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"location"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"startTime"</span> : <span class="hljs-string">"2020-12-02 13:55:59"</span>, <span class="hljs-string">"attackTypeTotal"</span> : { <span class="hljs-string">"scanrobots"</span> : <span class="hljs-number">330</span>, <span class="hljs-string">"FileInclude"</span> : <span class="hljs-number">4020</span>, <span class="hljs-string">"Javaouinstream"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Sensitivefile"</span> : <span class="hljs-number">4785</span>, <span class="hljs-string">"SQLInjection"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Webshell"</span> : <span class="hljs-number">12</span>, <span class="hljs-string">"XSS"</span> : <span class="hljs-number">599</span> }, <span class="hljs-string">"longitude"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"attack"</span> : [ ] } ] }</code>
想得到的数据结构,得到`malIps`的前两条数据,不要`attack`字段,如下:
<code class="hljs language-javascript">{ <span class="hljs-string">"_id"</span> : ObjectId(<span class="hljs-string">"60ab1cd22119d970838a7db8"</span>), <span class="hljs-string">"attackTypeAmount"</span> : <span class="hljs-number">7</span>, <span class="hljs-string">"logTotal"</span> : <span class="hljs-number">48500</span>, <span class="hljs-string">"attackTotal"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"attackTypeTotal"</span> : { <span class="hljs-string">"scanrobots"</span> : <span class="hljs-number">330</span>, <span class="hljs-string">"FileInclude"</span> : <span class="hljs-number">4020</span>, <span class="hljs-string">"Javaouinstream"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Sensitivefile"</span> : <span class="hljs-number">4785</span>, <span class="hljs-string">"SQLInjection"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Webshell"</span> : <span class="hljs-number">12</span>, <span class="hljs-string">"XSS"</span> : <span class="hljs-number">599</span> }, <span class="hljs-string">"malIpTotal"</span> : <span class="hljs-number">3</span>, <span class="hljs-string">"malIps"</span> : [ { <span class="hljs-string">"attackCounts"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"telecomOperator"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"finishTime"</span> : <span class="hljs-string">"2020-12-03 16:05:46"</span>, <span class="hljs-string">"isMalIp"</span> : <span class="hljs-literal">false</span>, <span class="hljs-string">"attackTypeAmount"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"ip"</span> : <span class="hljs-string">"192.168.0.112"</span>, <span class="hljs-string">"latitude"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"location"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"startTime"</span> : <span class="hljs-string">"2020-12-02 13:55:59"</span>, <span class="hljs-string">"attackTypeTotal"</span> : { <span class="hljs-string">"scanrobots"</span> : <span class="hljs-number">330</span>, <span class="hljs-string">"FileInclude"</span> : <span class="hljs-number">4020</span>, <span class="hljs-string">"Javaouinstream"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Sensitivefile"</span> : <span class="hljs-number">4785</span>, <span class="hljs-string">"SQLInjection"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Webshell"</span> : <span class="hljs-number">12</span>, <span class="hljs-string">"XSS"</span> : <span class="hljs-number">599</span> }, <span class="hljs-string">"longitude"</span> : <span class="hljs-string">""</span> }, { <span class="hljs-string">"attackCounts"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"telecomOperator"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"finishTime"</span> : <span class="hljs-string">"2020-12-03 16:05:46"</span>, <span class="hljs-string">"isMalIp"</span> : <span class="hljs-literal">false</span>, <span class="hljs-string">"attackTypeAmount"</span> : <span class="hljs-number">9758</span>, <span class="hljs-string">"ip"</span> : <span class="hljs-string">"192.168.0.113"</span>, <span class="hljs-string">"latitude"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"location"</span> : <span class="hljs-string">""</span>, <span class="hljs-string">"startTime"</span> : <span class="hljs-string">"2020-12-02 13:55:59"</span>, <span class="hljs-string">"attackTypeTotal"</span> : { <span class="hljs-string">"scanrobots"</span> : <span class="hljs-number">330</span>, <span class="hljs-string">"FileInclude"</span> : <span class="hljs-number">4020</span>, <span class="hljs-string">"Javaouinstream"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Sensitivefile"</span> : <span class="hljs-number">4785</span>, <span class="hljs-string">"SQLInjection"</span> : <span class="hljs-number">6</span>, <span class="hljs-string">"Webshell"</span> : <span class="hljs-number">12</span>, <span class="hljs-string">"XSS"</span> : <span class="hljs-number">599</span> }, <span class="hljs-string">"longitude"</span> : <span class="hljs-string">""</span> } ] }</code>
测试用数据插入:
<code class="hljs">insert({"attackTypeAmount":7,"logTotal":48500,"attackTotal":9758,"attackTypeTotal":{"scanrobots":330,"FileInclude":4020,"Javaouinstream":6,"Sensitivefile":4785,"SQLInjection":6,"Webshell":12,"XSS":599},"malIpTotal":3,"malIps":[{"attackCounts":9758,"telecomOperator":"","finishTime":"2020-12-03 16:05:46","isMalIp":false,"attackTypeAmount":9758,"ip":"192.168.0.112","latitude":"","location":"","startTime":"2020-12-02 13:55:59","attackTypeTotal":{"scanrobots":330,"FileInclude":4020,"Javaouinstream":6,"Sensitivefile":4785,"SQLInjection":6,"Webshell":12,"XSS":599},"longitude":"","attack":[{"dateTime":"2020-12-03 16:05:46","lineNo":48224,"log":"192.168.0.112 - - [03/Dec/2020:16:05:46 0800] \"GET /noindex/common/css//../../../WEB-INF/web.xml\u0000.jsp HTTP/1.1\" 404 242 \"-\" \"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21\"","filePath":"F:\\test\\logs\\apache\\access_log","type":"Sensitivefile","status":"404"},{"dateTime":"2020-12-03 16:05:46","lineNo":48272,"log":"192.168.0.112 - - [03/Dec/2020:16:05:46 0800] \"GET /noindex/common/fonts/documents HTTP/1.1\" 404 228 \"-\" \"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21\"","filePath":"F:\\test\\logs\\apache\\access_log","type":"XSS","status":"404"}]},{"attackCounts":9758,"telecomOperator":"","finishTime":"2020-12-03 16:05:46","isMalIp":false,"attackTypeAmount":9758,"ip":"192.168.0.113","latitude":"","location":"","startTime":"2020-12-02 13:55:59","attackTypeTotal":{"scanrobots":330,"FileInclude":4020,"Javaouinstream":6,"Sensitivefile":4785,"SQLInjection":6,"Webshell":12,"XSS":599},"longitude":"","attack":[{"dateTime":"2020-12-03 16:05:46","lineNo":48224,"log":"192.168.0.113 - - [03/Dec/2020:16:05:46 0800] \"GET /noindex/common/css//../../../WEB-INF/web.xml\u0000.jsp HTTP/1.1\" 404 242 \"-\" \"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21\"","filePath":"F:\\test\\logs\\apache\\access_log","type":"Sensitivefile","status":"404"},{"dateTime":"2020-12-03 16:05:46","lineNo":48300,"log":"192.168.0.113 - - [03/Dec/2020:16:05:46 0800] \"GET /noindex/common/images/documents HTTP/1.1\" 404 229 \"-\" \"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21\"","filePath":"F:\\test\\logs\\apache\\access_log","type":"XSS","status":"404"}]},{"attackCounts":9758,"telecomOperator":"","finishTime":"2020-12-03 16:05:46","isMalIp":false,"attackTypeAmount":9758,"ip":"192.168.0.114","latitude":"","location":"","startTime":"2020-12-02 13:55:59","attackTypeTotal":{"scanrobots":330,"FileInclude":4020,"Javaouinstream":6,"Sensitivefile":4785,"SQLInjection":6,"Webshell":12,"XSS":599},"longitude":"","attack":[]}]});</code>
查了官方文档,find 和 slice好像实现不了,需要用aggregate()
并使用两个单独的$project
可是官方没有对应的例子,咋都凑不出来。
https://docs.mongodb.com/manual/reference/operator/projection/slice/#examples
> db.aa.aggregate([{$project:{attackTypeAmount:1,logTotal:1,attackTotal:1,malIps: { $slice: [ “$malIps”, 2 ] }}},{$project:{“malIps.attack”:0}}]).pretty();
{
“_id” : ObjectId(“60ac678fe50c691e5e89e7ca”),
“attackTypeAmount” : 7,
“logTotal” : 48500,
“attackTotal” : 9758,
“malIps” : [
{
“attackCounts” : 9758,
“telecomOperator” : “”,
“finishTime” : “2020-12-03 16:05:46”,
“isMalIp” : false,
“attackTypeAmount” : 9758,
“ip” : “192.168.0.112”,
“latitude” : “”,
“location” : “”,
“startTime” : “2020-12-02 13:55:59”,
“attackTypeTotal” : {
“scanrobots” : 330,
“FileInclude” : 4020,
“Javaouinstream” : 6,
“Sensitivefile” : 4785,
“SQLInjection” : 6,
“Webshell” : 12,
“XSS” : 599
},
“longitude” : “”
},
{
“attackCounts” : 9758,
“telecomOperator” : “”,
“finishTime” : “2020-12-03 16:05:46”,
“isMalIp” : false,
“attackTypeAmount” : 9758,
“ip” : “192.168.0.113”,
“latitude” : “”,
“location” : “”,
“startTime” : “2020-12-02 13:55:59”,
“attackTypeTotal” : {
“scanrobots” : 330,
“FileInclude” : 4020,
“Javaouinstream” : 6,
“Sensitivefile” : 4785,
“SQLInjection” : 6,
“Webshell” : 12,
“XSS” : 599
},
“longitude” : “”
}
]
}
非常感谢回答
不客气的,以后可以多交流。
使用find语法更简单些
> db.aa.find({},{malIps:{$slice:2},”malIps.attack”:0}).pretty();
{
“_id” : ObjectId(“60ac678fe50c691e5e89e7ca”),
“attackTypeAmount” : 7,
“logTotal” : 48500,
“attackTotal” : 9758,
“attackTypeTotal” : {
“scanrobots” : 330,
“FileInclude” : 4020,
“Javaouinstream” : 6,
“Sensitivefile” : 4785,
“SQLInjection” : 6,
“Webshell” : 12,
“XSS” : 599
},
“malIpTotal” : 3,
“malIps” : [
{
“attackCounts” : 9758,
“telecomOperator” : “”,
“finishTime” : “2020-12-03 16:05:46”,
“isMalIp” : false,
“attackTypeAmount” : 9758,
“ip” : “192.168.0.112”,
“latitude” : “”,
“location” : “”,
“startTime” : “2020-12-02 13:55:59”,
“attackTypeTotal” : {
“scanrobots” : 330,
“FileInclude” : 4020,
“Javaouinstream” : 6,
“Sensitivefile” : 4785,
“SQLInjection” : 6,
“Webshell” : 12,
“XSS” : 599
},
“longitude” : “”
},
{
“attackCounts” : 9758,
“telecomOperator” : “”,
“finishTime” : “2020-12-03 16:05:46”,
“isMalIp” : false,
“attackTypeAmount” : 9758,
“ip” : “192.168.0.113”,
“latitude” : “”,
“location” : “”,
“startTime” : “2020-12-02 13:55:59”,
“attackTypeTotal” : {
“scanrobots” : 330,
“FileInclude” : 4020,
“Javaouinstream” : 6,
“Sensitivefile” : 4785,
“SQLInjection” : 6,
“Webshell” : 12,
“XSS” : 599
},
“longitude” : “”
}
]
}
确实需要使用aggregate并且结合两次project:结合您的答案,可以通过如下方式获取:
aggregate([{$project:{“malIps”:{$slice:[“$malIps”,0,1]}}},{$project:{“malIps.attack”:0}}]).pretty();
感谢您的支持
确实需要使用aggregate,结合两次project投影
aggregate([{$project:{“malIps”:{$slice:[“$malIps”,0,1]}}},{$project:{“malIps.attack”:0}}]).pretty();