MongoDB爱好者
垂直技术交流平台

MongoDB索引怎么失效了|索引案例一则

某日公司研发同事突然反馈MongoDB的某个分片集合按照分片键无法检索到对应内容,但是按照_id却可以检索到对应内容,一瓜未平,一瓜又起,另一个研发同事反馈该集合唯一索引失效了,存在重复数据,这简直是比对面老王有女朋友还让我震惊的事情。接下来我会给大家复现这个现象并说明具体的原理。

问题现象模拟

因为涉及到公司的一些信息安全政策故对集合的字段名进行替换,数据里面敏感信息进行值替换,但是不影响最终结果

MongoDB版本:  mongos 4.0.12  社区版本
数据库:mdb_test
哈希分片集合:mdb_test_coll
分片键:key3

集合索引信息如下,key3是分片键 ,还有一个联合唯一索引key3_1_key1_1_key2_1_key4_1。

db.mdb_test_coll.getIndexes()
[
    {
        "v" : 2,
        "key" : {
            "_id" : 1
        },
        "name" : "_id_",
        "ns" : "mdb_test.mdb_test_coll"
    },
    {
        "v" : 2,
        "key" : {
            "key3" : "hashed"
        },
        "name" : "key3_hashed",
        "ns" : "mdb_test.mdb_test_coll"
    },
    {
        "v" : 2,
        "key" : {
            "key2" : 1
        },
        "name" : "key2_1",
        "background" : true,
        "ns" : "mdb_test.mdb_test_coll"
    },
    {
        "v" : 2,
        "unique" : true,
        "key" : {
            "key3" : 1,
            "key1" : 1,
            "key2" : 1,
            "key4" : 1
        },
        "name" : "key3_1_key1_1_key2_1_key4_1",
        "background" : true,
        "ns" : "mdb_test.mdb_test_coll"
    }
]

模拟数据 

注:在使用下面数据的时候建议将每一条都调整到一行进行插入,每条记录的_id是不一样的

{
    "_id" : ObjectId("606ed113137fa535ac53d3ee"),
    "key1" : null,
    "key2" : 1033072826,
    "key3" : BinData(5,"N0YSB7m2iXUEzQnFsTQWPA=="),
    "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl"
}
{
    "_id" : ObjectId("5fb2df3b32b0f42dced04ea7"),
    "key1" : null,
    "key2" : 2100000000,
    "key3" : BinData(5,"AnQTYWNGHKoj4xx+KTjNxQ=="),
    "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl"
}
{
    "_id" : ObjectId("5fb2df3b32b0f42dced01127"),
    "key1" : null,
    "key2" : 2100000000,
    "key3" : BinData(5,"AnQTYWNGHKoj4xx+KTjNxQ=="),
    "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl"
}
{
    "_id" : ObjectId("5fb2df3b32b0f42dced01130"),
    "key1" : null,
    "key2" : 2100000000,
    "key3" : BinData(5,"AnQTYWNGHKoj4xx+KTjNxQ=="),
    "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl"
}

操作步骤 

问题现象一: 先模拟研发反馈的无法通过分片键查询到数据,但是可以根据_id查询到数据。将模拟数据第一条插入集合中。

mongos> db.mdb_test_coll.find()
mongos> db.mdb_test_coll.save({
... "_id" : ObjectId("606ed113137fa535ac53d3ee"),
... "key1" : null,
... "key2" : 1033072826,
... "key3" : BinData(5,"N0YSB7m2iXUEzQnFsTQWPA=="),
... "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl"
... })
WriteResult({
    "nMatched" : 0,
    "nUpserted" : 1,
    "nModified" : 0,
    "_id" : ObjectId("606ed113137fa535ac53d3ee")
})
mongos> db.mdb_test_coll.find()
{ "_id" : ObjectId("606ed113137fa535ac53d3ee"), "key1" : null, "key2" : 1033072826, "key3" : BinData(5,"N0YSB7m2iXUEzQnFsTQWPA=="), "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl" }

#按照_id查询可以得到数据,但是将条件换成分片键或者带分片键条件的联合查询条件都无法获得任何结果。

mongos> db.mdb_test_coll.find({"_id" : ObjectId("606ed113137fa535ac53d3ee")})
{ "_id" : ObjectId("606ed113137fa535ac53d3ee"), "key1" : null, "key2" : 1033072826, "key3" : BinData(5,"N0YSB7m2iXUEzQnFsTQWPA=="), "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl" }
mongos> db.mdb_test_coll.find({"key3" : BinData(5,"N0YSB7m2iXUEzQnFsTQWPA=="),"key2" : 1033072826})
mongos> db.mdb_test_coll.find({"key3" : BinData(5,"N0YSB7m2iXUEzQnFsTQWPA==")})
mongos> db.mdb_test_coll.find({"key2" : 1033072826,"key3" : BinData(5,"N0YSB7m2iXUEzQnFsTQWPA==")})

问题现象二,再来模拟一下研发反馈的唯一索引失效的现象。将模拟数据第二,三条插入集合中,请注意模拟数据二三两条除_id不一致,其余字段值都是相同的

mongos> db.mdb_test_coll.save({
... "_id" : ObjectId("5fb2df3b32b0f42dced04ea7"),
... "key1" : null,
... "key2" : 2100000000,
... "key3" : BinData(5,"AnQTYWNGHKoj4xx+KTjNxQ=="),
... "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl"
... })
WriteResult({
    "nMatched" : 0,
    "nUpserted" : 1,
    "nModified" : 0,
    "_id" : ObjectId("5fb2df3b32b0f42dced04ea7")
})
mongos> db.mdb_test_coll.save({
... "_id" : ObjectId("5fb2df3b32b0f42dced01127"),
... "key1" : null,
... "key2" : 2100000000,
... "key3" : BinData(5,"AnQTYWNGHKoj4xx+KTjNxQ=="),
... "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl"
... })
WriteResult({
    "nMatched" : 0,
    "nUpserted" : 1,
    "nModified" : 0,
    "_id" : ObjectId("5fb2df3b32b0f42dced01127")
})

mongos> db.mdb_test_coll.find().pretty()
{
    "_id" : ObjectId("606ed113137fa535ac53d3ee"),
    "key1" : null,
    "key2" : 1033072826,
    "key3" : BinData(5,"N0YSB7m2iXUEzQnFsTQWPA=="),
    "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl"
}
{
    "_id" : ObjectId("5fb2df3b32b0f42dced04ea7"),
    "key1" : null,
    "key2" : 2100000000,
    "key3" : BinData(5,"AnQTYWNGHKoj4xx+KTjNxQ=="),
    "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl"
}
{
    "_id" : ObjectId("5fb2df3b32b0f42dced01127"),
    "key1" : null,
    "key2" : 2100000000,
    "key3" : BinData(5,"AnQTYWNGHKoj4xx+KTjNxQ=="),
    "key4" : "hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hls/hl"
}

可以看见连续插入两条唯一字段值相同的记录,并且并没有触发唯一约束的,并且值是真实插入到集合中了。

原理分析 

先来说一下问题现象二的根本原因,之所以可以插入重复数据而没有报错是因为设置了failIndexKeyTooLong,先来看一下官方文档中failIndexKeyTooLong这个参数定义:

简单总结就是这个参数仅对mongod节点有效,并且这个参数主要作用就是如果当索引条目超过Index Key Length Limit 后的动作。默认为True也就是如果索引条目超过1024字节,报错给客户端。也可以设置为false,如果设置为False 超过1024字节的数据是可以插入到集合里面,但是不会在对应的索引里面,所以在查询的时候如果使用了对应索引,该索引并不会返回完整的数据给客户端(这块下面详细解释一下),这个参数是可以动态调整的。

动态调整语句如下:

db.adminCommand( { setParameter: 1, failIndexKeyTooLong: false } )

这里再解释一下什么是Index Key Limit, 也就是上面提到的Index Key Length Limit。看一下官方文档对这个参数的定义:一句概括就是索引字节必须小于1024字节。

举个🌰

failIndexKeyTooLong设置为true,然后插入数据。可以看见当设置为true的时候,插入模拟数据第四条是报错的。因为有一个唯一联合索引就是这四个字段组成的。

再来看一下failIndexKeyTooLong设置为false,当设置为false的时候是可以插入模拟数据第四条,并没有报错,而且也没有触发唯一约束,但是按照key3或者key3+key2联合条件查询不到数据。但是按照 id或者key2(最后插入的数据也是除了id不同,别的字段值都相同)单独去匹配可以获得数据。

实际上到这里已经解释清了研发反馈的所有问题,在failIndexKeyTooLong设置为false的情况下,按照key3以及key3+key2联合去检索的时候实际使用的都是联合唯一索引,但是实际上因为索引条目超过1024字节,故没有录入到索引中同时也没有触发唯一索引约束,所以是查询不到数据的。但是按照_id或者key2单条件去检索都是使用了各自的单独索引, _id以及key2的索引条目在本案例中都是没有超过1024字节,故是可以检索到对应的结果的。实际key3如果单条件查询的时候强制指定key3_hashed也是可以查询到对应结果的,因为按照当前的已知的hash函数处理过后的结果都没有超过1024字节的。(这块留给爱学习的小伙伴一个疑问,是否还有另外的方法让本案例复现,如果所有的索引条目都没有超过1024字节,但是唯一索引仍然失效)

总结 

对于研发同事来说,如果真是存在本案例情形,建议使用强制指定索引或者另外的索引去检索。同时也要反思一下为什么一个索引条目会超过1024字节,那么长的索引真的有必要存在么?对于DBA来说这个参数日常是否开启还是应该与研发一起商议在做决断。

实际从接到反馈到最后解决,总结就是可能日常很复杂的问题,到最后发现原理越简单,建议多读读官方文档。

最后由衷的感谢MongoDB中文社区的李丹老师对本文的技术审核,让我对一些细节加深了理解。

作者:张芷嘉

喜欢跑步,做饭,睡觉,出门三公里就困。使用mongodb两年,倍感丝滑。

添加小芒果微信(ID:mongingcom)进入中文用户组技术交流群。

MongoDB-全球领先的现代通用数据库
点击访问MongoDB官网www.mongodb.com/zh
Tapdata-异构数据库实时同步工具
点击访问Tapdata官网https://tapdata.net/
 

Mongoing中文社区

MongoDB中文社区微信公众号

扫描关注,获取更多精彩内容
社区网站www.mongoing.com
长按二维码关注我们
赞(40)
未经允许不得转载:MongoDB中文社区 » MongoDB索引怎么失效了|索引案例一则

评论 抢沙发

评论前必须登录!