0

问题描述:
MongoDB基于GridFS 实现文件的分布式存储,使用replica sets + sharding 后,报can’t find a chunk! 错误

环境:
linux x86 64位

client : java

主机搭建信息:
shard1 192.168.1.189
mongod shard1_1:30000
mongod shard2_1:30001
mongod config1:30002
mongs1:30005

shard2 192.168.1.189
mongod shard1_2:40000
mongod shard2_2:40001
mongs2:40005

操作步骤:
1.使用如下命令搭建 replica sets + sharding
1.1 创建目录:
[root@localhost bin]# mkdir -p /root/Desktop/md/shard1_1
[root@localhost bin]# mkdir -p /root/Desktop/md/shard2_1
[root@localhost bin]# mkdir -p /root/Desktop/md/config1

[root@localhost bin]# mkdir -p /root/Desktop/md/shard1_2
[root@localhost bin]# mkdir -p /root/Desktop/md/shard2_2
[root@localhost bin]# mkdir -p /root/Desktop/md/config2

1.2 配置replica set shard1:
./mongod –shardsvr –replSet shard1 –port 30000 –dbpath /root/Desktop/md/shard1_1 –logpath /root/Desktop/md/shard1_1/shard1_1.log –logappend –fork
./mongod –shardsvr –replSet shard1 –port 40000 –dbpath /root/Desktop/md/shard1_2 –logpath /root/Desktop/md/shard1_2/shard1_2.log –logappend –fork

用mongo 连接其中一台机器的27017 端口的mongod,初始化Replica Sets“shard1”,执行:
./mongo –port 30000
config = {_id: ‘shard1’, members: [{_id: 0, host: ‘192.168.1.189:30000’},{_id: 1, host: ‘192.168.1.189:40000’}]}
rs.initiate(config)

1.3 配置replica set shard2:
./mongod –shardsvr –replSet shard2 –port 30001 –dbpath /root/Desktop/md/shard2_1 –logpath /root/Desktop/md/shard2_1/shard2_1.log –logappend –fork
./mongod –shardsvr –replSet shard2 –port 40001 –dbpath /root/Desktop/md/shard2_2 –logpath /root/Desktop/md/shard2_2/shard2_2.log –logappend –fork

用mongo 连接其中一台机器的27017 端口的mongod,初始化Replica Sets“shard1”,执行:
./mongo –port 30001
config = {_id: ‘shard2’, members: [{_id: 0, host: ‘192.168.1.189:30001’},{_id: 1, host: ‘192.168.1.189:40001’}]}
rs.initiate(config)

1.4 启动配置服务器:
./mongod –configsvr –dbpath /root/Desktop/md/config1 –port 30002 –logpath /root/Desktop/md/config1/config.log –logappend –fork
注意,如果路径(/root/Desktop/md/config1)错误,启动失败,报错:child process failed, exited with error number 1

启动路由服务器(–chunkSize 1):
./mongos –configdb 192.168.1.189:30002 –port 30005 –logpath /root/Desktop/md/mongos.log –logappend –fork
./mongos –configdb 192.168.1.189:30002 –port 40005 –logpath /root/Desktop/md/mongos.log –logappend –fork

备注:
启动路由服务器时BadValue need either 1 or 3 configdbs
mongos启动只能绑定一个或三个配置服务器。

1.5 配置Shard Cluster
./mongo –port 30005
use admin
db.runCommand({addshard:”shard1/192.168.1.189:30000,192.168.1.189:40000″});
db.runCommand({addshard:”shard2/192.168.1.189:30001,192.168.1.189:40001″});

1.6 激活数据库及集合的分片
mongos> use admin
switched to db admin
mongos> db.runCommand({ enablesharding:”picturesDB” })
{ “ok” : 1 }
mongos> db.runCommand({ shardCollection : “picturesDB.pictures3m.chunks” , key : { files_id : 1 } })
{ “collectionsharded” : “picturesDB.pictures3m.chunks”, “ok” : 1 }

备注:
1.6.1 执行分片命令分片,使用collcetion chunks(picturesDB.pictures3m.chunks).
1.6.2 请切换到admin数据库再 进行分片操作。

访问服务器端口30005 ,40005 。

2.使用java存储图片(存储4000张3M图片),大概代码如下(前面存了1000张,没分片前是正常的):
for(int i = 1000 ; i < 5000; i++)
{
try {
// 存储fs的根节点
GridFS gridFS = new GridFS(db, "picture3m");
GridFSInputFile gfs = gridFS.createFile(new File(System.getProperty("user.dir") + "/3m_image.JPG");
gfs.put("aliases", Integer.toString(i));
gfs.put("filename", Integer.toString(i));
gfs.put("contentType", "3M_image.jpg".substring(filename.lastIndexOf(".")));
gfs.save();
} catch (Exception e) {
e.printStackTrace();
System.out.println("存储文件时发生错误!!!");
}
System.out.println("finish i = " + i);
}

3.使用java获取图片
File file = new File("D:\\file\\3M-1273.jpg");
GridFSDBFile gf = p.retrieveFileOne(, );//".jpg"
try {
// 获取fs的根节点
GridFS gridFS = new GridFS(db, "picture3m");
GridFSDBFile dbfile = gridFS.findOne("1273");
if (dbfile != null) {
return dbfile;
}
} catch (Exception e) {
// TODO: handle exception
}
long l = gf.writeTo(file);
System.out.println("result = " + l );

问题:
步骤执行到第三步时,获取id为1275的图片为null,并且报 can't find a chunk! file id: 557847e40e03135574e2fe34 chunk: 9 。

问题补充:
1.获取1274图片已经不完整,只有半张图片。

请帮忙查看下,是内部bug,还是其它问题?