MongoDB : Script to run Sharding with replica set on local machine

This simple script will help to you run sharing with multiple replica set on your local box. This makes very cool and uplifting, (if on linux use sudo / root to run shell script or commands manually. )

 

# MongoDB
# script to start a sharded environment on localhost

# clean everything up
echo "killing mongod and mongos"
killall mongod
killall mongos
echo "removing data files"
rm -rf /data/config
rm -rf /data/shard*


# start a replica set and tell it that it will be shard0
echo "starting servers for shard 0"
mkdir -p /data/shard0/rs0 /data/shard0/rs1 /data/shard0/rs2
mongod --replSet s0 --logpath "s0-r0.log" --dbpath /data/shard0/rs0 --port 37017 --fork --shardsvr --smallfiles
mongod --replSet s0 --logpath "s0-r1.log" --dbpath /data/shard0/rs1 --port 37018 --fork --shardsvr --smallfiles
mongod --replSet s0 --logpath "s0-r2.log" --dbpath /data/shard0/rs2 --port 37019 --fork --shardsvr --smallfiles

sleep 5
# connect to one server and initiate the set
echo "Configuring s0 replica set"
mongo --port 37017 << 'EOF'
config = { _id: "s0", members:[
          { _id : 0, host : "localhost:37017" },
          { _id : 1, host : "localhost:37018" },
          { _id : 2, host : "localhost:37019" }]};
rs.initiate(config)
EOF

# start a replicate set and tell it that it will be a shard1
echo "starting servers for shard 1"
mkdir -p /data/shard1/rs0 /data/shard1/rs1 /data/shard1/rs2
mongod --replSet s1 --logpath "s1-r0.log" --dbpath /data/shard1/rs0 --port 47017 --fork --shardsvr --smallfiles
mongod --replSet s1 --logpath "s1-r1.log" --dbpath /data/shard1/rs1 --port 47018 --fork --shardsvr --smallfiles
mongod --replSet s1 --logpath "s1-r2.log" --dbpath /data/shard1/rs2 --port 47019 --fork --shardsvr --smallfiles

sleep 5

echo "Configuring s1 replica set"
mongo --port 47017 << 'EOF'
config = { _id: "s1", members:[
          { _id : 0, host : "localhost:47017" },
          { _id : 1, host : "localhost:47018" },
          { _id : 2, host : "localhost:47019" }]};
rs.initiate(config)
EOF

# start a replicate set and tell it that it will be a shard2
echo "starting servers for shard 2"
mkdir -p /data/shard2/rs0 /data/shard2/rs1 /data/shard2/rs2
mongod --replSet s2 --logpath "s2-r0.log" --dbpath /data/shard2/rs0 --port 57017 --fork --shardsvr --smallfiles
mongod --replSet s2 --logpath "s2-r1.log" --dbpath /data/shard2/rs1 --port 57018 --fork --shardsvr --smallfiles
mongod --replSet s2 --logpath "s2-r2.log" --dbpath /data/shard2/rs2 --port 57019 --fork --shardsvr --smallfiles

sleep 5

echo "Configuring s2 replica set"
mongo --port 57017 << 'EOF'
config = { _id: "s2", members:[
          { _id : 0, host : "localhost:57017" },
          { _id : 1, host : "localhost:57018" },
          { _id : 2, host : "localhost:57019" }]};
rs.initiate(config)
EOF


# now start 3 config servers
echo "Starting config servers"
mkdir -p /data/config/config-a /data/config/config-b /data/config/config-c
mongod --logpath "cfg-a.log" --dbpath /data/config/config-a --port 57040 --fork --configsvr --smallfiles
mongod --logpath "cfg-b.log" --dbpath /data/config/config-b --port 57041 --fork --configsvr --smallfiles
mongod --logpath "cfg-c.log" --dbpath /data/config/config-c --port 57042 --fork --configsvr --smallfiles


# now start the mongos on a standard port
mongos --logpath "mongos-1.log" --configdb localhost:57040,localhost:57041,localhost:57042 --fork
echo "Waiting 60 seconds for the replica sets to fully come online"
sleep 60
echo "Connnecting to mongos and enabling sharding"

# add shards and enable sharding on the test db
mongo <<'EOF'
db.adminCommand( { addshard : "s0/"+"localhost:37017" } );
db.adminCommand( { addshard : "s1/"+"localhost:47017" } );
db.adminCommand( { addshard : "s2/"+"localhost:57017" } );
db.adminCommand({enableSharding: "school"})
db.adminCommand({shardCollection: "school.students", key: {student_id:1}});
EOF


 

NOTE:

Newer mongodb versions doesn’t support mirrored config servers. Thus, you need to setup them as an ordinary replSet. To do so, replace lines 71-73 by these:

mongod –replSet cs –logpath “cfg-a.log” –dbpath /data/config/config-a –port 57040 –fork –configsvr –smallfiles
mongod –replSet cs –logpath “cfg-b.log” –dbpath /data/config/config-b –port 57041 –fork –configsvr –smallfiles
mongod –replSet cs –logpath “cfg-c.log” –dbpath /data/config/config-c –port 57042 –fork –configsvr –smallfiles

And add thereafter these lines:

echo “Configuring config servers”
mongo –port 57040 << ‘EOF’
config = { _id: “cs”, members:[
{ _id : 0, host : “localhost:57040” },
{ _id : 1, host : “localhost:57041” },
{ _id : 2, host : “localhost:57042” }]};
rs.initiate(config)
EOF

Finally, replace line 86 by this one:

mongos –logpath “mongos-1.log” –configdb cs/localhost:57040,localhost:57041,localhost:57042 –fork

Advertisements

5 thoughts on “MongoDB : Script to run Sharding with replica set on local machine

  1. Hello Sir,
    i am getting the below mentioned error
    error :- mongos –logpath “mongos-1.log” –configdb localhost:57040,localhost:57041,localhost:57042 –fork
    FailedToParse: mirrored config server connections are not supported; for config server replica sets be sure to use the replica set connection string
    try ‘mongos –help’ for more information

    Please help me to resolve the issue
    OS centos 7
    MongoDB shell version v3.4.5
    taking help from the below mentioned link
    https://docs.mongodb.com/manual/tutorial/install-mongodb-enterprise-on-red-hat/

    Like

  2. hey I am gettting this error how can i resolve it ?

    sudo mongos –logpath “mongos-1.log” –configdb localhost:57040,localhost:57041,localhost:57042 –fork

    FailedToParse: mirrored config server connections are not supported; for config server replica sets be sure to use the replica set connection string

    try ‘mongos –help’ for more information

    Like

  3. when trying to start the mongos on a standard port im getting this error:

    FailedToParse: mirrored config server connections are not supported; for config server replica sets be sure to use the replica set connection string
    try ‘mongos –help’ for more information

    why is it so?
    please help!!!

    Like

  4. Newer mongodb versions doesn’t support mirrored config servers. Thus, you need to setup them as an ordinary replSet. To do so, replace lines 71-73 by these:

    mongod –replSet cs –logpath “cfg-a.log” –dbpath /data/config/config-a –port 57040 –fork –configsvr –smallfiles
    mongod –replSet cs –logpath “cfg-b.log” –dbpath /data/config/config-b –port 57041 –fork –configsvr –smallfiles
    mongod –replSet cs –logpath “cfg-c.log” –dbpath /data/config/config-c –port 57042 –fork –configsvr –smallfiles

    And add thereafter these lines:

    echo “Configuring config servers”
    mongo –port 57040 << 'EOF'
    config = { _id: "cs", members:[
    { _id : 0, host : "localhost:57040" },
    { _id : 1, host : "localhost:57041" },
    { _id : 2, host : "localhost:57042" }]};
    rs.initiate(config)
    EOF

    Finally, replace line 86 by this one:

    mongos –logpath "mongos-1.log" –configdb cs/localhost:57040,localhost:57041,localhost:57042 –fork

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.