As time goes and data grows it is required to scale any database to maintain the quick response time and performance. Initially we run query explain plan and make changes in the document structure to make them efficient and add/modify required indexes. However even after performing clean up and maintenance tasks, the time comes to scale and ensure availability, durability and fault tolerance.
First, we scale vertically
MongoDB, like most databases, craves RAM and IO capacity. It sometimes likes CPU. The simplest conceptual way of scaling performance for a MongoDB dataset is to give it more system resources without worrying about spreading the load across servers. Normally, this is painful for cost or operational reasons. Doubling the capacity of a production MongoDB replica set means swapping larger servers in, figuring out what to do with the old ones, and hoping the new ones are just the right size to keep things healthy for a long period of time.
Then, we scale out
At this point, scaling out MongoDB is easy. A well-built, sharded MongoDB dataset is easy to reason about and will scale linearly across other servers. The sharding needs the key to divide the data across several small / commodity servers grouped in a cluster.
Flexibility is a requirement of an evolving data set
MongoDB offers numerous features that make developers lives easier. It also offers features for scale. Using the scaling features at the wrong time means compromising on developer-friendly features (unique constraints, oplog usefulness, capped collections). There is a great deal of pressure on developers to use the MongoDB sharding features even when they’re not necessary, which makes their lives worse in aggregate. The most healthy MongoDB setups started with developers using features that helped them move faster, and evolved as understanding of the problem scope and appropriate scale increased.
For developers that use MongoDB, they should make smart decisions and don’t force themselves down a path before they even have a map. We say inspect and adopt in Agile 😉