From mysql to mongo in less than a week, chapter one

This week, we finally took the plunge at Lightapp, and migrated our database into mongo.

In the coming weeks I'll be writing a series of posts about the experience, along with some insights and tips, and (Of course) the end result, in numbers.

The first step was deciding we need mongo instead of mysql. This one was pretty simple. Our company's product reads data from numerous sources, then aligns them and analyses them.
Since we're using big EAV tables for storing all that data, mongodb is a far more suitable solution than mysql.
The stage was right, as our database only contains tens of millions of records, right before we migrate all our current customers (Which will scale it towards hundreds of millions). So, with the help of my good friend (And scale pro) Moshe Kaplan, we started the process.

To make things easier, we also decided on a '1 to 1' conversion. This means that we only switch our db layer, and do it without touching our application. Luckily we've designed our application just like that - Only 1 module was responsible for db connection and querying, thus it was simple to write a parallel mongo db module, and switch between the two seamlessly.

So the first major tip I recommend: 
Build your db layer as a 'replaceable' layer. This means no SQL queries or DB connections outside your layer. If you currently have a written application with lots of 'history', just search for all the queries, and replace them with function calls (And filter parameters) into one file.
You might even find that you can get rid of some duplicated code in the process :)

Second tip:
Fiddle with mongo a bit. You can use a tool like pentaho to pull data from your mysql db, and into mongo, and then login to mongo, do some queries - Learn the whole query and CRUD mechanism in mongo. It's pretty simple, and has lots of great documentation all over the web.

Third tip:
Don't expect miracles. As we only switched our query functionality at first stage, we were not expecting a huge performance gain. Optimisation takes time, and the big value we were expecting from mongo was by moving our processing engine (Data bucketing and aggregation) lower from our application layer into the db layer.

Next thing was to approach the conversion process itself, but this would be elaborated in the next post of the series.

Stay tuned for more :)

[edit:] continue to chapter 2.