Your lovely graphs are useless.

Visualisation used to be everything. If you need to make a point, show it with a nice graph.
A lot of times, when we demo our system to decision makers, they are really impressed with our ability to analyse their data and display it in graphs.

However, while increasing credibility and showing our software's lovely abilities, graphs have a learning curve, and sometimes it takes an expert to analyse graphs and conclude the necessary conclusions. While we accompany our clients during the initial use of the system, not too many of them are keen to learn the delicate art of data analysis, even if it looks great.

This is why we're starting to move towards insights. Instead of showing a scatter chart with quadrants and let the customer deduct where each group is located, we can display them a summary, showing each group's grade - This group consumes too much energy in relation to its throughput. this one is more efficient. A link to the graph is attached, for credibility, however once the customer trust your system, he won't even need it.

Build trust with your customer, and then only show him the bottom line instead of confusing him with complicated visuals. He will feel better treated as a decision maker rather than one of your analysts.

Do not bury the Laptop just yet...

Four years ago I had an argument with a peer of mine about Chromebooks.
We got a Chromebook from Google, I tested it and found it slow and useless, both as a work tool and a browser. Even battery life was poor.
Laptops (And PCs) seemed to be a decaying platform back then. Tablets were all the rage, and for consumer (And education) seemed like the go to platform.
My prediction was that ChromeOS, which was parallel to Android, was useless, and sales figures approved this.

But Google, with its limitless cash, kept on pushing.
Web tools got way better, becoming usable (Google docs, Office 365, Dropbox to name a few)

The first hit came from Samsung, which sold a 250$ Macbook air lookalike, with decent screen and battery life. It quickly became the most selling laptop in Amazon.

Finally, after finding out that tablets are a nice play tool, but not as good for education, the educational market also caught on.

I'm pretty sure that decent developer frameworks are on their way as well, though the road is still a bit long.

So now, I'm admitting I was wrong - The Google way of doing things (Throwing pre mature products to the market, and then throwing a lot of $$$ on them until they take off) actually works.

Unless you're Google plus.

From one hosting to another...

This weekend, we moved our cluster from Rackspace to Amazon.

It was a very tense process, as we need to transport GBytes of data seamlessly from one server to another, as well as shift our data writing server from the old database to the new one, while losing as little data as possible.
Eventually, it was done in less than two hours, most of which were exporting data from the old server and transporting it into the new one.

My main tips for performing such a move:
- Sit down before, and write the process step by step. Preparations for the new server, exporting and transporting the data, updating DNS services, testing. Discuss and review with your team.
- Review the process, and estimate risks and contingencies. What happens if it takes too long to transport the data? What happens if you have to roll back? What happens if the new server crashes?
- In case of DNS updates, it would be faster and more reliable to add the new domain ip to /etc/hosts instead of waiting for DNS to refresh, which might take time. Don't forget to jot down the IPs of your old servers, you might need them :)
- In our case, we also had some one premise sensors with no access to DNS, which means we had to use both the old and new stacks active, both working with the new database, until we're able to access those sites and change the destination IP.
- After you've got your plan laid down, do a dry run. There's nothing like a dry run to sort out bugs and add missing steps.
- Be 100% clear about who performs which step and when, but keep one person in charge of the whole process.
- Keep everyone in the loop - Upgrade was performed Friday night, so we decided to do it from home.. We were a team of three, using Google Hangout to communicate while everyone is online.

Eventually, process went quite well, Amazon SSD servers are fast and zippy, and we're ready for our next scale challenge, which would have us scaling to tables larger than 100m records while keeping our high performance standards.

Shifting weights... And Cluster Performance.

At the beginning, there was one machine.

One server ran our database and web application.

But as our database began to scale, we needed to separate them.
Separating the servers is easy - Just define an 'internal' network, and let the machines communicate via the internal leg.

However, this caused another problem - We began to experience long lags when transferring large datasets between the machines.
When dataset was larger than X items, performance began to degrade, and was substantially slower due to network lag. Below that, performance was actually better.

Part of it was due to node's mysql drivers. They're just slow. (Compared to native mysql drivers)
The other part was network lag.

So how do you solve it?
Part of my original design was using mysql as a data store - Only use simple sql, no arithmetical operations and aggregations. This allowed us to move from mysql to mongodb easily, and eased CPU load for the database.

But now it's time to shift the weight, because bandwidth is our new bottleneck.
So we're moving some of the heavier processing into the database so that a smaller dataset would transfer faster. Much faster.

Performance is a delicate art of balance. Always remember that paradigms change as scale changes. Don't be afraid to shift weight in order to keep your system balanced.

Recruitment - A different take

Growing as a company could sometimes be difficult.
You need to be able to recruit people (Or outsource them, which will not be handled in this post), train them and get them to be productive as fast as you can.

For a small startup, this is even harder - Every position counts, and every worker might be responsible for great success, or miserable failure. This can make the recruitment decision a very critical one.

For our next recruit, which should be a tier 3 support engineer, we've decided to handle the process a bit differently.

First, we decided to describe the typical workday for the position. What the person would do when he comes to the office, which would be his tasks, how would he work. We write that down.

After that analysis, we focus on what he (Or she) needs to know in order to perform his tasks. It could be deep knowledge of our product, SQL know how, MS Office skills or whatever. Divide those skills into skills he would need to know "From home", generic skills that could be taught, and specific (Our product) know how.

Once we know that, we can actually start looking for the right person. The search would be much more focused, as a lot of skills which you might consider as prerequisites will be shed during the process.
We would also have a clear training plan for that person.

In the end, I love to say that I'm looking for excellent PEOPLE - As most skills could be taught.

Happy hunting!

Behold, the future!

This week, I've spent two days at the Google TLV Campus, participating in an amazing workshop given by Prime design studio.
Though I'll probably cover the workshop in another post (Waiting for 'official' photos), I would like to share one significant insight I've got:

During brainstorming for our workshop project's specs and features, Omri (Prime's CEO) sat with us, and while we were looking at a certain concept, asked us to look forward to the future of that product. What would such a product do 10 years from now? 20 years?

So we sat down, throwing down sci-fi inspired ideas ranging from laser grids and quad-copters to terra forming robots (Yes, it was THAT crazy).

[Side note] The two basic rules of brainstorming are: Write down everything, and never argue. Anything is possible during brainstorm, even the most ridiculous ideas.

After that, we looked at the result, and understood what we'll be working on. And though it was different from the original project and looked a bit like a moon shot, it started to seem possible, and the end result (And presentation) was awesome.

After contemplating on the whole process, I've come to realise that we almost never look at our product 10 years from now.

And we should, because a lot of those features could be implemented today. 

Successful companies are great at this, because they create the future of products now. The best example is Apple, but in a smaller scale, Waze, Nest and 23andMe are great examples as well.

Show your customers the future they want, and you will own it.

The future, according to us. Visioned product is top left.

No, you are not like us.

There's a time in a person's career where he is either managed by younger people, or managing younger ones.
Nearing 40, you begin to develop a generational gap with some of your surroundings. Either you need to recruit younger talent, work with (Or report to), or manage younger people. This has all happened to me before, nevertheless, a gap of more than 10 years means a generational gap.
I might hang out in the same social networks. Maybe even (If you're still) go out to the same cool places, or like the same music. I may even look younger than your real age (Of course I do, and so do you! :)).

But still, I'm older. Which means that (If you look at it in reverse), I might as well be 120 to them.
I won't go into sociological terms like the 'x'/ 'y'/ whatever generation, but as far as I know, the two best things you can do are:

- Don't think you are 'one of them'. Acting/ behaving/ chasing habits won't make you younger. It will be like my mom getting into Instagram and starting to post comments on every one of my kids' pictures, backwards to one year ago. Just don't.

- Do try to understand the culture. The habits. The language. Much like understanding the recent buzz tech tool or design language, you do need to keep up, just keep up in your own terms, pace and perception.