Slow down to go fast.

Too long ago, I was starting a new sport - Dirt bike riding.

Intensive, extreme, physically punishing and amazingly rewarding sport, with lots of skills to learn.

During one of my first rides, in between trees, rocks and trails, one of the riders gave me a tip I do not practice often enough:

To go fast, you also need to learn how to ride really slow.

This actually makes perfect sense. You need to practice every nuance of your ride, do it right and repeat, until doing it fast becomes trivial.

However, if you want to improve, from time to time you need to slow down and review what you've been doing thoughtless - You might find out you're doing something wrong there.

Sometimes you need to go back to the things you do quickly on auto pilot and do them slow and methodic.
You might find out you can still improve.

Deus Ex Machina

AI.
Everybody's talking about AI.
The major issue I have as an engineer is the gap between AI as it is perceived and what it actually is right now.
Rather than being a new life form, with conciseness, AI is actually a specific skill we teach the machine to do for us. Be it drive a car from A to B, perform a specific medical diagnosis or get you an insurance policy, it's a collection of independent features, that even when aggregated, are still features.
The more interesting field, which actually might cause the rise of the real AI, is the regenerative networks. Basically, a regenerative network is one where you teach a computer a skill, and then ask it to perform this skill. Some chat bots are a good example. They learn how people communicate and can then imitate what they've learned.
Another great example, with unexpected results, is Google's deep dream, where the machine paints its associations from the image input.

That's where I see AI evolving into - Not just an imitation of human capability, but a whole new take on how reality is perceived.

(Image taken by me and processed with Google's deep dream engine)

Responsive?

One of the major premises of CSS3 back in the days was "Responsive design".
The ability to dynamically change display object's properties when changing screen size.

Then, with retina displays, came other problems. What is the "real" screen resolution? What should be asset size?

Everyone was at it. Devs were resizing their screens like nuts, measuring ems, rems or whatever. Playing with thresholds.

However, I have yet to see an 'app' website which is more than just a landing/ marketing page/ search result implement responsiveness with success. The good ones I've seen are actually an 'm' design - Which means that mobile users are redirected to a whole different site.

This brings us back to the basics of designing web/ apps. Mobile users have other needs and limitations. You can't just reorder and enlarge elements and expect your solution to work.

This rule counts especially if your desktop web uses a heavy, black magic "modern" web framework - Your mobile users would suffer. Mobile web clients should be as minimal and light as possible, otherwise battery life (Phone heating up) and sluggish response would result in bad conversion rates.

Remember when Steve Jobs threw away Adobe Flash from the first iPhone, thus condemning it to death? That was the exact reason.

Backoffice Matters.

Most b2c companies are busy on making their service great. Delightful, easy to use, cutting edge etc.

Thus, most of the efforts of a startup are concentrated in "the app".
However, as a company scales (Or plans to scale), the backoffice becomes more and more essential to the company's ability to face the growing stream of clients.
Most companies address those issues too late.

Remember to shift your focus at the right point, or else your team will be putting out fires and not scaling your company.

App = Sell, Backoffice = Scale.

The Age of Compute Services

In the past ten years we've seen the "Rise of the cloud".
Almost everyone moved their in house IT and infrastructure to Amazon/ Rackspace/ IBM/ Azure/ Google/ Whatever, got rid of their huge racks and pizzaboxes, and went on with their lives.
Enter the rise of Devops - A caste of people who control the know how of building, deploying and maintaining your servers. Mostly due to poor documentation of the host platform.
Then Docker came along - The ability to create containers for applications, which makes it even easier to run them - However, Docker comes with its own disadvantages (Irrelevant to this post.)

But recent years show a new trend - Where services like Amazon Lambda, Google Cloud Functions are actually code containers (Rather than full blown OSs).

This changes the paradigm even further - No OS needed. Just code, deploy and go. Connect it to a file system (Amazon EFS)/ storage/ database/ whatever as a service, and you've got yourself a full working server with no OS, no deployment, all managed, secured, scaled and charged on demand.

This changes the economy of cloud services, as you actually pay for what you consume, and not for what you MIGHT consume (Even if you scale down during downtimes, it's never immediate)

It also changes the paradigm of programming - No need for security concerns (All managed), just a collection of volatile services.

In this new paradigm, cloud cyber, virtualization and devops businesses should take notice. Soon, their business might be completely irrelevant.

Slides from the Aleph "Summer of bots" event

Though my bot didn't win the challenge, it was a great experience. 20 teams applied, and Ricky was a finalist.

Here are my slides from the final event.

Ricky

Think inside the box!

The cliches of creative thinking could sometimes be counter productive.

As you all probably know, "think outside the box" and "find a creative solution" are common phrases in problem solving. Letting yourself loose of current constraints (Of, sometimes, resources, technology and money), and trying to find a solution outside of your current method of operation.

In our world, this could lead to adventurous, cost and money taking processes. Rewriting something because it doesn't work. Using a new library which solves "everything". Changing and switching components. Redesigning your UX.

However sometimes, the solution is not far away from what you already have.

Take a small piece of paper and jot down all the resources you already have in hand which could be involved with this problem. Now start writing possible solutions USING THOSE RESOURCES AND ONLY THEM.

You might just find you already have a solution inside your box.

DevOps is the new Server side

A long time ago, the weight was different in the software world. Server side programming was considered to be the pinnacle of engineering, with lots of knowledge and skill requirements.

But open source changed all that. Most server side problems are already solved by off the shelf code, and server engineers became more integrators and architects than actual programmers: It's pretty easy to connect everything, build a logic layer (That might be complicated - But mostly due to large amount of product requirements and not complexity), an API layer, a database et voila!

As server code becomes simpler (And client code becomes more complex and rich), the main challenge for backend and application engineers became the DevOps side.
Building, the application, integrations, dependencies and frameworks, automating processes and tests, securing, backing up and scaling the system have become the skills to learn.

And while those skills used to be owned by selected devops engineers, more documentation and better tools allow every server side engineer with network and os knowledge the ability to do all of the above.

All of this, of course, helps you focus on what's really important as a developer: Delivering.

Microservices ecosystem (Credit: Sequoia)

Asus x205ta vs. Macbook Air 11

My little daughter is 9, and we've decided it's time to get her a laptop.
I was looking for something that has a small form factor, no moving parts, and inexpensive.
Was considering a Chromebook, but then came across the x205ta from Asus - A small, Chromebook-like device, capable of running Windows 10.

Price was under 200$, so not much of a risk, and after receiving the machine with windows 8.1, I had to upgrade to windows 10 (Left the upgrade working overnight.)

After using it a little bit, the obvious question was - How does it work in comparison with my (Now wife's) Macbook Air 11?

So here are some conclusions:

- Specs: The Asus has budget processor, 32 GB drive, 2 GB ram and costs 25%. Though these specs are quite nothing in comparison with the Air's Haswell processor and 4/128 RAM, But you can get a 128GB micro SD and extend your HD capacity for dirt cheap.
- Form factor: The tiny Asus is a bit lighter, smaller width, and thicker than the Air 11.
- Finish: No comparison here. The Asus feels much cheaper. Its keyboard and touchpad are ok. The screen has the same resolution, but isn't as good. But it's completely ok, considering it costs quarter than the Mac.
- Performance: Well, you can't really compare a budget processor and 2 GB RAM to a top of the line one with 4 GB. But Windows 10 actually works really well, runs office and browsing fine, and (Compared to the unusable Metro UI) is very comfortable to use. It's not a gaming platform.
- Battery life: Now this is the first time I've seen a windows laptop runs so long between charges. More than 12 hours (For browsing and office usage). The Air? 9 hours was my record.
- Software: The most important software for my kid is office, and though Office runs on both laptops, Hebrew support for Office still lags on Mac OS. The Asus comes with 1 year free Office subscription.

In conclusion - This is a very smart buy for students. Cheap, light, fun to use, and runs everything you need - It's not a tool for heavy programming, photoshopping or gaming, but rather for browsing and office applications. The price is the main disruption here - Why get a tablet for your kid when you can get her a fully functional laptop?
This is quite like comparing apples and peaches (Pun intended) - Eventually, if I'm a casual user, this is a no brainer - The Asus takes it. But if I'm a heavy user and need an ultraportable, the x205ta isn't an alternative to the Air. I would look for something more in the area of the Air's 800-900$ MSRP (Yes, they MSRP for less now...).

My take on the Google/ Alphabet rehash

So Google gets a new CEO, and the leaders form a parent company named Alphabet.
I've written quite a lot about Google's ability to change, and pressure reinventing themselves out of being an ad agency.
Looks like they've come to the conclusion that they can't. Google will remain Google, and Alphabet will be... Something else?

The real question is, if you don't change the people in charge, will you be able to do new things?

Time will tell. One thing is certain: They have enough cash to begin with.

Don't solve this.

Part of the fun in building a new product is creating and writing it. A lot of the creation process includes problem solving - We need to do this feature, we need to create an interface for that etc. etc.

However, a lot of these problems have already been solved.
Don't waste your time re-solving problems yourself in an early stage. Do some research - You'll most likely find off-the-shelf components, even if they're not 'tailor made' for your needs.

Put your efforts on your new product, not on re-solving problems that were already solved.
This will save you a lot of time to focus on what's really important - Your value.

Ad Blocking - A different approach

This is nice:
Wired (And probably other content sites) is now making a lot of its cash from displaying ads.

However, as ad blockers are common practice, this hurts their living. A lot.
As solutions like DSERO are getting more and more common, there's also the direct approach method.


Worked in my case. I've whitelisted Wired.

How Netflix cracked the formula for hit shows

I'm a sucker for comics. Specifically super hero comics. And, after seeing Netflix's Daredevil, I was taken by how accurate and well done it is. It's one of the best adaptations I've seen to a comic series. Fast paced, good action, just blunt and dark enough.

That got me thinking - How the hell did they do that? After "Orange is the new black" and "House of cards", Netflix seem to be churning out great TV, again and again.

The answer is simple. Netflix, which started as a mail rental service (Just because broadband infrastructure wasn't relevant back then) and moved on to online video rental, simply know exactly what we want as viewers.
They know which movies we like.
Which movies we watch until the end, and which movies we "break" in the middle.
They also know which scenes we run over and over again.

By tagging every scene in every content, they have the ability to create the perfect show for each one of their viewers. They can actually make script decisions (Exactly how the story will be built) according to data.

Furthermore - In the future, story would not be linear as we know it, but every subscriber will get his own, personal, customised version of the show. Yoni Bloch's Interlude might actually close this gap from the production end.

That's pretty amazing.

That Magic Factor

Apple came out very strong from yesterday's event.

Though nothing's really new (Macbook model was long due, though it's much more stunning than I've expected), there is one thing I take from this:


The Magic Factor.

Watch wearables, and successful ones, exist for years now. Garmin, Pebble, Android wear - They're all there. But still, none of them generates the excitement that the Apple watch does.

Than I've looked at last year's major Apple releases, and found out that each year, there was a feature released that was either magical or breakthrough - None of the competitors had it, and it worked like magic:

- Retina screen (iPad and than Macbook Pro)
- Macbook air battery life
- iPad mini form factor
- Touch ID
- iMac 5K
(And now) The watch, force touch, HR monitor, research kit...

Apple seems to have a 'game changer' out at least once a year. One thing that creates a buzz. When was the last time you were THAT excited about a product from Microsoft, Samsung or Google?

My Next Laptop

Google, The One Trick Unicorn.

A lot of people like having the discussion of picturing Google as this evil, all seeing dystopian future type conglomerate.
While Google does all but admitting to be one (A monopoly with more power than politicians, buying AI and robotic companies, creating self driving cars etc.), it's still a bit far from reality.

So let's take a step back, and see what Google is. It's the biggest ad tech company in the world. Google makes money from pushing ads.

Not a content company. Not a hardware/ space/ AI company. Not a social network, email provider. Not a hosting company.

A huge ad company.

How big is it?
It's about as big as Microsoft in terms of market capacity, but earns 44% less. And Microsoft does a lot more business than just ads.
Google is almost twice as big as Facebook, but Facebook is leaner and growing like mad these days - And video monetising is yet to come.

But the biggest fear of Google, being a one trick unicorn (I refuse to call Google a pony), is being disrupted. Because people are aching to disrupt them.

So Google is not just trying to take over the world. It MUST do that in order to survive. It needs another meaningful business. Something that IBM, Apple, Microsoft and other tech giants which are here to stay have done.

The Cloud Will Be Disrupted.

The cloud is where it's all at.. A centralised, managed, virtual place, where all the data and logic go. It's comfortable. It's better than in house IT. It's safer than keeping your docs on your PC.

For my 100th blog post, I would like to make a careful prediction about the future of applications.

Like traditional IT before, the cloud also has its drawbacks. The biggest one: It's not really free, and it's not really yours.

The docs you put on Google, mails you send via your free provider, images you post on Facebook, files you put on dropbox - Unless you start paying a monthly tax for these conglomerates, is theirs to use, and sometimes delete.
Some of the big companies built an eco system (Google, Facebook, but also airBNB, Uber and a lot of the shared economy companies), which gets a lot of the population to be dependent on them for their monthly pay check.
Some of these companies have enough cash and political power to bend governments.

The shared economy is best for the customer, however it puts a lot of power in the hands of too few companies. Imagine that all taxi drivers at the states belong to Uber.

Part of the fear of societies and companies from bitcoin is not the currency itself, but the democratic, P2P model of its application - A distributed application, in which logic, as well as data, could be shared by every user, not just by the service provider - Because every user is, essentially, a service provider.

Imagine an airBNB service hosted by every airBNB user, containing all the users in his country - Without the need to pay commission to the airBNB. Or a 'floating' Uber like service, where drivers and customers connect and bid directly.

This will get the prices even lower, and give more to the specific service provider, rather than cut a fat commission to an already fat company.

The infrastructure is already there - In my opinion, we will start see those applications during the coming years. Like P2P shook the media world, decentralised applications will shake and disrupt the cloud.

Your lovely graphs are useless.

Visualisation used to be everything. If you need to make a point, show it with a nice graph.
A lot of times, when we demo our system to decision makers, they are really impressed with our ability to analyse their data and display it in graphs.

However, while increasing credibility and showing our software's lovely abilities, graphs have a learning curve, and sometimes it takes an expert to analyse graphs and conclude the necessary conclusions. While we accompany our clients during the initial use of the system, not too many of them are keen to learn the delicate art of data analysis, even if it looks great.

This is why we're starting to move towards insights. Instead of showing a scatter chart with quadrants and let the customer deduct where each group is located, we can display them a summary, showing each group's grade - This group consumes too much energy in relation to its throughput. this one is more efficient. A link to the graph is attached, for credibility, however once the customer trust your system, he won't even need it.

Build trust with your customer, and then only show him the bottom line instead of confusing him with complicated visuals. He will feel better treated as a decision maker rather than one of your analysts.

Do not bury the Laptop just yet...

Four years ago I had an argument with a peer of mine about Chromebooks.
We got a Chromebook from Google, I tested it and found it slow and useless, both as a work tool and a browser. Even battery life was poor.
Laptops (And PCs) seemed to be a decaying platform back then. Tablets were all the rage, and for consumer (And education) seemed like the go to platform.
My prediction was that ChromeOS, which was parallel to Android, was useless, and sales figures approved this.

But Google, with its limitless cash, kept on pushing.
Web tools got way better, becoming usable (Google docs, Office 365, Dropbox to name a few)

The first hit came from Samsung, which sold a 250$ Macbook air lookalike, with decent screen and battery life. It quickly became the most selling laptop in Amazon.

Finally, after finding out that tablets are a nice play tool, but not as good for education, the educational market also caught on.

I'm pretty sure that decent developer frameworks are on their way as well, though the road is still a bit long.

So now, I'm admitting I was wrong - The Google way of doing things (Throwing pre mature products to the market, and then throwing a lot of $$$ on them until they take off) actually works.

Unless you're Google plus.

From one hosting to another...

This weekend, we moved our cluster from Rackspace to Amazon.

It was a very tense process, as we need to transport GBytes of data seamlessly from one server to another, as well as shift our data writing server from the old database to the new one, while losing as little data as possible.
Eventually, it was done in less than two hours, most of which were exporting data from the old server and transporting it into the new one.

My main tips for performing such a move:
- Sit down before, and write the process step by step. Preparations for the new server, exporting and transporting the data, updating DNS services, testing. Discuss and review with your team.
- Review the process, and estimate risks and contingencies. What happens if it takes too long to transport the data? What happens if you have to roll back? What happens if the new server crashes?
- In case of DNS updates, it would be faster and more reliable to add the new domain ip to /etc/hosts instead of waiting for DNS to refresh, which might take time. Don't forget to jot down the IPs of your old servers, you might need them :)
- In our case, we also had some one premise sensors with no access to DNS, which means we had to use both the old and new stacks active, both working with the new database, until we're able to access those sites and change the destination IP.
- After you've got your plan laid down, do a dry run. There's nothing like a dry run to sort out bugs and add missing steps.
- Be 100% clear about who performs which step and when, but keep one person in charge of the whole process.
- Keep everyone in the loop - Upgrade was performed Friday night, so we decided to do it from home.. We were a team of three, using Google Hangout to communicate while everyone is online.

Eventually, process went quite well, Amazon SSD servers are fast and zippy, and we're ready for our next scale challenge, which would have us scaling to tables larger than 100m records while keeping our high performance standards.

Shifting weights... And Cluster Performance.

At the beginning, there was one machine.

One server ran our database and web application.

But as our database began to scale, we needed to separate them.
Separating the servers is easy - Just define an 'internal' network, and let the machines communicate via the internal leg.

However, this caused another problem - We began to experience long lags when transferring large datasets between the machines.
When dataset was larger than X items, performance began to degrade, and was substantially slower due to network lag. Below that, performance was actually better.

Part of it was due to node's mysql drivers. They're just slow. (Compared to native mysql drivers)
The other part was network lag.

So how do you solve it?
Part of my original design was using mysql as a data store - Only use simple sql, no arithmetical operations and aggregations. This allowed us to move from mysql to mongodb easily, and eased CPU load for the database.

But now it's time to shift the weight, because bandwidth is our new bottleneck.
So we're moving some of the heavier processing into the database so that a smaller dataset would transfer faster. Much faster.

Performance is a delicate art of balance. Always remember that paradigms change as scale changes. Don't be afraid to shift weight in order to keep your system balanced.