Deus Ex Machina

AI.
Everybody's talking about AI.
The major issue I have as an engineer is the gap between AI as it is perceived and what it actually is right now.
Rather than being a new life form, with conciseness, AI is actually a specific skill we teach the machine to do for us. Be it drive a car from A to B, perform a specific medical diagnosis or get you an insurance policy, it's a collection of independent features, that even when aggregated, are still features.
The more interesting field, which actually might cause the rise of the real AI, is the regenerative networks. Basically, a regenerative network is one where you teach a computer a skill, and then ask it to perform this skill. Some chat bots are a good example. They learn how people communicate and can then imitate what they've learned.
Another great example, with unexpected results, is Google's deep dream, where the machine paints its associations from the image input.

That's where I see AI evolving into - Not just an imitation of human capability, but a whole new take on how reality is perceived.

(Image taken by me and processed with Google's deep dream engine)

Responsive?

One of the major premises of CSS3 back in the days was "Responsive design".
The ability to dynamically change display object's properties when changing screen size.

Then, with retina displays, came other problems. What is the "real" screen resolution? What should be asset size?

Everyone was at it. Devs were resizing their screens like nuts, measuring ems, rems or whatever. Playing with thresholds.

However, I have yet to see an 'app' website which is more than just a landing/ marketing page/ search result implement responsiveness with success. The good ones I've seen are actually an 'm' design - Which means that mobile users are redirected to a whole different site.

This brings us back to the basics of designing web/ apps. Mobile users have other needs and limitations. You can't just reorder and enlarge elements and expect your solution to work.

This rule counts especially if your desktop web uses a heavy, black magic "modern" web framework - Your mobile users would suffer. Mobile web clients should be as minimal and light as possible, otherwise battery life (Phone heating up) and sluggish response would result in bad conversion rates.

Remember when Steve Jobs threw away Adobe Flash from the first iPhone, thus condemning it to death? That was the exact reason.

Backoffice Matters.

Most b2c companies are busy on making their service great. Delightful, easy to use, cutting edge etc.

Thus, most of the efforts of a startup are concentrated in "the app".
However, as a company scales (Or plans to scale), the backoffice becomes more and more essential to the company's ability to face the growing stream of clients.
Most companies address those issues too late.

Remember to shift your focus at the right point, or else your team will be putting out fires and not scaling your company.

App = Sell, Backoffice = Scale.

Think inside the box!

The cliches of creative thinking could sometimes be counter productive.

As you all probably know, "think outside the box" and "find a creative solution" are common phrases in problem solving. Letting yourself loose of current constraints (Of, sometimes, resources, technology and money), and trying to find a solution outside of your current method of operation.

In our world, this could lead to adventurous, cost and money taking processes. Rewriting something because it doesn't work. Using a new library which solves "everything". Changing and switching components. Redesigning your UX.

However sometimes, the solution is not far away from what you already have.

Take a small piece of paper and jot down all the resources you already have in hand which could be involved with this problem. Now start writing possible solutions USING THOSE RESOURCES AND ONLY THEM.

You might just find you already have a solution inside your box.

Don't solve this.

Part of the fun in building a new product is creating and writing it. A lot of the creation process includes problem solving - We need to do this feature, we need to create an interface for that etc. etc.

However, a lot of these problems have already been solved.
Don't waste your time re-solving problems yourself in an early stage. Do some research - You'll most likely find off-the-shelf components, even if they're not 'tailor made' for your needs.

Put your efforts on your new product, not on re-solving problems that were already solved.
This will save you a lot of time to focus on what's really important - Your value.

Ad Blocking - A different approach

This is nice:
Wired (And probably other content sites) is now making a lot of its cash from displaying ads.

However, as ad blockers are common practice, this hurts their living. A lot.
As solutions like DSERO are getting more and more common, there's also the direct approach method.


Worked in my case. I've whitelisted Wired.

How Netflix cracked the formula for hit shows

I'm a sucker for comics. Specifically super hero comics. And, after seeing Netflix's Daredevil, I was taken by how accurate and well done it is. It's one of the best adaptations I've seen to a comic series. Fast paced, good action, just blunt and dark enough.

That got me thinking - How the hell did they do that? After "Orange is the new black" and "House of cards", Netflix seem to be churning out great TV, again and again.

The answer is simple. Netflix, which started as a mail rental service (Just because broadband infrastructure wasn't relevant back then) and moved on to online video rental, simply know exactly what we want as viewers.
They know which movies we like.
Which movies we watch until the end, and which movies we "break" in the middle.
They also know which scenes we run over and over again.

By tagging every scene in every content, they have the ability to create the perfect show for each one of their viewers. They can actually make script decisions (Exactly how the story will be built) according to data.

Furthermore - In the future, story would not be linear as we know it, but every subscriber will get his own, personal, customised version of the show. Yoni Bloch's Interlude might actually close this gap from the production end.

That's pretty amazing.

That Magic Factor

Apple came out very strong from yesterday's event.

Though nothing's really new (Macbook model was long due, though it's much more stunning than I've expected), there is one thing I take from this:


The Magic Factor.

Watch wearables, and successful ones, exist for years now. Garmin, Pebble, Android wear - They're all there. But still, none of them generates the excitement that the Apple watch does.

Than I've looked at last year's major Apple releases, and found out that each year, there was a feature released that was either magical or breakthrough - None of the competitors had it, and it worked like magic:

- Retina screen (iPad and than Macbook Pro)
- Macbook air battery life
- iPad mini form factor
- Touch ID
- iMac 5K
(And now) The watch, force touch, HR monitor, research kit...

Apple seems to have a 'game changer' out at least once a year. One thing that creates a buzz. When was the last time you were THAT excited about a product from Microsoft, Samsung or Google?

My Next Laptop

Your lovely graphs are useless.

Visualisation used to be everything. If you need to make a point, show it with a nice graph.
A lot of times, when we demo our system to decision makers, they are really impressed with our ability to analyse their data and display it in graphs.

However, while increasing credibility and showing our software's lovely abilities, graphs have a learning curve, and sometimes it takes an expert to analyse graphs and conclude the necessary conclusions. While we accompany our clients during the initial use of the system, not too many of them are keen to learn the delicate art of data analysis, even if it looks great.

This is why we're starting to move towards insights. Instead of showing a scatter chart with quadrants and let the customer deduct where each group is located, we can display them a summary, showing each group's grade - This group consumes too much energy in relation to its throughput. this one is more efficient. A link to the graph is attached, for credibility, however once the customer trust your system, he won't even need it.

Build trust with your customer, and then only show him the bottom line instead of confusing him with complicated visuals. He will feel better treated as a decision maker rather than one of your analysts.

Behold, the future!

This week, I've spent two days at the Google TLV Campus, participating in an amazing workshop given by Prime design studio.
Though I'll probably cover the workshop in another post (Waiting for 'official' photos), I would like to share one significant insight I've got:

During brainstorming for our workshop project's specs and features, Omri (Prime's CEO) sat with us, and while we were looking at a certain concept, asked us to look forward to the future of that product. What would such a product do 10 years from now? 20 years?

So we sat down, throwing down sci-fi inspired ideas ranging from laser grids and quad-copters to terra forming robots (Yes, it was THAT crazy).

[Side note] The two basic rules of brainstorming are: Write down everything, and never argue. Anything is possible during brainstorm, even the most ridiculous ideas.

After that, we looked at the result, and understood what we'll be working on. And though it was different from the original project and looked a bit like a moon shot, it started to seem possible, and the end result (And presentation) was awesome.

After contemplating on the whole process, I've come to realise that we almost never look at our product 10 years from now.

And we should, because a lot of those features could be implemented today. 

Successful companies are great at this, because they create the future of products now. The best example is Apple, but in a smaller scale, Waze, Nest and 23andMe are great examples as well.

Show your customers the future they want, and you will own it.

The future, according to us. Visioned product is top left.

Clear off the table

[ taking some time off the ongoing mysql->mongo series ]

Been visualising data a lot lately.
Scatter charts, realtime indicators, good ol' line charts and, of course, tables.
It's really easy to fill the screen with loads of information, screaming colours and as many numbers floating around the screen. This makes orientation difficult for the user, especially non-technical users.

Some of our users simply need insights and recommendations, but need the graphs for credibility.

This is a great post from darkhorseanalytics explaining how a good table looks like. [hint - not your good ol' office 1998 excel highlight table]

http://darkhorseanalytics.com/blog/clear-off-the-table/

Way too lean.

Everyone's going lean these days. Work lean and cheap, go to market as quickly as possible, fail fast.
While this approach works great in our time and age, it causes a lot of products (Jelly, for example) to come half baked.
The philosophy behind it is correct. Test the market as fast as you can, to validate your idea and gain ground. If you fail enough times, you might even succeed once. And one good success is all you need, right? (Yes, I am being ironic.)

However, unless your service goes ballistic right after launch, you might never know if it works or not. Hell, even if it does - What's your measure of success?

99% of the apps do not pass the 10000 download barrier. So what does success mean? You might fail your product a bit too fast there. You might even not know why it failed.

My advice would be to be a bit more thorough about your product. Get real feedback from as many people as you can (2nd hand friends are even better, as they don't owe you anything. You can even pay a symbolic token like a t shirt or something).

Don't launch too early, don't fail too fast. Thinking twice might not align with the cranky, wham bam lean approach, but might save you a lot of time and heartache.

Deliberate Mistake, correct end result

I've spent my army service days as an infantry soldier.
A lot of my unit's field practice included night navigation - Where you learn a path (By heart), and sent to navigate few miles at night, and collect a few waypoints.

The most challenging part of these drills was not walking with 35 pounds of gear on, or even seeing what's ahead of you to avoid pitfalls. It was memorising the path, and walking through it - If you don't know everything by heart (Sometimes you only have 20 minutes to prepare your gear and learn the path), you're bound to get lost. If you get lost, you walk more. If you walk more, you'll be more tired. And so begins a downward spiral no one likes.

I found that the most effective way for me to study these paths was using the deliberate mistake method: Where you choose a point on the map that's easy to recognise and reach, and from there walk to the nearest waypoints. This would sometimes mean walking more than the 'direct' approach, but you'd be less likely to forget a turn or a hill, and less 'marks' to count when you're walking, at night, with your gear on.

Nowadays, sometimes when I look at decisions I need to make, I sometimes take the 'wrong' decision, to reach a larger goal sooner - Like coding some ugly patch - I know I can correct this mistake later, but I also know I would also reach my goals on time, with confidence.

You can stop developing your product now, thanks.

Once a startup's MVP has been released, there's always the question of what's next.
A lot of startups tend to use the time after sorting out the quirks and the bugs to develop more features and enrich the product.

But sometimes, adding more features to your product before even finding out what works will only distract your new users, who are trying to get to know your product.

A good example might be Google Plus vs. Twitter.

Twitter has remained essentially the same as it's been since its inception. Google Plus has overhauled completely more than once, and is crammed with tons of features. While Google stuffed its social network with more and more features (Share and video and an amazing image viewer and more menus and what not), and how to implement them different than Facebook, Twitter focused on refining its mobile and web experience.

So next time your R&D department got spare time, have them improve performance, clean up code and take care of automation and scale rather than add more buttons and features that might just make your product too rich and too redundant.

Your users will appreciate more if they wait 1 second instead of 5 for a response rather than another copycat feature.

New stack, full stack.

In the past month I've been busy rebuilding the LightApp stack from scratch.
LightApp is a very cool startup in the industrial energy management field.
The product's essence is collecting data from various sources (Sensors, meters, production databases) of huge factories and analysing it to find ways to improve their effectiveness and reduce their costs and carbon footprint.
Previous product was built over LAMP stack (Linux/Apache/PHP/MySql), using the webii PHP framework, and had a lot of limitations - One example was heavy relying on SQL to perform statistical analysis, which was costly and shown poor performance and response time.

We've decided to rebuild the client (Not the database) with node.js, and after less than a month (We're a team of two) we've managed to almost completely finish our rewrite.
For UI we've decided to use bootstrap and amcharts, and manage the data ourselves (Rather than use a framework like angular or backbone).
Over the server side, express was used for main routing, mysql for database connection, and nunjucks for server side HTML.

The application doesn't have too many end users (Expecting few hundreds during 2014), but each user is performing heavy statistical analysis operations (Like a distribution of hourly data per category over a period of 6 months) - Which should be done really fast, and it is. Querying large data sets and crunching them in node is amazingly fast, but in the near future we will move our data to mongo, which can perform this analysis even faster.

Developing the whole stack in Javascript is fun - Though the language has a lot of limitations, it's fun to use, easy to debug and test, and, for our use, proves to be cost effective. Node.js asynchronous programming mode takes some time getting used to, but if you've dealt with AJAX on the client side, you'll be quite familiar with it.

I would highly recommend every programmer to try writing a full stack JS application. It's amazing how Javascript has evolved in the recent years.

Rewrite your Product, and live to tell about it

In every organisation life, there comes a time when its product is due to a big overhaul.
From dated GUI to dying Database, a Business Logic layer that's filled with patches, or even personal changes ('The only girl that knew that code has left...').
So now the company gives you, her beloved manager, a clean slate to rewrite the product.

Supposedly a dream come true, right?
In a past post, I've discussed the new system dilemma, a situation where you're already knee deep inside the rewrite, and pressure is rising.

This post should help you avoid that terrible situation.

The most important rule when writing a product from scratch is deciding what it's NOT going to do in phase one (See time frame in the second rule). Trying to match all the features the old system supports (Sometimes while it's still being developed by tier 4) will result in an endless effort.
Focus, and be transparent with what you'll deliver and what you won't. Put an emphasis on what your rewrite brings to the table that the old stack doesn't.

The second most important rule is DON'T LINGER. 3 months. Show something (Even a playable mockup) after 1 month. Show progress every week.
Three months is a long time, and mind that things can change, but it's a short enough time frame that things won't turn completely around.

When phase one is near the end, you can "Let the company in" and plan the next stages.

Focus and Agility bring results. Go for it!

The essential startup toolkit

Not too long ago, software development was an expensive deal.

Development tools used to cost a lot of money, project management and documentation tools cost money, servers cost money, internal network, telephony, office space etc. etc.

Software development practices, on the other hand, aren't that different - Technology is, but the product cycle is largely the same - scrum and agile like methods exist for 20 years now, they were just called micro management :)

I remember coding in Borland C or early versions of Visual C++ (There was no visual studio back then, just a collection of MS tools), writing documentation in Word, updating my Project file, and using a shared network Novell drive. My computer was running windows 3.11, which was considered the standard OS. And none of that was free.

And that was only money. Setting up your business took time. Buying software (At a store...) gave you a lot of discs. Configuring a company network required a specialist. Installations took forever .(Remember disc 3 out of 14?)

Now all you need to have is a computer. It can run a free version of Linux. And if you don't compile, in could also be a cheap Chromebook. Hell, even if you DO compile, there are SAAS platforms which do that as well.

Servers? Basic SSD hosting would cost you ~60$ per year. Monthly internet access + Router + Cell phone would cost ~50-100$ Monthly. Need linux servers? Get a Pi for 25$. Need a phone to test? Use your own :). The only hardware that's relatively expensive would be if you want to develop iOS app, that would set you a minimum of 600$ for a mac mini and 230$ for an iPod touch... (Plus a 99$ yearly developer subscription...)

And all the rest is free. From task management to development tools to server software and source control, everything is completely free (To a certain scale, of course).

The only thing that's really expensive?
Your time.

Now go and make the most of it!

Performance - Where it counts.

A good report does magic with large data.
It is supposed to take millions of records, and summarise them into a visual entity that transforms into something meaningful for the user.

Recently, in one of my sessions, a client wanted me to focus on improving performance for a specific report in his system.
Report was taking 5 minutes to generate, clogged the system's database, and was comprised of one of the largest SQL statements I've seen in my life.
Instead of starting to spend hours and hours trying to optimise and dig deep, something that would cost my client a lot of money, I asked him a simple question:

- Who produces this report, and how often does it happen?

Turns out it was a weekly (Even monthly in some customers) report, printed and handed out to a manager.
Assigning a periodic process running on a backup database, and automatically sending the result via mail to the manager saved a lot of money and headache to my client.

Performance is important, but you should always focus on improving it where it makes a real difference.

Do it where it counts!

Performance issues? Remain calm and analyse.

Web companies are all about growth.
But also about speed.
It's all about getting your great idea out there, as fast as you can, which means scale and performance issues could be left 'for later' (Which is mostly a good idea at the beginning stage, when resources are sparse, and time is an issue).
Once your service scales from 100 users to 10000, you will encounter performance issues. If you have an app, response times would slow to a halt. A site, data loads slow.
In 2013 Google IO UI sessions, it was mentioned that app users tend to close their app when waiting for more than 10 seconds. While that might sound harsh, imagine tapping, waiting 10 seconds, then tapping again, waiting again - I would uninstall such an app, or never visit such a website (Unless they have something I really, really need, in that case I'll look for a competing product before uninstalling)

Clear your head. Analyse the situation.

So, what do you do?
First, don't pannick. Performance issues happen to everyone. (Kidding)
But be focused, because time is of the essence here.

An 'automatic' go to solution is upgrading your hardware - I wouldn't go there unless I was really desperate - This is hardly ever a scalable solution.

First thing you would want to do is analysis.
The base for all solutions shouldn't be by intuition alone - This could lead you to waste time on ideas that may or may not bring the solution, and waste valuable time.

- Analyse each step by time, dump it to a log file. Find the 10 most time consuming tasks, and select the easiest ones to take care of.
- Analyse DB calls - See which ones are being performed to most, or which ones take a lot of time.
- Analyse insertion rates - Maybe a queuing mechanism could solve some bottlenecks.

- Always use a perl script to 'clean' logs, do count operations on logs, and a spreadsheet to calculate and sort stuff - These are the best ways to get insights.
- Call a friend. Four eyes are always better than two.
- Call an expert. It's cheaper to pay someone for two expensive days of work and get your issues solved than lose a week of momentum.

And a final note - Don't try to rewrite your server in two days of 'no sleep'... ;)

Good luck.

Ah, the funnel

Ah, the wonderful funnel.
The science of users acquisition, retention, and conversion... The web has moved from pageviews, to clicks, to conversions.
And so should you!

Got your product up? Want millions of users to register and use it? Want to convert non paying users to paying users?
Some fun funnel basics.
- If your users register for free, make the initial registration almost transparent. email, that's it. (At least until you got enough viral buzz). Maybe not even by logging in and allowing an app to 'post to your facebook profile'. We just met.
- Be nice with mails. Linkedin has the habit of 'bombing' you. Twitter bombs you once you neglect them. Actually, everyone bombs you unless you learn the tedious process of disabling or throttling mail traffic. However, you're not LinkedIn, and every user pressing 'unsubscribe' would be lost.
- Try to personalize the mail message. Even if it's automated, a personal call for action is much better than a generic one.
- Always mention what value you give to your users. Real value, no slogans, especially for paying ones. You can also do that by giving value statistics to your user (250 people have looked at your profile...)
- Use analytics. If your registration has more than one step, see how many 'leaks' you've got, and improve. Try to find out which people opened a form, but never filled it. Those might need an extra push. You can automate that one as well.
- Remember what differentiates you from the competition, or from similar services. Remind that to your audience as well.

Happy hunting!

p.s. - If you're a startup, register here, and get featured!