Death By Metrics

‘A little knowledge is a dangerous thing’ – Alexander Pope


TL;DR – Graphs, dashboards, and metrics give a psychological illusion of control and understanding which is usually false and can be worse than no information at all.

Why is it so easy to fall subject to crappy metrics? Let’s consider a story. You’re a project manager and you just got yelled at because the engineering project you manage is already 2 sprints behind. You feel helpless. You asks the engineers why everything is taking longer and they give technical explanations about unknowns that you can’t see at all. You really don’t know anything.

Then something magical happens: you discover you can look at the engineers’ pull requests, and see how many PRs they make each day and how many diffs are in them. You start to look at these and start to notice patterns. Maybe less PRs get done on work from home Mondays… are they slacking off? When all you have is a nail, everything looks like a hammer.

And you just ruined the company. Suddenly you’re wondering why Bob has the fewest commits, why there are fewer PRs on work-from-home Mondays, why there has been a decrease in number of PRs last month. The answers respectively are: Bob is the only one to comment  his code so everybody except him is sped up because of it, sprints start on Mondays and fewer PRs have 1 day turn-around so this biases your results. Everything your one tool might lead you to assume has been entirely incorrect.

Or worse, you decide to evaluate engineers based on their Jira tickets. You have engineers estimate their tickets and then see if they can meet their goals on time. Suddenly you have incentivized your engineers to overestimate tickets, play “hot potato” with responsibilities (“Not really a bug! Ticket Closed!”), not help each other (teaching time just makes you look slower), and not invest in long-term architectural improvements that won’t show immediate benefits.

Or you judge the success of a UI change based on a funnel. In an attempt to get the best open-rate for your marketing email you have somebody come up with various subject lines and pick whichever has the highest open-rate. This is an A/B test. Suddenly you’re sending out emails with link-bait titles like “See what your friend posted about you,” that get a high open-rate but immediately get deleted for being deceitful once opened. Or your new button placement gets tons of clicks because it’s too close to the scrollbar.

Or worse yet, you decide to judge the success of your company based on Monthly Active Users. 
You have your awesome facebook game and have 30 million MAUs. You force all of your users to send out invites to all of their friends and Viola 31 million MAUs! You change your code so that every one of your users must send invites to friends every week. Now you have 32 MAUs. The graphs look pretty convincing, you get promoted. And you wrecked the company. A year later facebook changes the rights to your app so that you can’t spam its users, your users have been pressured by their friends to stop spamming invites, your entire company has become a laughing stock in the gaming industry and is now lampooned everywhere for the rest of its existence.

The common pattern here is  individuals trying to substitute a simplistic chart in place of deep industry experience. It is people with finance degrees changing videogame experiences, MBAs insisting on multiple A/B tests to change an icon, or a non-technical hiring manager not calling a Bill Gates for an interview because he doesn’t have a CS degree.

WHY I love PHP Storm

It’s rare I go out of my way to just celebrate something, I usually try to have a balance and nuanced view and don’t like the mindset of attachment. However, I wish somebody had introduced me to this tool sooner. Thus this article is on WHY I love php storm (not that I love php storm).

Saying good things about the IDE is worthless to me if you don’t explain why, and nobody has ever explained WHY php storm is so much better than the alternatives.

Just to help you make an informed decision on how much/little my word is worth, these are the IDEs I’ve used enough to compare against for PHP&JS development:

Aptana, DreamWeaver, Eclipse, Netbeans, Notepad++, Zend Studio


The why:

  • Go to file quickly (without using the mouse), using partial search with wildcards, with an instantaneous dropdown. So I can hit command-shift-N, type public/*user*.js and see a list of matching files, hit down to the correct one, press enter, and have it open, all with negligible latency.
  • Other shortcuts are very useful, easily customizable, like back and forward.
  • I can search my workspace without using the mouse, and it’s very quick, and intelligently stops if it finds >1000 matches.
  • Spell-checks w/ the ability to add to dictionary. Because I’m a good speller this functions as both an error-catch (mistyped a variable name?) as well as a way to make team interactions better (how annoying to use somebody else’s misnamed function every time).
  • The refactors are good.
  • It nails both PHP and also Javascript, in one IDE.
  • It doesn’t crash. I leave it running for weeks, literally. I don’t close the app when I put my computer to sleep, even over the weekend often. Even after weeks it uses less than a gig of ram (no memory leaks), and is still fast.
  • Intelligently looks at doc blocks to infer return type.
  • Can reliably “go to definition” (by keyboard shortcut too) across documents in both PHP and JS.Javascript:
  • Validates JS. Can understand strict mode and ensure your code follows it if used.
  • Understands JSON correctly and validates on the fly.
  • Monitors for global variables and unused variables.
  • Perfectly understand prototypal inheritance for its autocomplete etc.