In our latest Developer Economics survey we were really surprised to see a dramatic fall in the level of adoption of User Analytics tools. A year ago they were the most popular category of third party tool with 38% of mobile developers telling us they used them. This time, only 21% of developers said the same thing. Why has there been such a significant drop? Are the hordes of new developers flocking to mobile just unaware of what’s available? Is all analytics being tainted by the spying scandals? Why does it matter?

The new mobile developer tsunami

At WWDC this June, Apple said that they’d gained almost 50% more developers in the last year. The vast majority of those will be iOS developers (rather than OS X and Safari). Android has been growing even faster in terms of new apps, so it’s likely there was even greater growth in the mobile developer population there. Windows Phone also saw significant growth in developer adoption. With so many new developers getting started building mobile apps we’d expect that they’re not all aware of the tools available to help. However, the reduction in user analytics adoption was greater than for other tools, indeed some tools categories saw increased adoption. So, although lack of awareness is clearly a contributing factor it’s not the whole story.

Tracking behaviours, not people

Information on government spying activities leaked by Edward Snowden has turned sentiment amongst many of the technically aware against digital surveillance. Perhaps some developers feel they shouldn’t be spying on their users’ actions? [tweetable]Does adding analytics to your app violate user privacy? Not usually.[/tweetable] User analytics tools are primarily for examining correlations between user behaviours and changes in the product or its marketing. If you make changes to a sign-up screen you need to see if it improves the fraction of users who sign up. If you’re trying to convert free users into paying users, it’s vital to analyse which changes in design or pricing result in greater revenues. Even to improve a paid up front or entirely free app, understanding what users do when they’re using it is incredibly valuable. As long as you’re not collecting personally identifying data and associating it with activity then it’s hard to see this as spying. Even segmenting users for behavioural targeting of functionality or advertising seems harmless as long as they’re anonymous. When analytics providers aggregate data across apps to build up user profiles for advertising, that may cross the creepy line for some. If this bothers you then choose an analytics provider appropriately, or collect your own data. If it doesn’t then having a privacy policy that clearly states who sees what data and what it’s used for is the responsible thing to do.

Coding blind

[tweetable]Updating and iterating on apps without analytics is apparently incredibly common[/tweetable]. It’s unlikely that all of these developers are regularly running user testing panels to get detailed feedback. Is everyone coding blind? To avoid constant app updates which would annoy users (and not even be possible with the review cycle on iOS) developers need to include multiple changes in every new version. If user sign-ups or revenue goes down after an update, how can you tell what went wrong? If things improve, which changes helped? Which parts of your app do people use the most? Are you working on improving features that no-one is using anyway? Without some kind of analytics, how do you find the answers to these questions? Might you be wasting a lot of effort?

Just integrating a third party analytics tool isn’t enough to answer these questions either. Apps need to be instrumented to log analytics events for every major action in the application and how long users spend on various activities. Beyond that, to really understand the effects of changes, users need to be analysed as cohorts – groups that started using the app around the same time. The value of an app to its users may increase or decrease over time and users who have got used to a UI are more likely to be unhappy with any changes. For an app that’s just starting to grow a user base, a change that causes it to lose 10% of existing users might still be positive if it increases revenues from new users. To know the real effects of some changes, it’s necessary to analyse specific sets of events as funnels. For example, in a registration or purchase process, increasing the number of people that start the process is no good if the number that end up completing it is reduced because they feel misled.

Tool choices

Developers that decide to embrace analytics have a lot of choices, however, the market is quite polarised. By far the most popular tools are Google Analytics and Flurry. These are (mostly) free. Google Analytics has the advantage with Android developers because it is integrated with Google Play, allowing direct tracking through the acquisition and download process into usage. It does have a (very high) limit on usage. If your app gets incredibly successful you’ll have to use the premium tier, which has a flat annual fee of $150k. Flurry on the other hand is free at every scale, although not as advanced as Google Analytics in terms of segmentation, funnels and visualisation of data. Both services are collecting data across apps, effectively to sell to advertisers.

There’s also a premium-only market with better support for cohort analysis, funnels, segmentation and visualisation. Paid offerings also link the analytics direct to actions, allowing automated real-time messaging to users, or the sending of push notifications to specific groups. Mixpanel and Localytics are two examples of tools in this space. These premium tools are also designed to be a lot more usable by non-technical staff, allowing a marketing team to analyse and react to analytics without always needing help from the development team. These tools have a fairly low user or datapoint count for a free tier and then start charging hundreds of dollars per month for successful apps. However, this means that they don’t sell the data from your app to anyone else.

A third option is the relatively little known Amazon Mobile Analytics solution. So far it’s relatively basic on the analysis front and doesn’t have the prettiest interface. However, you can collect 100 million events per month for free and beyond the free tier it’s just $1 per million events. By contrast, Mixpanel charge $2,000 per month for 20 million events. Amazon don’t share your data with anyone or report on it (no word on whether they use it internally for their own purposes).

Give it a try

Thinking about your apps as places to experiment and measure how the users react takes a change of mindset. If you’re not just building apps for fun then this shift can make a massive difference to your success.

If you’ve had a major change in results because of something you discovered using analytics, share your story with us in the comments.