Game Analytics: Build or Buy?

build_or_buy_suga_660w

Deciding to build or buy the components of our game analytics platform is one of the biggest decisions we have to make. I’ve seen many questions on Quora on this issue. Often people ask which is the best analytics platform. From my experience, the only person that can answer that is the person that asked it. Every other opinion is biased either by positive or negative experiences or commercial interest.

The problem is the question. Instead of asking which is the best solution, the question should be which solution best fits my needs. This post is about that. Defining needs that may exist and which should be addressed.

Defining Build or Buy

Building means that you code your solution or parts of your solution. As your game analytics sophistication grows, coding is inevitable. It can go from extracting some data from an API to a full stack of game analytics. Or something in between. Or parts of it.

Buying means that you hire a service. Same as with building, you can buy a part or the whole stack as long as it meets your needs. This speeds things up but raises constraints. Most discussions that I have on the matter of hiring an external service often bring cost as a variable. I believe that is a fallacy because building, but more important, maintaining a game analytics platform also has a cost and often not a so small one.

The biggest difference between building or buying is not the cost but the constraints. When you build you define what are acceptable constraints. When you buy, the constraints are imposed by the design of the service you are considering. I like to define what features I want and then assess the constraints put on those features. The balance between the constraints, the cost to buy versus the cost to build and the return define what to build and what to buy.

I don’t mind mixed solutions where some parts are bought and others are built. In fact, that’s what our BI stack at Miniclip is at the moment and game analytics with parts bought and parts built is no exception.

What parts do you need?

What do you need from your game analytics platform? Reporting? Standard or custom? A/B Testing? Content Management? Custom Events? UA integration? BI integration?

I’ve listed below the standard parts and their flavours so you can make your own personalised list and better assess if you can buy or you should build.

Data handling meaning receiving events, ETL and database administration of the resulting reporting tables. Many platforms have highly efficient workflows to handle data. So efficient they can have the event data available almost in real time independently of the amount of events you send. There are three points to consider and those are the three Vs of big data:

  • Volume: The biggest problem with the volume in game analytics is not how to handle a small or big volume of data. The biggest problem is how do you handle the transition. If you cannot handle big volumes of data ingestion, ETL and database administration and optimisation you should buy, not build, if you are expecting them in the future.
  • Velocity: How fast do the data need to be available for consumption? This is mostly an ETL problem. In my experience, anything shorter than a daily update is rare. However, your games might need an hourly or even real-time treatment. Different platforms have different capacities which present different constraints. Building is as difficult as the velocity you need.
  • Variety: Event data are semi-structured data often in the form of JSON or XML. You may also get structured data from databases and, more rarely, non-structured data like images or text. In my experience, this isn’t much of a concern. Either you take events or you access APIs. What you’ll need to consider is if you’ll need do address different data variety. If you need to address different varieties of data my advice is you consider building and not buying since any constraint in this area is a constraint too many.

Reporting means making data accessible through visualisations. Unless you have the technical capacity of developing in-house visualisation products, buy. The first thing that crosses my mind when I think about reporting is that my objective is to deliver the insight and the technology for this is completely separated from my core skills. Anyone in my team as the skill to create visualisations for analysis or data products but those are different. Our “manual” visualisation are supported directly by the programming languages we use. If I had to build a solution for reporting at Miniclip, I’d probably prefer to expand from that base than to create a reporting tool from scratch.

But there are some caveats to this. The first one is power versus usability. The most powerful and hyped tools often have a burden of usability. You can build beautiful and insightful dashboards, but the regular user can’t, regardless of the marketing of such tools. What you need to ask yourself is if you really need that power? Do you really need your business users, meaning producers and stakeholders, to deeply slice data? I’ll go as far as asking, do you want them to considering the risk of incorrect conclusions being drawn? This is a fine question for another post…

The second caveat is if reporting is fixed or customised. Can you create your own custom dashboards thus reflecting your games, company and objectives? Or standard dashboards with fixed metrics? The obvious questions seem to be custom dashboards, but that might not be the case. There are analytics platforms with well built main dashboards and the data is still accessible for visualisation and drilling.

Last but not least tools and integrations. You can buy or build them individually and there are analytics platforms that have all or some or none. If your decision is to buy don’t choose a platform for what it might do. Choose a platform for what you can do with it. Below is a list of tools and integrations that you should tick or cross. That will help you when choosing individual tools or analytics platforms.

  • A/B Testing is a major power tool and one that is not simple to put up. Some platforms offer great A/B Testing tools and there are individual A/B Testing alternatives. Building an A/B Testing tool is not for the weak of heart and you should know design of experiments really well before following the building path. One thing that you should always check is how the A/B Testing reporting is done and depending on your level of sophistication, if you can query user and events data to do your own analysis.
  • Content Management Systems are often the foundation of A/B Testing. The reason for this is that after you make a decision, the message with the winning group must be passed to all your users. But that is not all CMS do. If you rely on a configuration to add content or change variables in your game, it is likely that you will benefit greatly from a CMS. It is fairly common that analytics platforms support CMS if they have A/B Testing.
  • Programmatic Access to data is important if and only if you do data analysis or you want to connect other tools to your databases. This integration is only relevant if you are buying the service of handling your data, which you will be if you hire an analytics platform. If you are building it this is available out of the box.
  • Custom Dimensions and Reports is a bit of an elaborate one. The problem is if you can or cannot cross user or event dimensions to produce reporting tables or datasets. It is not a problem with visualisation, it is a problem with data handling and only if you buy the data handling. Imagine this question on a level based game: what is the distribution of players per level per retention day? There are two different dimensions (level and retention day) that are obtained from two different events (play level and session). If you have questions like this you need to understand how the service you are buying handles it.
  • Integrations with External Sources usually UA and advertising. There might be others but these are often the ones that get more attention. If you need these integrations your service should allow direct integration. There are other ways but it is much cleaner when server to server communications is established.

This post was a record breaker in number of words which means I wasn’t able to explain it shortly. My apologies for that, I prefer shorter posts. My hope is that despite the length it gives you a good insight on how to decide to build or buy and what to look for while making the decision.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s