Game analytics can be very simple or go wide, far and deep. The trick is to define what it is that you will want on a given timeframe. The length of the timeframe depends on how sophisticated and complex are your objectives.
This post will go through the role that sophistication and complexity take in defining both your objectives and the analytics stack to support them.
The role of sophistication and complexity
Sophistication is what you are able to deliver with your analytics. This means that reporting retention may be your next big step. Or maybe reporting retention is “business as usual” and you are focusing on things like machine learning algorithms to predict the log odds of a player churning. More sophistication is a sign of evolution, but it also means that we already handled the smallest things. The time to produce more impactful insights and data products also increases. Those insights and tools are likely to step up the sophistication a notch further. It is an improvement feedback loop that grows in time at each iteration.
Complexity is the technological facet of sophistication. You’ll need more tools or integrations from these tools to achieve new objectives. You can lower complexity by hiring the services you need thus increasing implementation velocity, often at the expense of the constraints by design which I will discuss on later posts.
Baby steps first, monumental flights later
This should be obvious, but I did the mistake of aiming for the moon… while riding a donkey… with a flat tire. Sorry I’m well known for dumb nonsense (and often failed) jokes and it is easier to make fun of me when I’m exposing my own mistakes.
Once upon a time I had a brand new analytics platform to show around. Which I did. I could crunch millions of events in an hour or so. Productivity, reporting and depth of insights improved in a couple of weeks. One of those insights was about the progression of players through the difficulty ranks of one of our games. Decisions were made, retention, engagement and revenue grew.
The data were wrong. Not because of the platform but my experience with it. All is well when it ends well I guess. It did end well because the problem was correctly identified and it affected a fraction of players. However neither my knowledge of the platform or my analysis were correct. On another context, it could have been catastrophic.
We will always be missing some piece of the data science puzzle on a fast-moving industry such as game publishing. Sometimes is knowledge, if you are like me, there’s always one more book to read, one more course to do and one more podcast to hear. Other times it is technology. Either we don’t have it or, like the case presented, we don’t fully dominate it.
That is why our steps need to be firm and our objectives clearly defined and clearly communicated. Nothing affects our recommendations more than lack of trust by decision makers. For that reason, don’t plan for machine learning if you don’t have organisation wide reporting or you’ll be shooting for a monumental flight before you can walk.
The biggest problem with this is that anyone can fit a powerful prediction model with one line of code and get a prediction with one other line of code. It seems so easy, why not doing it? Same for A/B testing… it’s so obvious how it works. These are just two examples of things that seem really easy to do and aren’t.
Baby steps first, trust me on this.
The three types of analytics
When I say baby steps I’m referring to the first type of analytics. These are called descriptive analytics. In a nutshell, this is reporting and if you have the skill, analysis. You can’t act on data if you don’t see where the problems or opportunities exist. This is achieved by describing the past, hence the name descriptive.
The second type of analytics is predictive analytics. This is about the future. What will happen and what is our confidence of that happening. Statistical learning, A/B testing and other forms of randomised controlled trials fall in this category.
Last but not least, the third type of analytics: prescriptive. These are about action based on prediction models. This is the home of machine learning just to name the buzz word of the moment.
Other things fall in these three types, but this should give you a clear view on how sophistication evolves. Which leaves us only with timeframes.
Timeframe as a function of current state
Based on my experience timeframes length depends on what stage you are at.
Plan for 6 to 12 months in the early implementations of your analytics stack. Think of it as the descriptive stage. These first steps are mostly about reporting and should (I’d go as far as say must) include retention and The Holy Trinity of Monetisation. The 6 to 12 months is not really the time it takes to implement unless you are building it. That will be surprisingly fast if you hire a 3rd party analytics platform! Most of the time this early stage takes is to get used to the data overload if there’s no data analyst or scientist with a background in game development around. Things will be very confusing until the dust settles. There will be some growing pains, mostly technical. The biggest I faced was to be sure that revenue was correctly reported.
As you get more experienced and reporting becomes part of your day to day routines you’ll begin to look forward to what you might achieve with data. These next steps can be about many things and they depend on what itch you want to scratch. Overall you can think of it as descriptive analytics on steroids and as much predictive analytics as you need. Engagement, A/B Testing and data analysis were our focus at Miniclip but my advice is that you focus on what brings value to the organisation which can be integration with external data sources (UA and advertising come to mind), real-time analytics, segmentation, etc. Unless you have a lot of resources, you won’t do it all. Define your objectives, prioritise them and implement them in the next 12 months.
At this point in time you will need knowledge beyond basic statistics and game development, production and design. Depending on what you want to do, data analysts, scientists and engineers may be the most important resource you’ll need to plan for. You should have data analysts or scientists already if you are doing A/B Testing or any other form of randomised controlled trial.
From this point forward things will be pretty sophisticated and complex. Maybe you want your A/B Testing on steroids (I know we do, you’ll learn why soon). Maybe you want predictions and recommender systems for personalised user experience and offer. Maybe you want a full front end self-reporting revamp. Maybe you want to do research. Whatever it is, do as many as your output or your team’s output can provide with projects that span no longer than 6 months. This is all about enhancing all types of analytics there are and the only place where prescriptive analytics should be implemented.
Maybe I’ll have a different view in a year but where I stand I would say that everything in this third stage is a data project. At this point in time your studio, your games and yourself will be deep into data science and depending of your games exposure, big data. You can see what your game analytics team and technology stack can provide. Things that probably will solve problems that no one else knows that can be solved by you.
The last section outlined several stages of sophistication. It does not take too long to reach it as long as your foundations are strong. Don’t prescribe if you can’t predict, and don’t predict what you can describe. If you follow this simple mantra, you’ll be fine.