Applying machine learning to programmatic buying in a simple way
0  |  24. maj. 2017 af Anders Munkesø Kjærbøll
Data, Medier, Tech

Machine learning (ML) and Artificial Intelligence (AI) have been trending topics for a while now, and a lot of milestones were surpassed in 2016 where the perhaps most notable achievement was made by the Google owned company Deepmind and their AI AlphaGo which beat the world 2nd ranking Go player in the game that many AI experts beforehand would have predicted not to be possible before 2025. Along with this development has arrived data science methods to train algorithms such as “neural networks”, “random forest”, “deep learning” etc. It might seem overwhelming with all these buzzwords flying around, so how can brands start applying this advanced discipline in their media buying as a start and what is readily available to build upon?

First we need a problem that ML can solve. Let’s imagine that you arrive in Las Vegas and is given $1,000 to play the one-armed bandits for. You are only in town for 24 hours, and want to maximize your return within that timeframe. When you arrive at the one-armed bandits arcade it becomes apparent that all the machines have different probabilities of winning and different pay-outs, so you are faced with a lot of unknowns. In ML this is referred to as the multi-armed bandit problem where the gambler faces a dilemma in a trade off between spending money exploring new one-armed bandits and exploiting those that are found to provide an acceptable return. This explore/exploit tradeoff is also faced in “reinforcement learning” which was the method used to train AlphaGo and is widely used to train ML models.

If you think about this scenario and how it translates to media buying, you could look at the top 100 publishers in Denmark and list their top 1,000 banner placements which represents the one-armed bandits. From here, you can define a goal to train your buying algorithm; that $10 can be used to “explore” each placement and if this investment leads to a number of clicks (or even better, a conversion) then the placement is exploited further. This way, the buying algorithm will constantly discover well performing banner placements and exploit them while still learning from exploring new placements. Programmatic media buying is of course much more complex for several reasons: 1. Inventory fluctuates due to supply and demand 2. new content is constantly generated by the publishers 3. frequency needs to be capped, and the list goes on. But in general the explore/exploit approach is still valid to build upon if the algorithms are set up optimally and nurtured along the way.

When an explore/exploit algorithm is launched it will rely heavily on spending money exploring placements and thereby have poor performance, but as good performing placements are discovered and exploited, the performance will increasingly improve. Here is an example from an algorithm where the split in media spend between explore/exploit is plotted over the course of 4 months:






As the split between investing media spend in exploring/exploiting banner placements shifts over time, so does performance where the next graphs shows the volume of conversions that are being generated daily by the buying algorithm and the accumulative “cost-per-conversion” (CPA) which is constantly decreasing in this example.




And so, ML has optimized programmatic media buying.

From a programmatic campaign manager’s perspective, this development could look like the machines are taking over which to some degree is the case. Yet a lot of training and testing is currently required to reach this result, where the human interference is still needed. New issues arising during a campaign flight that requires human decision making could for example include accelerating the exploit development by forcing the algorithm into this state for short campaign bursts or to increase the scale of the algorithm when performance goals are being met. In GroupM we have worked for 1½ years with adding custom decision trees on top of the algorithms which is used for further fine-tuning which require additional decisions from the campaign managers and data scientists. On top of this, issues with brand safety, fake news etc. are difficult to incorporate in the model which leaves out plenty of other decisions.

So in short, the point with this blogpost is that any brand can start using ML tomorrow by taking initial steps, whether ML is used to create a chatbot, a voice recognition app, or a media buying algorithm – people will still be needed.


Del denne artikel på de social medier