Hi folks, I'm the author of the post — I wrote it because I think a lot of brands tend to pluck numbers out of thin air when they're talking about marketing targets, without knowing how they're affecting their bottom line.
This might seem like econ 101 as zwaps mentioned, but there are a lot of brands who don't take anything like this approach still.
Interesting post. "All models are wrong, some models are useful" - Box. This one seems useful and is simple to explain.
It started me thinking about the distribution of customers who convert, viewed as a function of ad spend. I am not sure there is any reason this distribution need have a particularly simple shape, or even be continuous. E.g. maybe some customers can be addressed and convert at a modest ad spend, while if you double the ad spend you don't get any more conversions, then if you double ad spend again maybe suddenly you're out bidding a competitor and the number of conversions shoots up, perhaps giving you a better overall net profit than if you stopped earlier with a modest budget.
This might mean that the curve we're trying to maximise (net profit) has more than one local maxima, or might not even be continuous.
There's probably also an explore/exploit tradeoff here as well: how much of the total budget should you spend sampling to try out different ad spends across the whole range of plausible values (from 0 up to the long term value of a conversion, I guess) to get enough data to start optimising.
That is a fascinating line of reasoning. I wonder if there is some way to leverage historical data (for other brands, maybe?) to figure out potential local optima.
I original thought this was a nice "set it and forget it" marketing scheme, but your comment makes me think otherwise.
If so, great question. I haven't looked into this much but wouldn't expect it to be parabolic - that was just an approximation to illustrate the points raised.
Alright, makes sense. I guess a parabola is the primordial (at least, to high school teachers) example so I understand why that is what you picked.
In terms of whether a parabola makes sense for the example, it's difficult for me to say. From my personal experience I would say this could maybe be done with differential equations and something akin to an R_0. Then you would also have steady states, but the advantage would be that you can actually see how your other parameters (coefficients in the differential equations) influence the conversion rate (in my analogy R_0).
Really, you're going to use a logarithm because it fits one data point better?
Besides you might as well just use all your datapoints directly. There's no real need to interpolate between them (and if you really want to optimize your ad costs that finely, don't use a function that predicts -infinty gross profit if you don't use ads).
> The graph above features a linear line of best fit. Clearly we can see above that the data isn't linear, and a linear line of best fit doesn't make sense
I think the reason for not using a linear fit was that it "doesn't make sense", but the reasoning is not given. It would be interesting if the article explicitly said why it doesn't make sense, and why the somewhat arbitrary choice of the log function does. As you point out, gross profit will not be massively negative when there is 0 ad spend.
By my eye, the gross profit as a function of ad spend looks approximately linear in the region where data was sampled: say gross profit = 400 + 0.5*ad spend . If you plug that in & then optimise for total profit the most profitable non negative choice of ad spend is of course zero!
So in this case a structural modelling assumption that isn't well explained or justified (log vs linear vs any other function) has a very large impact on the answer. It seems like we're trying to maximise a function in a region where we don't have observed data.
Since the problem as given is data poor and modelled in a way that is trivial to compute, perhaps it is a reasonable candidate for: running some more experiments to get more data points ; or using a statistical method that can express the uncertainty of our structural modelling decisions & parameter estimation (e.g. a Bayesian analysis starting with a prior distribution of possible fits over a richer class of functions with plausible behaviour near zero) since we don't have a physical theory that justifies a particular functional form of the assumed shape of the relationship.
Fun reading for a university class, but the real world doesn’t work this way. Optimizing for a target cpa is never a possibility because at an agency or in house marketing team your given x budget and expected to spend it.
I’ve never seen someone underspend a budget and be thanked for it.
Indeed. I would like to see further reading on the methodology of adjusting your spend to maintain adequate levels of conversion while hitting your budget.
Of course, many agencies can quite quickly find places to spend money beyond AdSense :)
Exactly, this appears effective projecting the fixed budget for one specific tactic, on a product with geo or demo targeting limitations.
The blog post failed to mention that when scaled up there is much more in play than strictly consumer acquisition (e.g. brand awareness) that is just as important when marketing in most B2C verticals.
You're right that there's more than just last-click cost per conversion, but that doesn't mean this approach is incompatible with these extra complexities.
You might run conversion lift tests as a way to calculate the impact of increased brand awareness, and you could plug the data from these into the method outlined in the post. There you'd be looking for the optimal cost per incremental conversion rather than just optimal cost per conversion.
This is just one idea, hopefully shows that the post's method doesn't have to just use last-click conversion data.
While true, this kind of analysis would still be useful for the allocation of that budget. You should still aim for maximum profit when you actually can calculate that value and dump the rest of your budget somewhere else that makes sense but is much harder to quantify (market research, brand marketing).
In any case, my experience is that "in the real world" is code for just giving in to systemic inefficiency and purposefully mismanaging your resources.
This is a good point, and you're right that some brands work this way.
I've worked agency and in-house, and most of time I've worked without budgets, just trying to maximise volume at a particular CPA. More brands are moving this way, but a few do still just stick to fixed budgets.
Contrary to the dominant narrative of this Hacker News post’s comment thread, I find this mathematical modeling exercise to be rather informative and inspiring. Life sustaining. It can be considered practice that keeps the vital muscles active.
How more convoluted can this explanation be?
The question is equivalent to just "Find the maximum of nb_conversion × (cost_per_conversion - margin_per_conversion)", and the only interesting question is the relationship between the cost per conversion and the number of conversions, which is not a matter of math but of practical statistics.
Much more easily, instead of plotting profit versus ad spend, plot (profit - ad spend) vs ad spend. Choose the highest point. If it looks like there might be a higher point not covered by your points, like how in OP it looks like the cheapest ad spend is the best, try going even lower. Instead of extrapolating, test. Extrapolating is much harder than interpolating and often requires some causal or structural understanding.
This might seem like econ 101 as zwaps mentioned, but there are a lot of brands who don't take anything like this approach still.
Keen to hear people's thoughts :)