Sack the Manager Part II

I wrote on here recently about sacking managers, specifically with regard Queens Park Rangers in the Premier League, but it turns out my own team, Oldham Athletic, is also struggling, and as a result naturally has fans calling for their manager, ex-Man City Paul Dickov, to be sacked.

I linked to a paper by three sports economists that looked at data between 1972 and 1997 on managerial tenure and changes, which showed that over that time, any managerial change was associated with a 3-month dip in performance - not necessarily the thing to do mid-season if you're already bottom with only 4 points.

I then had a few Twitter exchanges with both QPR and Oldham fans regarding managerial changes, and both said essentially one thing - their team is different, and/or 2012 is different to 1997.

The great thing about these statements is they are testable. The data exists out there in oodles. We can get information on all managerial tenures from Soccerbase, squads from Soccerbase, and results from Soccerbase (back to the 1800s!) or ESPN.

Data collected, merged and in an Excel spreadsheet, you can get going investigating.

There's huge amounts that could be done with this data that interests economists as well as sports fans, but probably of most interest here is what's the impact of managerial change? There's a variable in the dataset called manager_change, which is 1 if in that match, the manager of the team is different from in the previous match. We'd anticipate that the upheaval from a managerial change would play out over a longer period of time than just one match, so manager_change_1month and manager_change_3month are 1 if the team's manager has changed in the last one or three months.

What we're essentially talking about here is tenure - length of time in a job. The longer, the better? Or, is the relationship quadratic (improving to a point then deteriorating)? There are two variables in the dataset, tenure and tenure2, which allow you to look into that.

I ran a regression, using outcome as the dependent variable (which is 0.5 for a draw, 1 for a win, 0 for a loss), and regressed on tenure, tenure2, and the three managerial change variables above. The output is:

outcome = -.0000828** tenure + 0.0000000589*** tenure^2 - 0.025 manager_change + 0.001 manager_change_1month - 0.089*** manager_change_3month + error

The stars denote how significant the coefficients are (email me if you want the actual output), and they show that there's a U-shaped quadratic effect of tenure:

So it takes a while for a new manager to bed in! These results should be treated with a lot of caution (only since 2001, no other controls for team performance, type of manager separation, etc), but they strike some chord of common sense. It takes at least a full season, maybe two, for a manager to have any effect - in fact the effect plotted above says that for the first four years in the job, the new manager is simply playing catch-up to the point at which he arrived - but after that, the only way is up.

Furthermore, the results above show that in the first three months after a change, there's a further negative impact on top of what's plotted here - a drop of about 9% in the win probability of the team.

Now, the big question we can answer here now is: Are QPR and Oldham different? The way we do this is to add in dummy variables for those clubs. We can interact those dummies with the managerial change variables too, in order to be very careful about whether these clubs are different. And then we can test the significance of these dummies. The results: Oldham are certainly not different - the joint F-test of the two dummies (intercept and slope) is insignificant, with a p-value of 49%, but QPR stake a slightly better claim to "differentness" - the p-value on their joint F-test is 7.7%, close to the 5% conventional significance level we take, but not breaching it.

Is this over-sciencing things? No, it's not - we have terminally short memories, and forget things. Regression techniques like this can take into account every match, every managerial change not just since 2001 but if extended, back to the late 1800s. Every contention you throw at me (things have changed with squad sizes, for example), can be factored in - as mentioned, from Soccerbase we can learn how many players a team fields each season, giving a good idea of how high squad turnover is now, and whether that makes any difference.

The moral of the story is - get out there and play, use the data, and learn what it is telling us. It appears to tell us that the chairmen of QPR and Oldham should hold fire before getting rid of their managers...

Sack the manager!

Having just completed the lectures on the economics of sport, one thing we didn't have time to cover was managerial issues. A hugely common reaction of football supporters (and I don't doubt it's restricted to football) is when a team starts to struggle to call for the manager to be sacked (and failing that, the board too).

Clearly in the workplace if an employee isn't particularly good, it's best if they can be removed and someone more effective put in place. However, it is guaranteed that someone better can be found? Will the disruption be sufficiently small to make it worthwhile?

Does it work though?  The evidence suggests now; this paper by a couple of prominent sports economists, suggests not - in fact in the subsequent three months the team then underperforms. They look at about 25 years of data and find that there's no obvious improvement in the team's performance after a manager is replaced.

The biggest problem, of course, is that we never observe what economists call the "counter factual" - what would have happened had things been different. Hence, QPR fans can complain endlessly about how Mark Hughes has "taken them backwards", yet the fact is we don't know where QPR would be now if they had a different manager in charge since Mark Hughes was appointed.

Into that absence we can inject either some economic theory, or we can try and use data, which is what the paper linked above does. If we can look at enough episodes of teams doing badly getting rid of their manager (and not doing so), then we can see what happens, on average.

It tells us that, on average, it's not effective getting rid of a manager, yet teams persist in doing so...

Interesting post with important lesson

I just came across this blog post on an apparent relationship between obesity in the UK and Premiership revenues.

As the post shows using a scatter plot, the two look impressively highly correlated, and the author points out that the R^2 is 0.93.  It looks like either the Premiership's success causes more obesity, or the more obese people are, the more successful the Premiership is - not exactly the positive impact on health outcomes we might hope for!

However, further down the post, the author plots both series against time and you can clearly see that they are highly non-stationary - i.e. they trend upwards. The technical lesson to be learnt here is that these are two non-stationary series, and hence any strong correlation between the two will almost be erroneous, or "spurious" - i.e. not really there. That's because the regression model doesn't include a time trend and hence as the other variable closely resembles a time trend, it takes that place.

The less technical but equally important lesson is what the blog author emphasises - correlation does not imply causality.  That's a fundamental lesson to always be aware of. Alone, economic data can tell us nothing other than correlation. Only combined with some economic theory can we start to get any sense of causality.