Sack the Manager Part II

I wrote on here recently about sacking managers, specifically with regard Queens Park Rangers in the Premier League, but it turns out my own team, Oldham Athletic, is also struggling, and as a result naturally has fans calling for their manager, ex-Man City Paul Dickov, to be sacked.

I linked to a paper by three sports economists that looked at data between 1972 and 1997 on managerial tenure and changes, which showed that over that time, any managerial change was associated with a 3-month dip in performance - not necessarily the thing to do mid-season if you're already bottom with only 4 points.

I then had a few Twitter exchanges with both QPR and Oldham fans regarding managerial changes, and both said essentially one thing - their team is different, and/or 2012 is different to 1997.

The great thing about these statements is they are testable. The data exists out there in oodles. We can get information on all managerial tenures from Soccerbase, squads from Soccerbase, and results from Soccerbase (back to the 1800s!) or ESPN.

Data collected, merged and in an Excel spreadsheet, you can get going investigating.

There's huge amounts that could be done with this data that interests economists as well as sports fans, but probably of most interest here is what's the impact of managerial change? There's a variable in the dataset called manager_change, which is 1 if in that match, the manager of the team is different from in the previous match. We'd anticipate that the upheaval from a managerial change would play out over a longer period of time than just one match, so manager_change_1month and manager_change_3month are 1 if the team's manager has changed in the last one or three months.

What we're essentially talking about here is tenure - length of time in a job. The longer, the better? Or, is the relationship quadratic (improving to a point then deteriorating)? There are two variables in the dataset, tenure and tenure2, which allow you to look into that.

I ran a regression, using outcome as the dependent variable (which is 0.5 for a draw, 1 for a win, 0 for a loss), and regressed on tenure, tenure2, and the three managerial change variables above. The output is:

outcome = -.0000828** tenure + 0.0000000589*** tenure^2 - 0.025 manager_change + 0.001 manager_change_1month - 0.089*** manager_change_3month + error

The stars denote how significant the coefficients are (email me if you want the actual output), and they show that there's a U-shaped quadratic effect of tenure:

So it takes a while for a new manager to bed in! These results should be treated with a lot of caution (only since 2001, no other controls for team performance, type of manager separation, etc), but they strike some chord of common sense. It takes at least a full season, maybe two, for a manager to have any effect - in fact the effect plotted above says that for the first four years in the job, the new manager is simply playing catch-up to the point at which he arrived - but after that, the only way is up.

Furthermore, the results above show that in the first three months after a change, there's a further negative impact on top of what's plotted here - a drop of about 9% in the win probability of the team.

Now, the big question we can answer here now is: Are QPR and Oldham different? The way we do this is to add in dummy variables for those clubs. We can interact those dummies with the managerial change variables too, in order to be very careful about whether these clubs are different. And then we can test the significance of these dummies. The results: Oldham are certainly not different - the joint F-test of the two dummies (intercept and slope) is insignificant, with a p-value of 49%, but QPR stake a slightly better claim to "differentness" - the p-value on their joint F-test is 7.7%, close to the 5% conventional significance level we take, but not breaching it.

Is this over-sciencing things? No, it's not - we have terminally short memories, and forget things. Regression techniques like this can take into account every match, every managerial change not just since 2001 but if extended, back to the late 1800s. Every contention you throw at me (things have changed with squad sizes, for example), can be factored in - as mentioned, from Soccerbase we can learn how many players a team fields each season, giving a good idea of how high squad turnover is now, and whether that makes any difference.

The moral of the story is - get out there and play, use the data, and learn what it is telling us. It appears to tell us that the chairmen of QPR and Oldham should hold fire before getting rid of their managers...