LONG READ Does luck balance out across the season?

As a result of rugby only recently venturing into the stats age, we still have plenty of widely held beliefs about how rugby works. Some have legs whereas others have a limited life span as facts come in and disprove them. In this article we look at two of the most common assumptions and interrogate them. Which will survive by the end?

Defence wins championships

This is an incredibly widely repeated assumption. Not just in rugby though, pretty much every sport which has a defence component has heard this assumption. Is it true though? Let’s answer it once and for all.

First of all, what does the saying mean? The idea is that defence matters more than attack when it comes to winning leagues. This can be difficult to quantify, almost no team has won the league when seriously deficient in either attack or defence. But we can calculate the quality of a team’s attack and defence by looking at their points scored, and points conceded. If we look across a wide enough time span and with enough data points, we should be able to conclude whether attack or defence wins championships.

 Ranking by Points Scored

Ranking by Points Conceded

*Indicates season impacted by Covid where no winner was crowned. Winner is deemed to be the table leader at the postponement.

The first thing you might be surprised by is how few of the champions finished top of either category. 14 of the 36 teams finished top by points scored, that’s just 39%. The total is lower for points conceded where just 9 of the 36 teams finished top there, that’s 25%. Just four teams finished top of both categories during their championship campaigns.

The average finishing position for our champions by points scored was 2.4 and by points conceded it was 3.2. That implies, along with the data above, that it’s actually attack that wins championships. The team with the best attack wins more often than the team with the best defence and the winning team generally has better attack than defence.

There is more to look into however. For example, you might have noticed that I have included North Lancs/Cumbria. I wanted to include them for two reasons; firstly the idea that defence wins championships should hold true no matter the level and secondly, they have no play-offs.

That play-off factor is crucial. The play-offs allow unexpected winners. In a league with no play-offs, you would expect the best team to win, and that would especially be the case in a league with more matches. We will speak more about that in the second mythbuster below. However, the Top14 play-offs, as it currently stands, allow six teams in. That makes it far more likely that a less good team would win overall. And we see that in the data. On average, Top14 champions have finished in 3^rd place by points for and 4.4^th by points against. Both are the lowest of the leagues we have looked at. North Lancs/Cumbria, which has no play-offs, has an average finish of 1.8^th for points for and 2.3^rd for points against, both the best of the leagues we have looked at. The Premiership, which allows just four in the play-offs, finishes second by that metric.

To answer the question, does defence win championships? No. Actually, attack wins championships. A more boring answer is that good attack and good defence wins championships. An even more boring answer is that good attack, good defence, and playing in a league with no play-offs wins championships.

Luck Balances Out Across a Season

There is a general feeling in rugby that, at the end of a league, the best team will have found their way to the top of the rankings and luck will play an insignificant role. It’s a pleasing idea. We know that upsets happen in individual matches, but surely across the 20 or so matches teams play in a season, we won’t have any shocks. The best team will prevail.

There is a truth to that. The more times a thing happens, the less random chance will play a part. Imagine Leigh Halfpenny challenged you to a kicking contest, but he let you set the rules. You should challenge him to a one kick, winner takes all contest. The more kicks in the contest, the more likely it is that the most skillful kicker would win. It would make sense that the same is true in a league where the more matches lessen the impact of luck. We will see if that is true.

Luck is a hugely emotive topic. Fans like to imagine that luck plays either no part or a very small part in sport. After all, luck by its nature, can’t be accounted for. You can’t sign a player, get a new coach, or move to a different stadium to change your luck. We often confuse skill with luck, and vice versa. One area where this happens is close matches.

Across sports the idea of clutch performances is rife. Certain players or teams can just turn on when they really need to perform. Or so the thinking goes. Unfortunately, as pleasing as that idea is, there’s no evidence that teams can consistently be successful, or unsuccessful, in those tight matches.

Saracens are a great example of this. This year they have had five close matches, those are matches decided by three or fewer points. They have won every single one. Last year they also had five close matches. They lost every single one. This season 23% of all matches have been decided by three or fewer points. Last season it was 24%. That’s almost a quarter of every match where the result is essentially a coin flip.

We can see this by looking at the teams from last season who had winning records in close games. There were five; Bath (3 wins – 1 loss), Leicester (4-1), Saints (3-2), Sale (3-1), and Wasps (3-2). They had a combined record of 16 wins and 7 losses. This year those same teams, although Wasps haven’t featured, have a record of 6 wins and 6 losses. Last year five teams had a losing record; Bristol (1-2), Gloucester (3-4), Harlequins (2-3), Newcastle (1-2), and Saracens (0-5). A combined record of 7 wins and 16 losses. This year they have had a record of 14 wins and 9 losses. If this were truly a skill, we would expect to see those winning teams continuing to win and the losing teams continuing to lose. Actually, sometimes teams continue to win, sometimes they continue to lose. It looks a lot like chance.

Those close game results have a massive impact on the league as well. London Irish have lost all five of their close games this season. If they had won just two of those, they would have six additional points. That would have propelled them easily into the play-offs. Equally, Quins have lost four of their five games. If they had done what Saracens had done this year, and won all five of their close games, they would be in the play-offs.

Close games aren’t the only factor where luck plays a role in who wins a league. But they are a key part. As the length of the season increases the importance of skill also increases. The 162-game long baseball season is hard for a lucky team to win. But a rugby season is short and most have a key element which lets luck back in – play-offs. By putting a tiny league at the end of the season, luck storms back into contention.

We saw that in the 2017-2018 Top 14 season. Castres finished 6^th in the league with a points difference 179 points worse than league winners Toulouse. Then, in the three game shoot out they beat Toulouse, Racing 92, and Montpellier to win the title. Were they the best team in the league? The trophy in the chairman’s office says they are, but the stats say they were somewhere between the 6^th and 8^th best team in the league.

Does this matter? Does rugby lose something by being so luck dependent? I don’t think it does and there’s no way to change it anyway. Unless we find a way to have a 300-game league. Even that might not be quite enough. Crucially though, luck doesn’t decide everything, and it probably doesn’t even decide the majority of league outcomes. But, to answer the rugby myth, no – luck doesn’t balance out across the season.