When the NBN was first floated one of the primary goals was to “level the playing field”. Telecommunications usually favours the large player due to the amount of capital required for infrastructure. For 20 years Telstra called pretty much all the shots in the Australian telco industry and played a game of cat and mouse with it’s arch-nemesis the ACCC. Telstra was famous for favouring its own retail division over any of its wholesale customers for example. While NBN (along with the structural separation of Telstra) has done much to level the playing field, there have been a number of smaller, apparently less significant, decisions that seem to be going in the opposite direction.

Phil Britt of Aussie Broadband has publically criticised a proposed extension of the time before NBN will force telcos to switch off their old “special services”. Firstly I would like to point out I fully support Phil’s position, but I would like to add some meat onto the bones. Yet again we see the big telcos manipulating NBN to their own ends at the expense of Australia.

So why should you care? If your NBN is delivered over FTTN (Fibre-to-the-node) technology, which is the case for the vast majority of Australians, then a delay here will mean that you will have slower, less reliable internet for longer.

One of the downsides of FTTN is that it uses the same copper that is also used for phone lines, ADSL and also these “special services” – predominantly a phone technology called ISDN used by many businesses. Unfortunately when you run two copper cables next to each other, there is a tendency for the signal to jump from one cable to another – this is called “crosstalk”. The higher the frequency of the signal, the more that this is a problem and FTTN uses very high frequency signals. This crosstalk causes interference.

During the migration period when people are both using the old technology and the new FTTN technology (called in NBN parlance “co-existence”), to reduce this interference NBN have reduced both the frequency range and the signal power of the FTTN nodes. Because of this people have reduced speed and a less reliable internet connection.

Under the original FTTP rollout, there was an 18 month migration period when all the regular services (phone and ADSL) were forcibly migrated across to the NBN. There was actually no technical reason for this (copper and fibre run just fine side-by-side), it was purely a political and economic one. At the time the remaining copper based “special services” were the can that was kicked down the road to be dealt with later. However under the FTTN technology a decision to delay has real consequences.

Under FTTN the same 18 month migration period exists for phone and ADSL. We were all looking forward to the co-existence period ending after 18 months and people’s connections being turned up to full power, however because of these special services NBN still hasn’t turn off the “co-existence” mode at the node and is now talking about delaying it again.

So why would the big telcos be requesting a delay on this? As usual it is all about money. Firstly a telco typically earns more money on the old legacy services than the newer NBN ones. Secondly a telco has to engage with their customer and help manage the process for migration, this of course costs money. Thirdly whenever there is a network change, it typically is a time when a customer will re-evaluate its choice of supplier.

Indeed the biggest barrier to competition in the telco space is sheer customer inertia – if a service is basically working most customers are, quite reasonably, nervous to make any changes. This is unfortunately due to their experiences with the major telcos who regularly stuff things up during migrations.

The large telcos would very much like their customers just to do nothing and keep buying from them. However they are doing this at the expense of the quality of the internet for the rest of us. They have had plenty of time (years) to plan and execute this, if they are still not ready, then clearly they don’t want to be ready and are using this as yet another way of keeping the more nimble challenger telcos off their turf.

Damian Ivereigh
CTO Launtel
19/09/2018

Summary: By manipulating who ends up own the infrastructure upgrade we can create a model that significantly incentivises people to perform a TCP upgrade earlier rather than later. Thus accelerating Australia’s eventual upgrade to FTTP.

It is fairly well known that the take-up of NBN’s Technology Choice Program has not been that great. When Malcolm Turnbull proposed his Multi Technology Mix model he said that those who wanted full fibre could upgrade for “a few thousand dollars”. It hasn’t turned out that way with most upgrades costing over $10,000 and many significantly higher than that.

So why is it so much more expensive? The reason is that NBN have used a “cost recovery” model, where anyone who asks for an upgrade has to pay for all the upgrade costs to the entire infrastructure needed to deliver FTTP to that person’s premises. In the early days, that even included purchasing an expensive OLT device – about $100,000 – that sits in the hub (often the old exchange). This OLT is capable of servicing hundreds of subscribers. We had an upgrade quote come back to us of over $120,000 because of this. They have since rolled out OLTs so thankfully they are no longer charging for this.

Assuming that this upgrade proceeded, the next person, maybe a neighbour, to upgrade would then be able to pay nothing for that infrastructure and just pay the extra bits that was needed – a significantly reduced amount. It is understandable why NBN did this, they want to make sure that any FTTP upgrades were at worst cost neutral to their budget.

This creates a significant disincentive to go first, a “first mover disadvantage”, which mean that few people want to go ahead unless they had a very high motivation. Unfortunately due to NBN’s high pricing of the higher speed tiers combined with a quirk in the CVC model, few RSPs actually offer speeds above 100Mbps (with the exception of us in Tasmania). So the speed motivation is not there, there are others like reliability and latency, but this often isn’t enough.

NBN in their latest pricing review is proposing some changes that will make offering the higher speed tiers (above 100Mbps) more financially viable for RSPs – in particular a cheaper 1000/50 speed tier. So this particular problem may be about to be fixed.

However what about the first mover disadvantage problem? Could we turn the problem on it’s head and actually create a first mover advantage and still not put any pressure on NBN’s finances?

Creating a first mover advantage

Suppose after an upgrade, the purchaser gets a certificate that details the upgrade performed and the costs of every segment of that upgrade. That person now “owns” all the parts of the upgrade. If another person comes along asking for an upgrade, that purchaser must buy all those shared parts from the previous owner – mediated by NBN. The new upgrader now owns those parts that were used in the new upgrade and is given a new certificate.

This would not require a significant amount of work, because NBN already keeps careful records of the parts of an upgrade and indeed many of the costs are standardised with their contractors.

The net result is that while the first mover still has to initially pay for all the upgrade costs, the quicker they do it, the more likely they are to be able to get a substantial portion of the upgrade costs back from other upgraders. Indeed the last person to upgrade would get landed with all the costs of the shared infrastructure. Of course there is nothing to stop the first upgrader encouraging the future upgraders by giving a discount on those costs. It would after all be a free market.

You could even see RSPs or other bodies financing these upgrades in return for receipt of the certificates. Banks could loan against them or add them to the mortgage. Consortiums could be formed that would pay for and receive all the certificates.

There are some variations on this that would allow each subsequent upgrader to pay a share of the costs rather than the full costs, but that would change it to being a neutral advantage. By making the default be “all the costs”, any variations can be done by individual negotiation. The important change that NBN have to agree with is to not allow subsequent upgraders a free ride on the earlier upgraders. The later upgraders must buy those certificates first.

Another minor variation is that NBN pays a lease payment for that infrastructure to whoever owns it (has the certificate). They are after all getting extra revenue from the higher speed tiers. While this may be a lot of process for a relatively small amount of money, it may be enough to partly defray the interest costs of the owner and further encourage takeup.

At any stage NBN could decide for operational reasons that they want to actually own the infrastructure (rather than paying lease payments), in which case they have the right to buy back those certificates at face value.

I believe this should be an attractive proposal for NBN. It ticks the boxes of not costing NBN any more money, turbo charging the inevitable FTTP upgrade that Australia needs and of course is essentially a free market approach that fits with the neo-liberal agenda that both sides of government seem to identify with these days. 

Let me know what you think.

Damian Ivereigh
Launtel

There has been some controversy about an announcement by Seven Networks to broadcast the Japan Olympics in 4k over Optus’ 5G network. The controversy being that they will only be offering 4k through Optus.

Optus is clearly using this as a way to drive customers onto their new 5G network and have at least taken an implicit swipe at NBN and its lack of performance.

Many people are upset that they will be unable to access this streaming due to the lack of 5G coverage or that they don’t want to buy the appropriate equipment to allow them to receive this.

So was this a good decision by Seven? I believe it is, however the reasoning will surprise you. It does not mean that 5G is some great network that will beat NBN. The reason goes to the heart of a growing problem in the internet live streaming industry that unless solved is headed for a train wreck.

It is worth remembering that Optus has already had a big failure when they sub licensed some of the Soccer World Cup games from SBS and attempted to broadcast them over their mobile network. It was a massive failure, even triggering the then PM, Malcolm Turnbull, to put a call into the CEO of Optus.

So why did it fail? Optus completely underestimated the demand, indeed given the scale of the failure involved I actually wonder if anyone in management & marketing even consulted the IT and Networking engineers to ask about capacity.

With any streaming you always have to budget to send a separate copy of the video to every single person who asks for it. If a lot of people are asking for it, this traffic quickly multiplies up. So if a video streaming in HD (using about 5Mb) is being requested by 100,000 people that amounts to 500Gb of traffic. Now with appropriate banks of servers in datacenters that is quite doable. When sending pre-recorded video (youtube, Netflix etc) you can also play some tricks like send the data in bursts and allow the receiver to store (“buffer”) many seconds of video ahead that can make up for any slowdowns or congestion (i.e. render them invisible).

However live streaming creates a number of unique challenges. Firstly depending on your definition of “live” there really can be no prebuffering – otherwise you would be delaying the video (no longer “live”). This means that the servers must generate a very smooth stream of data at 500Gb, any congestion or delays and the video stops or breaks up, leading to a very unpleasant watching experience. Secondly players will get out of sync (maybe the user paused it briefly) with each other so that each stream is being sent slightly different data. That position in the stream has to be tracked for each individual user, more computation and data fetching power required in the servers.

I don’t know the number of subscribers to the Optus Soccer, but clearly either the servers being used or the network delivering the data could not keep up.

This is all bad enough in HD, but if you bump that up to 4k (which uses 25Mb per stream) and now instead of 500Gb of traffic, you now have 2.5Tb of traffic to support those 100,000 users.

So it is highly possible that Seven were actually staring down the barrel of not offering 4k at all: they just could not justify the expense of the server hardware required (even when rented – which is what content companies such as Akamai do) to supply the 4k streams for the two weeks of the Olympics. It is also extremely hard to test these things – it’s not like you can corral 100,000 users in for a system test!

So if Optus came to them offering to do it over 5G there are several distinct advantages for Seven: Firstly it would mean they could say they are actually streaming in 4k. Secondly it will be to a limited audience (due to the lack of 5G coverage), thus limiting any issues with capacity. Thirdly and this is most important: they will likely use a technology called “Multicast”, which is possible over 5G.

Multicast

So what is Multicast? As I explained above the problem with internet streaming is that it requires the data to be individually sent to each user who wants to watch the stream, every user sets up a new session all the way back to the server. This is called a “Unicast”.

If you compare this with the traditional (“over the airwaves”) broadcast TV, the transmitter just sends a single copy (stream of data) everywhere and each receiver just passively receives the signal. There is no extra load put on the transmitter just because someone new decides to start watching.

In attempting to duplicate this behaviour over the internet, some years ago the internet engineers came up with a concept called Multicast. To understand consider that the internet is built a bit like a tree, with the transmitting server at the base of the trunk and the users at the leaves. What Multicast does is allow users to register with the immediate branching point below it (a router) that it wants to receive the streaming traffic. If that stream is not in the routers list of streams it is “subscribed” to it will send a message (“subscribe me”) to it’s next router below that it wants a copy and so on down to the base.

The transmitter then just sends a single copy of the stream up the trunk to the first router which sends a copy up each branch to the next router until it reaches the user at the leaf. The key difference is that it scales well – each router only has to send a copy of the stream to the routers immediately above it, probably a few tens or hundreds of copies at most.

In the 5G context the final branching point will be the tower which will send out a single copy of the stream for all the users to receive.

The key point is that there is no extra bandwidth or server load required as each new user starts watching.

There only one downside: each user will be playing exactly the same thing, but for live streaming this is exactly what is needed. Any pausing or rewinding of the stream must be done by the player storing the data locally.

NBN and the bigger picture

When NBN was first designed they did incorporate Multicast into the FTTP product, however it was not done well. It was designed for companies (ironically they probably had Foxtel in mind) to have their own separate connection and to always use that connection to provide a multicast service. Of course there was no takeup since that is not how people buy streaming services these days. Multicast was then completely left off the other NBN technologies (FTTN, HFC etc).

Multicast does not work well for the video on demand type streaming (e.g. Netflix), because by making it “on demand”, you can’t join multiple streams into a single one – different people will be watching different things or watching different points in the stream.

However we are left with a problem, without multicast, live streaming will always be problematic over the NBN, not because the NBN isn’t fast enough (the usual criticism), but because unicasting a live stream at any scale will just put too much load on everything, not only the network but also the streaming servers.

It should be noted that 5G isn’t the magic answer here, it is only going to work in this particular instance, where a single company is essentially controlling the content creation, network delivery and the player hardware.

I fully expect that come the Olympics in 2020, we will see significant issues with live streaming of popular events. If we don’t then, then it will be at the next major live streamed event.

Damian Ivereigh
Launtel
10 Dec 2019