Are Mining Pools Becoming A Problem?

Bitcoin miners provide a valuable service to the ecosystem. In exchange for the work they do securing the network, they are rewarded by the same network they protect. This sound and elegant design by Satoshi is surely one of the most remarkable aspects of Bitcoin.

What is increasingly being forgotten, however, is that there is more to mining than merely hashing.

A person engaging in the entire process must run a node to get reliably updated on the most recent state of the blockchain, then begin construction of a new block. This involves verifying the validity of the previous block, discovering unconfirmed transactions and usually selecting the most lucrative of them, constructing a generation transaction in which they pay themselves, building multiple merkle trees of these transactions, and finally hashing to actually solve this block. The transactions within the block template will constantly change as new ones get broadcasted to the network and when a new block is found by someone else, the miner must switch to building on top of that along with dumping all the transactions now already in the blockchain to populate a new template.

Fork Activations

As you can see, hashing to actually solve the block is just one part of this process. A Bitcoin mining ASIC is also only capable of hashing. In the current environment, all other aspects of mining are generally delegated to mining pools. This has spawned some confusion. For example, in any circumstance where there is a discussion about activation of soft forks via version bit flipping within block templates, people will refer to this process being a MASF – “Miner Activated Soft Fork” – and someone will always have to clarify that this responsibility falls solely to pools and that pools are not miners. They may also point out that miners are still ultimately in charge as if they desire the upgrade and the pool they are mining with doesn’t, they can simply switch pools. [For clarity, in the rest of this article I will refer to those only participating in hashing and leaving all other aspects of mining to pools as “hashers”.]

Back to soft forks – in the current environment where >99% of blocks are constructed by the same dozen entities, it becomes more accurate to call these “Pool Activated Soft Forks” which no one does, contributing to a dangerous illusion: that mining can be considered decentralized merely due to distribution of hashrate. This claim is simply not credible when all the hashrate is beholden to a tiny group of pools and thus the contents of Bitcoin’s blockchain going forward ultimately will not include anything these few entities consider unacceptable, as well as a whole host of other issues.

By not engaging in any other aspect of mining beyond hashing blocks constructed by pools, Bitcoin miners have largely abdicated a critical component of their role. The fact that this is not only possible but also the path of least resistance indicates that we have a systemic issue.

Pools And Blockspace Markets

The implications of merely hashing and having a pool do everything else stretch far beyond soft fork activation. For example, miners presently are entirely unaware of what blocks will look like once solved, meaning that a miner performs work while blindly trusting that the block contains only desirable transactions. But you have a blatant violation of that trust in blocks such as this one – this is the famous block that kicked off the “ordinals” craze. Notice how the transaction fees the miners who worked on this block would actually enjoy amount to a measly ~$200 in BTC, in contrast to the blocks either side of it both averaging ~$5,000 in BTC.

Block space is valuable – that’s part of what makes Bitcoin work in the long term – but in a world where just a handful of players can have a template they construct end up in the blockchain, those same entities have near-exclusivity to sell this space and be paid out of band in exchange for it. Are they obligated – or even likely – to be forthright with their miners that they are doing this? Certainly not in this case as the intention was to surprise everyone. Going forward will they forward on to their hashers payments they receive for selling blockspace out of band?

Simply put, while the incentives for a pool and its hashers typically align with regard to maximizing profit, a pool has the option of selling blockspace for things other than regular Bitcoin transactions, while a miner’s income is more limited unless the pool chooses to be transparent and agrees to share revenue. Even if they do, verification requires the pool’s permission as opposed to verifying money earned from subsidy and transaction fees (also tricky with FPPS pools, more on that later).

Further implications of pools being Bitcoin’s centralized constructors of block templates stem from the fact that – on a more fundamental level, there are twelve “super nodes” with their own “super mempools”.

This cascades into people dealing directly with pools and ignoring mempools altogether. Some contend that the mempool is doomed regardless – and that the current state of centralized template construction is merely accelerating this, but it’s certainly not desirable in any case and it would be overly pessimistic to make this assumption in a world where genuinely decentralized template construction is somehow made realistic. Then out-of-band payments must make their way to a larger group of people if whoever is purchasing the block space wishes to make it into the chain in the same time frame. This would likely be more transparent and reminiscent of the way things currently work. Conversely, “super nodes” would hopefully be broken up into smaller pieces and thus no longer be able to offer the same guarantees.

To deviate from this aspect of mining let’s shift focus to how payouts are currently handled.

Pool Payout Models

Nearly all pools pay their hashers via FPPS (Full Pay Per Share) or something similar. One exception is ViaBTC offers PPLNS (Pay Per Last N Shares) in addition to FPPS. Antpool also offers PPLNS but hashers must forfeit all transaction fee revenue – this speaks to the point that I will soon endeavor to make – essentially that FPPS is not a model that works well in a world where transaction fee revenue is what is of relevance rather than subsidy. It should be mentioned that Braiins pool (formerly Slushpool) uses a system referred to as “score” which in practice is quite similar to PPLNS.

What is the reason for this overwhelming preference for FPPS? From the hasher’s perspective, they get paid no matter what happens on the blockchain. This is congruent with the purpose of pooled mining – greater consistency of income. FPPS offers more consistent payouts because the pool pays based on projected revenue and settles-up with the blockchain independently.

This makes life extremely easy for miners who want to minimize issues resulting from cash flow disruption, but there are of course drawbacks – major ones that I hope to highlight here.

FPPS first and foremost requires that the pool become the custodian of all freshly mined bitcoins. These cannot be forwarded on to miners for a minimum of 100 blocks as freshly mined bitcoins are unspendable until after this and in practice, the mined coins can have nothing to do with what the miners are ultimately receiving when making withdrawals from the pool. The risks of third party custody should be obvious to almost everyone reading this article so I’ll skip it and move on to other issues with FPPS.

The next concern comes from the fact that more generally, an FPPS pool is a significant intermediary between hashers and the network itself. We have already established that hashers aren’t privy to what the blocks they are working on will ultimately look like until after they are solved. FPPS means that they are now also unconcerned with whether blocks are even found or not, it’s the pool’s problem. Ignoring the increased predictability of payouts (should a pool never decide to rug its hashers) we must acknowledge the tradeoffs of doing this.

Miners getting paid directly by Bitcoin itself – possible in alternative schemes like PPLNS or of course solo mining – can expect to be fully rewarded for their contributions including transaction fees. An FPPS pool can only do this as a post-hoc calculation because there is simply no way to predict what fees will amount to when establishing what hashers actually receive per share. A pool cannot simply assume that fees will be some value greater than 0 and credit miners with this as they mine because should fees drop below this value, they would simply be paying the miners out of their own pocket. They must periodically divide up fees and attribute them to miners once actually in the pool’s custody.

From the hasher’s perspective, complete trust in the pool is required since verification is next to impossible without the pool’s full transparency and cooperation. Previously, as alluded to above, this was less of an issue since most mining revenue came from subsidy with only a sprinkling of sats in transaction fees – but this increasingly isn’t (and indeed cannot be) the future of Bitcoin mining. Going forward, miners will earn primarily from transaction fees and those are simply harder to predict and monitor when using a pool than the subsidy.

Contrasted with a payout scheme like PPLNS where hashers accept increased variability (the pool’s luck becomes the hasher’s luck too), we see that the mining ecosystem has overwhelmingly elected to prioritize consistency of payouts over the ability to verify what is received. More perversely, some hashers actually prefer this — wishing to present themselves to governmental authorities as a kind of “hashing service” entirely disconnected from Bitcoin–some proudly so. This is because FPPS is such a radical deviation from the ideal miner/pool dynamic that it’s once again hard to describe what the hasher is even doing as “bitcoin mining”.

In effect, the FPPS pool is a large solo miner paying hashers to solve its blocks. After which they have an internal and opaque process by which they figure out what to pay their hashers. To really illustrate the point the hasher could (and in some not-so-hard to imagine scenarios would) even be paid its fees in something other than Bitcoin.

Why not? If you don’t care if any blocks get found let alone what they look like before construction, why not just get paid fiat by a solo miner to point your ASICs at them in whatever the most convenient currency is? Bitcoin is not always the most frictionless option, but even if it were, it’s reasonable to imagine continuing down a path where “hashing” may be performed by as many entities as you like, but all done on behalf of a tiny group of “pools” whose permission the entire network needs to get anything into the actual blockchain.

Who Is Hashing Anyway?

Let’s look at this in a wider context. We have already mentioned that some larger players wish to distance themselves from Bitcoin as far as possible, thus happily delegating as much Bitcoin related activity to their pool as possible. The pools are wide open to regulation, and a large amount of their hashrate is only too happy about it.

This again introduces economic irrationality from the perspective of the network itself, manifesting in behavior such as the mining of blocks that meet certain arbitrary standards. When this occurred in the past, it didn’t last long due to backlash from the community, and the absurdity of trying to aggressively appease a jurisdiction’s shifting regulatory scheme without even being asked to do so. But the fact that that it was an option betrays the risk of having centralized construction of block templates. Will miners in one jurisdiction try to ban or refuse to process transactions stemming from another? Will miners simply be an extension of a government or influential bad actor? There are concrete examples of pools declining transaction fees to profiteer out of band, at times simply to comply with regulatory pressure. This once again appears economically irrational from the perspective of the network.

The most extreme recent example of this was the 19 BTC transaction fee paid in a transaction in a block ultimately found by F2Pool, ostensibly in error. As a FPPS pool, they became the custodian of the 19 BTC mining fee and chose to give it back to the person who made the mistake. This demonstrates perfectly the price of placing too large an intermediary between your miner and the Bitcoin network. In a PPLNS pool this would be less likely to have happened. Not because PPLNS pools are necessarily trustless or non-custodial, but by virtue of it being possible to monitor and verify fee revenue at the exact moment blocks come in, this would possibly have been harder for the pool to attempt having likely already credited miner’s accounts internally with their share of the mined funds causing greater backlash. Although nothing is in principle different until you contrast what would have happened should a pool make payouts to its miners in the coinbase/generation transaction itself. In that scenario the money would have already been in the miner’s custody and interception of fee revenue by the pool would have been impossible. So in this example a pool’s desire to seem generous or fair cost its miners $500,000 in fee revenue making a decision on behalf of them it should not have been in a position to make.

Next Issue: 51% And Other Attacks

This should be simple to explain: at this point everyone knows what a 51% attack is. What is far less understood though is that (up until the network routes around it,) 51% is the requirement for this style of attack to be a guaranteed and perpetual success rather than merely disruptive.

In reality, any entity that has more than 20% of the network can cause issues via a multitude of attacks, some being executed in the wild and only rarely discussed, which I will get into later. But before we do that, we can stare aghast at the network which has a pitiful two entities with a combined hashrate reliably greater than 51%. Worse yet, one of the largest pools not-so-carefully disguises that it is responsible for another 10% of blocks found through yet another large pool with whom the parent company maintains a strategic partnership. The fact that this pantomime persists does not inspire confidence.

There are two usual responses to this. Firstly, people point out that hashers can simply vote with their feet and switch pools if they ever combined forces to 51% attack. Secondly, that any pool would be insane to attempt it for the simple reason that disrupting bitcoin would cause the price to fall and no one invested in the ecosystem would ever want that. The second argument ignores human history and further assumes that people can never be coerced into behaving destructively and thus causing disruption simply for disruption’s sake or other nefarious purposes. (It also doesn’t take into account the fact that the market is often not necessarily a good indicator that there are issues with Bitcoin, see the forkwars of 2017.)

The first argument however makes a more solid assumption that hashers would always switch in a scenario where one pool does indeed get too large. Indeed, if pools tried to do this reality would kick in and we’d realize that despite constructing 99% of our block templates, pools aren’t actually miners. We also have a case study of Ghash.io which famously death-spiraled having spooked everyone by exceeding 40%.

Great, so we’ve demonstrated that this isn’t really an issue, hashers can be relied upon to just hop to another pool. (In reality, if large mining operations are all tied up in red tape it’s a far less reliable assumption but let’s at least proceed as though we’re fairly confident that this attack isn’t likely.)

Unfortunately, awareness of the fact that hash power will migrate away from any pool that exceeds a scary threshold leads to them self-regulating – but not in a way that helps because they do not need to genuinely maintain a hashrate below a threshold, they simply need to make it appear that way. This essentially amounts to accepting all the hash power they can get while forwarding it on to other pools as necessary to avoid alerting the world to their ability to wreak havoc.

So this leaves us with an unknowable picture of the network. 30% of blocks can be overtly found by the largest pool and be acceptable to everyone, while a further 10% of total network hashrate is still pointed at that pool and just secretly being directed to one or multiple smaller pools. The hashers responsible for that 10% are unlikely to realize it’s being used this way (and it gets even harder to detect with stratumV2 – more on this later).

This already less-than-ideal scenario gets far worse when you take into account the fact that this redirected hashrate can be used to harm smaller pools via the block withholding attack.

This is as follows – the attacker engages in the mining process mostly as a normal user of the victim pool. As a result, they get a share of the reward from any block the pool finds as expected. The rewards then ultimately end up with the attacker who can then pay the actual hasher without having to lose any money. So far the only harm caused is the incorrect impression of the pool’s hashrate as being smaller than it actually is but the smaller pool remains unharmed.

Now the harm occurs if they decide to not tell the victim pool when they find a block. This has the effect of making the victim pool appear unlucky. They appear to simply be finding fewer blocks than they should be and are paying out a reward split among more participants than are actually honestly mining – i.e necessarily running at a loss assuming they don’t make up the losses some other way.

If an FPPS pool is attacked this way, they must burn revenue paying miners out of pocket to make up for the difference. If they are PPLNS their miners wonder why they aren’t getting what they’re supposed to be getting. Either way, block withholding is anticompetitive and can destroy the victim pool by giving it a bad reputation.

From the attacking pool’s perspective, let’s say they make up 5% of the victim pool’s hashrate. This means they still make 95% of the revenue expected and the pool looks 5% less lucky than expected. This is easily enough to kill the pool whilst the 5% loss on the redirected hashrate will be of far less significance to the bigger pool. If it only represents 1% of the bigger pool’s total hash power then the attacker is only losing 5% of 1% of its expected rewards – 0.05%. This is a no brainer advantage to any malicious, significantly sized mining pool that is just prepared to act unethically.

The smaller the pool, the more vulnerable they are to this attack. The larger the pool, the more likely they are to block withhold a competing, smaller pool. This risk increases as large pools approach levels where their total hashrate begins to scare the community, which further motivates them to at least stash hashrate in smaller pools, even if they don’t actually attack with it or execute attacks infrequently enough for the problems to ultimately get dismissed as variance. Indeed – decreased variability is already enjoyed by larger pools due to more consistent payouts from the network which translates into being able to operate within tighter margins and thus be in a position to charge their hashers less. From the perspective of every miner/pool that isn’t under attack this attack means that they will enjoy lower difficulty as the Bitcoin network adjusts for there being fewer overall blocks.

Is block withholding merely theoretical? Absolutely not. Several mining pools were attacked in this exact way even as early as 2015. It is extremely difficult to thwart as a pool must monitor all workers and make a calculated decision to kick them off the pool and/or withhold payments to them should they be unlucky to a point of statistical impossibility and the pool able to reasonably assume they are acting maliciously. Attacks of this nature also incentivize pools to want to “know their hasher” and custody payments which of course makes life harder for those wishing to mine permissionlessly.

Regardless, the overall effect from all this is that people will prefer mining with larger pools for yet another reason.

We have publicly seen statements from large miners declaring that they are switching away from smaller pools due to getting payments that did not meet expectations.

This is extremely undesirable as larger pools and the larger hashers that use them are more easily encumbered with regulatory burden and thus prone to engaging in behavior that damages Bitcoin, going beyond even centralization of block templates and temporary custodianship of all block rewards.

The pools become effectively deputized, enforcing bureaucratic nonsense on “behalf of” their hashers. The two largest pools currently require that their users jump through a ton of hoops, including identity-exposing processes that should not and must not become necessary for someone to be able to mine bitcoin outside of solo mining.

To make one final point on block withholding beyond it threatening to make life harder for smaller pools and anyone wishing to hash with them, I say to anyone who might still be tempted to dismiss it as purely theoretical (even though its demonstrably happened in the past) – do we think it’s normal for pools to remain a consistent and apparently tolerable size organically? This would imply new hashrate coming online always somehow managing to distribute itself at least somewhat evenly. We must believe a pool can spring into existence, grow prodigiously and then just….stop….at right around the threshold needed before people get spooked. Do we see pools begging people to stop mining with them or straight up limiting account creation and kicking miners offline that exceed a permitted hashrate within existing accounts? We of course do not.

The two more probable scenarios are that either hashers are collectively self-regulating (unlikely, as mining with smaller pools now famously means earning less bitcoin even if the reasons I’ve presented in this article don’t entirely account for why – not to mention that examples of mass exodus from a pool were extremely noticeable the few times they have happened) – or – pools are simply misrepresenting the amount of hashrate they have pointed at them.

To add to all this, smaller pools have yet another issue: they can go days without finding blocks. A larger pool won’t go longer than a few hours. This is a question of resolution – the higher your hashrate, the closer you are to expectations over the short term, and this unfortunately results in a minimum threshold below which a pool cannot expect to make up for periods of bad luck at which point it just becomes impossible to compete.

The two-week periods between difficulty epochs means a reasonable number of blocks must be found within that two-week period so that any bad luck has a shot at being balanced out by subsequent good luck. If not, if – for example – the pool has a projected block rate of 1 block every 13 days and doesn’t find a block before the difficulty adjusts upwards causing them to drop to a projection of 1 in every 15 days, that prior window has closed forever. If it’s a PPLNS pool, the hashers have earnt less than they otherwise might have. If it’s an FPPS pool, the pool has burnt a lot of cash and/or become bankrupt.

This means there are only so many pools that can exist, at least ones that operate the way today’s pools operate. There simply cannot be hundreds, because many of them would keep collapsing in periods of bad luck due to having less than 1% of the network hashrate and therefore potentially not even being able to reliably find one block per day, encountering potential periods of weeks without blocks. This is a limitation placed on us by Bitcoin itself.

How Are Miners And Pools Communicating?

The protocol by which miners and pools communicate is Stratum (slowly but surely being superseded by StratumV2). StratumV1 is both ancient and deeply flawed. Firstly, all communication is done in plaintext. This means ISPs are not only privy to the fact that you’re mining but also the scale to which you are doing so, and they – along with anyone else that can snoop traffic on your network – can perform MITM attacks resulting in you using your machines and power on someone else’s behalf. This has been abused before by unknown attackers to hijack hashrate away from the intended pools.

Aside from a number of inefficiencies, StratumV1 also fails to offer miners a practical way to construct their own block templates and still enjoy mining in a pool. All these issues are addressed with the extremely desirable StratumV2 (originally “GBT”, then “Better Hash”) which we will return to later.

Hardware/Firmware

Before getting to the solutions, we’ll deviate from discussing pool/miner dynamics – as this article would be incomplete if we failed to bring up the fact that there are only two companies manufacturing ASICs at any meaningful scale – Bitmain and MicroBT. There are others, but realistically almost all hashing is occurring on machines manufactured by those two companies.

This is not good for obvious reasons and essentially stems from the fact that chip fabrication is extremely difficult to do and thus hyper-centralized.

It’s outside the scope of this article to go into solutions here, but there are folks working on making home mining something far more practical (in North America the main issue being the requirement for 220-240v and dealing with the obnoxious noise). The contention among those working on these pleb-mining projects being that if it becomes doable for enough every-day bitcoiners, they can start to represent a significant percentage of the total hashrate of the network, which is preferable to most mining operations operating at a scale where they are wide open to regulatory interference.

This task is made far harder by the fact that the firmware is closed source. Even custom firmware that can “jailbreak” an ASIC tends to be closed source in order to ensure those using it pay dev fees (i.e the cost for your awesome aftermarket firmware is mining on behalf of the team making the firmware.)

The stock firmware on ASICs – particularly Bitmain’s – is a great indication of how comfortable they have become with their dominance of the market. Beyond being closed source, it’s clearly malicious. You are forced to mine on their behalf upon powering up an Antminer – though a miner can at least prevent this from happening by blocking the connection (or installing aftermarket firmware, but then you pay dev fees instead and those can’t be blocked without the miner refusing to mine at all.) Bitmain has also been caught several times adding malicious backdoors to the firmware for their miners (see Antbleed), and actively works to lock out aftermarket firmware developers.

The fact that stock firmware does this is frankly outrageous and clearly highlights the dire need for competition in ASIC manufacture.

Would anyone feel comfortable if the rules of the network were enforced by closed source bitcoin nodes? Further, imagine those nodes caused users to lose BTC to the developers of that software – and we all knew that was happening. Would anyone accept that? When it comes to mining, almost no regard is paid to the sovereignty of its participants. Of course node software and ASIC firmware are not of equivalent importance and we of course place more scrutiny on the former as we should, but the latter is not immaterial and is certainly being unacceptably neglected.

With all that said, let’s move on to some of the solutions, focusing in particular in increasing the scope of what’s possible as a miner and improving on existing models.

P2Pool

There is not much to say on this beside the fact that it decentralized basically every aspect of pooled mining. While this does many desirable things at a small scale, it requires that every user download, verify, and track the shares of every other user and prove to each other that they are accounting for everything correctly in their templates. Achieving this in an adversarial environment at any scale is essentially an impossible task. Due to the fundamental nature of pooled mining, far more resources are required than what is needed to run a Bitcoin full node, not to mention making things more complicated for the miner.

For these reasons it has been ignored by most, and used only by more technical users or idealists who – understandably – cannot bring themselves to mine with the alternatives.

StratumV2

This is most certainly the lowest hanging fruit. It offers practical remedies for many of the issues mentioned in this article.

Firstly, by allowing encrypted communications between pools and hashers, ISPs and any other entity with access to your network traffic will no longer become trivially aware of the fact that you are mining (or the extent to which you’re doing so). “MITMing” you into hashing on an attacker’s behalf consequently also becomes impossible, or far less trivial.

Secondly and perhaps most significantly, it’s also capable of allowing hashers to construct their own block templates, so while pools would remain trusted coordinators of reward splits, and likely still custodians of block rewards – this would nonetheless represent a shift in power away from pools towards miners and be unequivocally a good thing.

Lastly, there are a few other improvements that I encourage you to check out here.

A world in which StratumV2 is the norm, along with enthusiasm from miners to actually construct their own templates (ideally a pool would offer an incentive to miners who did this) would enjoy a far more resilient Bitcoin.

The community is essentially unified in working towards upgrading the mining ecosystem to StratumV2, but historically miners have generally avoided using these solutions due to additional effort (albeit trivial compared to p2pool) and no incentive to do so.

Rounding up

There is great room for improvement with or without StratumV2. What’s needed is a pool that offers miners the ability to take direct custody of their coins while mining. This requires that a pool (or its hashers) construct block templates in which miner’s rewards are paid out directly in the coinbase/generation transaction contained within every block. The fact that this is impractical under the FPPS system means any pool doing this would face reluctance from some miners, but those who switched would enjoy greater transparency as Bitcoin itself would – above some threshold – be paying them directly with an easy to verify split of subsidy and fee revenue. This can be coupled with pools – pre-stratumV2 – at least making miners aware of block templates constructed on their behalf prior to blocks being solved, and post-stratumV2 simply needing to verify that all miners are constructing templates that accurately reflect reward splits without the scaling implications of all miners having to do this continuously.

The pool can also address the reluctance of miners to make their own block templates by offering incentives for miners who do so, by – for example – charging them lower fees. It seems that if miners are unwilling to take on the burden of doing this even once it becomes practical again, then this additional incentive might become necessary.

The above suggestions would dramatically improve things.

Many initiatives and announcements are coming up regarding ASIC manufacture and pool infrastructure that hopefully should be welcome developments for anyone interested in ensuring mining trends towards greater decentralization.

This is a guest post by Bitcoin Mechanic. Opinions expressed are entirely their own and do not necessarily reflect those of BTC Inc or Bitcoin Magazine.