Posts on Murch ado about nothing

What is the Waste Metric?

Fri, 06 May 2022 00:00:00 +0000

Coin Selection

Funds are tracked in discrete portions, so called Unspent Transaction Outputs (see UTXO model). Generally, a wallet’s funds are split across multiple to numerous such UTXOs. When a user sends a transaction, their wallet software needs to pick some of their UTXOs to fund the transaction. The process of picking the input set of a transaction is referred to as Coin Selection.

The primary goal of Coin Selection is to raise sufficient funds to pay for the recipient outputs and fees of the transaction, but there are secondary goals as well:

low immediate fees: As transactions must bid for the blockspace they occupy, we want transactions to be reasonably sized to limit their cost.
low overall fees: All UTXOs must be spent eventually to allow us to reassign their value to new outputs. Sometimes it may be opportune to use additional inputs for a transaction crafted while there is little blockspace demand to consolidate funds into fewer UTXOs.
financial privacy: As the blockchain is public and will exist forevermore, the details of our transactions are visible to all. This means that we may reveal financial information to our business partners when we pay them, and companies surveilling Bitcoin usage in general may be able to glean additional data from our usage patterns. We want to limit the information we leak about the wallet’s history, its UTXO pool, and total funding.

As a means to achieve our secondary goals, it is useful to consider the composition of the wallet’s UTXO pool. We want funds to be split in useful ways. For example, UTXOs of homogeneous values are less useful than UTXOs of diverse values. We want to have a sufficient count of UTXOs to not reveal all of our funds to business partners and surveillants in each transaction, but few enough that our wallet isn’t fragmented into little pieces, where we combine a lot of transaction history and incur high fees with every transaction.

Take for example two wallets that each have 1 ₿ trying to send 0.6 ₿. One has the UTXOs [0.5, 0.5], the other has [0.1, 0.2, 0.3, 0.4]. The first wallet would need to use both its UTXOs and send itself a change output of 0.4 ₿. All of its funds would be in flight, it would have no confirmed UTXOs left to send an independent transaction, and the recipient would learn that the sender has an additional 0.4 ₿. The second wallet might use the 0.2 ₿ and 0.4 ₿, which would leave two other UTXOs untouched in the wallet, and not require creation of a change output.

Generally, the combination space spanned by a diverse wallet is more densely populated than a homogeneous wallet. Split 4×0.25 ₿ a wallet with 1 ₿ can only combine the amounts {0.25, 0.5, 0.75, 1}, while a wallet with [0.1, 0.2, 0.3, 0.4] could combine all of {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1}.

Feerate sensitive coin selection

Wanting to pay low fees for the current transaction but also minimizing the overall fees paid over the lifetime of the wallet may conflict. Especially if a wallet is involved in a large volume of payments, simplistic approaches to Coin Selection like always using the largest UTXOs first, will cause the wallet’s UTXO pool to increasingly fragment over time. The biggest contributors to transaction weight are the inputs. When then a large amount is paid while there is high blockspace demand, a heavily fragmented wallet might be forced to use numerous inputs which then causes a high transaction fee. Obviously, we want to keep our transaction weight at high feerates low to minimize the transaction fee. To that end, a wallet operator might create consolidation transactions for the express purpose of defragmenting the wallet. While these additional transactions allow minimizing costs at high feerates, a similar effect can be achieved by opportunistically using more inputs than necessary when creating transactions at low feerates without requiring users to manually intervene. As more UTXOs are spent at lower feerates than at high feerates, the average feerate at which UTXOs are spent is lowered which decreases the overall lifetime fee expenditures of the wallet.

The waste metric

The waste metric provides a heuristic per which a wallet’s automated coin selection can achieve feerate sensitive coin selection.

weight: total weight of the input set
feerate: the transaction’s target feerate
L: the long-term feerate estimate which the wallet might need to pay to redeem remaining UTXOs
change: the cost of creating and spending a change output
excess: the amount by which we exceed our selection target when creating a changeless transaction, mutually exclusive with cost of change

The resulting waste score is a measure of the fees for the input set at the current feerate compared to spending the same inputs at the hypothetical long-term feerate. The waste score also accounts for the cost of creating a change versus dropping funds that exceed the selection target to fees in the case of a changeless transaction. When using the waste score to evaluate the input set candidates produced by multiple coin selection attempts, the scoring results in a preference for minimal input sets at high feerates and a preference for larger input sets below the long-term feerate estimate. Lighter modern output types have a lower waste score at high feerates and a higher waste score at low feerates, causing legacy inputs to be preferentially spent at low feerates and modern inputs preferred at high feerates. Change avoidance is always preferred as long as the excess doesn’t exceed the cost of change.

Unfortunately, the waste metric cannot capture all of our concerns: we do not account for privacy, and just optimizing for the best score would degenerate at the extremes. At high feerates, a single input solution would always win, while the best score at low feerates would be achieved by spending all UTXOs in the wallet for every transaction. We therefore rely on our selection of coin selection algorithms to produce sensible input set candidates and only use the waste metric to select from those candidates, rather than optimizing for the best waste score outcome.

A breakdown of different output types and their address formats

Thu, 23 Dec 2021 00:00:00 +0000

I was inspired by a recent reddit post to write my own description of the various single-sig output formats in Bitcoin. I’ll be covering only output types that make use of a single signature.

“Legacy Outputs” aka P2PKH

Pay to Public Key Hash was the first output type that got an address standard. Addresses for P2PKH outputs start with “1” and use the Base58Check encoding. The address encoding provides a checksum and represents a shorthand to communicate recipient information—which improved the UX over the prior situation which required the sender to handle the recipient’s full public key or non-standard output script to send a transaction.

Example: 1MDPuAy9WCbNQin71j9S3MKAAe9mGBRNVx

Issues:

High transaction weight
Case-sensitive addresses
Bigger data footprint than Pay to Public Key

“Wrapped Segwit Outputs” aka P2SH-P2WPKH

Segwit aimed to introduce a new family of output types which we refer to as native segwit, but the authors realized that adoption of a new address format would take time. The segwit softfork additionally introduced wrapped segwit outputs as a transitional solution, to be forward-compatible to wallets that could send to P2SH addresses. Since P2SH was rolled out in March 2012 and already widely used for multisig, most wallet software supported sending to P2SH when segwit was activated in 2017.

P2SH-wrapped Pay to Witness Public Key outputs are indistinguishable from other P2SH-uses until spent. All P2SH addresses start with the prefix “3” and use the Base58Check encoding like P2PKH addresses.

Wrapped segwit outputs lock funds to a P2SH Program, but their input script contains a Witness Program that redirects the script validation to the Witness Stack. While this achieved forward-compatibility, the redirection requires additional data that needs to be relayed on the network and stored in the blockchain. Since witness data is discounted by 75% when assessing transaction weight, wrapped segwit inputs are cheaper to create, even while their data footprint is larger than P2PKH.

All P2SH addresses start with “3”.

Example: 3Mwz6cg8Fz81B7ukexK8u8EVAW2yymgWNd

Issues:

Case-sensitive addresses
Redirection to the witness stack for evaluation adds extra data
Even bigger data footprint than P2PKH

Native Segwit Outputs

Instead of locking funds to a P2SH Program, the output script for native segwit outputs directly contains the Witness Program. This way, native segwit inputs only need a witness stack, and make do without the redirection data needed in the wrapped segwit construction, leaving only a zero-length stub for the input script. Native segwit outputs are represented with bech32 addresses for version 0, and bech32m for version 1–16. The bech32 encoding is single-case, making it easier to note down and dictate as well as more efficient to encode in QR codes. The bech32 encoding is engineered to provide error-detection guarantees. Bech32 and bech32m addresses start with “bc1”.

0) P2WPKH aka Native Segwit v0

Often simply referred to as “native segwit”, Pay to Witness Public Key Hash outputs lock funds to a public key hash similar to how P2PKH works. However, P2WPKH inputs provide the public key and signature in the Witness Stack instead of the input script, thus benefiting from the witness discount. Bech32 addresses encoding version 0 native segwit outputs start with “bc1q”, because “q” encodes 0 in bech32.

Example: bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4

Issues:

Bech32 addresses are still not supported by some wallets and services

1) P2TR aka Taproot aka Native Segwit v1

With the recent activation of Taproot, we add native segwit v1 outputs to our portfolio. Pay to Taproot outputs lock funds directly to a public key in the output’s witness program, which means (for single-sig uses) that the input only needs a single script argument, a signature, instead of needing to provide both a public key and signature like P2WPKH. P2TR uses Schnorr signatures, which are more compactly encoded than ECDSA signatures, reducing the signature size from 71-72 B to 64 B. This means that P2TR has the smallest data footprint even while the overall weight of input and output is slightly bigger than for P2WPKH. In addition, more complex spending conditions can be encoded in the leaves of the taptree that’s tweaked into the public key contained in the witness program. Bech32m addresses encoding P2TR outputs are longer than the bech32 addresses encoding P2WPKH outputs, since public keys are longer than the 20-byte hash used in P2WPKH. Addresses for P2TR outputs start with “bc1p”, because “p” encodes 1.

Example: bc1pay2tapr00tajnawrkf897ccgsmk4e0x8ng5g3rv3qzd7jzfy2zxspy50gj

Issues:

Bech32m addresses are brand-new and not yet supported by many wallets and services

Cost Considerations

All four described output types satisfy single-sig usage, although P2TR can do a lot more under the hood. Generally, the transaction cost is cheaper for newer output types: Legacy > Wrapped Segwit > Native Segwit. While the overall cost of P2TR input and output is slightly higher than that of P2WPKH, P2TR shifts a portion of the cost from the input to the output. When you don’t know at what feerate you’ll need to pay to spend your funds later, you should keep them in P2TR outputs, since they’ll have the smallest input cost. Likewise, you should prefer P2TR when others are paying you: the sender pays the output cost while the recipient pays the input cost. Although, you may still bump into some counter-parties that cannot send to bech32, and many that cannot send to bech32m, yet, the economic incentives are clear. If your preferred wallet or service doesn’t support bech32(m) yet, please do ask them to do so.

If you’re considering your transactions’ data footprint on the blockchain, you should also strictly prefer P2TR as it get you the most bang for the byte (see column “raw [B]” in table above). The data footprint for the output types is P2SH-P2WPKH > P2PKH > P2WPKH > P2TR.

2-of-3 inputs using Pay-to-Taproot

Tue, 18 Aug 2020 00:00:00 +0000

This article was originally published on my medium page on 2020-08-18.
This article was amended on 2020–12–20 to improve the description of the P2TR output scripts. Thanks to Matthew Zipkin, for pointing out the prior imprecision.

The Bitcoin community has been abuzz for a few years about bringing Schnorr signatures to Bitcoin. Since then, the idea has evolved into three formal Bitcoin Improvement Proposals: ‘BIP340 — Schnorr Signatures for secp256k1’, ‘BIP341 — Taproot: SegWit version 1 spending rules’, and ‘BIP342 — Validation of Taproot Scripts’. Respectively, they define a standard for Schnorr signatures in Bitcoin, introduce the Taproot construction, and formalize the new Script v1 instruction set. Taproot also iterates on the previous proposal of Merklized Alternative Script Trees (MAST).

There are two ways of spending a pay-to-taproot (P2TR) output. The first way is called keypath spending. P2TR funds are locked to a single public key similarly to pay-to-public-key (P2PK) outputs. The output script appears in the transaction as OP_1 <pubkey>. This script can be satisfied by providing an empty input script and a Schnorr signature of the corresponding private key in the witness program. The keypath is the default spending path. It is used in single-sig and collaborative multi-party spending.

Under the hood, the public key is a composition of an inner key and the Merkle root of a Taproot tree (the inner key is tweaked with the Merkle root). The inner key can optionally be a composite itself, for example multiple public keys aggregated via the MuSig scheme. These compositions are possible due to Schnorr signatures being linear.

The second way of spending a P2TR output uses one of the Taproot leaves. We call this scriptpath spending. First, the existence of the leaf needs to be proven which is done by revealing the inner key, the Merkle path to the Taproot leaf, and the script encoded in the leaf. Then, the spending conditions of the leaf’s script are fulfilled.

The remainder of this article explores the input costs of 2-of-3 multisig Taproot constructions. Some familiarity with the details of the proposals is helpful. Check out the Taproot overview and the excellent summary of the three BIPs by the Bitcoin Optech Group if you are looking to refresh the details.

The multisig user story

We are looking at a wallet construction using three keys held by two or three parties. The first key is held by the user, the second key is held by the service provider, and the third key is a backup key either held by the user or a key recovery service. We are assuming that the first two keys are hot and can be used in an interactive signing scheme as employed by MuSig. The backup key may be held cold e.g. on an air-gapped system in which case interactive signing should be avoided.

2-of-3 multisig with two hot keys and one (optionally) cold key

Classically, a 2-of-3 multisig address is constructed in the form of a pay-to-scripthash (P2SH) output of the form 2 <pubkey1> <pubkey2> <pubkey3> 3 OP_CHECKMULTISIG. This construction can logically be expressed as ‘(A ∧ B) ∨(A ∧ C) ∨(B ∧ C)’ which could also be thought of as “one of three 2-of-2 multisig scripts”. The latter translates nicely to Taproot: we can encode the preferred condition of spending with the user and the service key (the two hot keys) in an aggregated pubkey for the keypath, and put the two spending conditions using the backup key in leaves of the Taproot tree. Two variants are explored: one where the backup key is capable of participating in the interactive MuSig signing scheme, another that falls back to a simpler multisig scheme where signing is non-interactive e.g. because the backup key is air-gapped and the multiple roundtrips required for MuSig are inconvenient.

The most likely spending option with the two hot keys is encoded as an aggregated key in the keypath. The two backup spending options are put into leaves of the script tree.

Keypath spending costs

In the general case, all parties agree on a course of action and collaborate to use the keypath. This is the most cost-effective way to spend a P2TR output and allows multiparty spenders (and other complex spending conditions) to be indistinguishable from single-sig spenders.

Costs of keypath input

An input commits to a specific unspent transaction output (UTXO) which is defined by the transaction that created it and the position in that transaction’s output list. The UTXO entry provides the spending conditions to be satisfied by the spender. An nSequence field allows encoding replaceability, and the witness provides the spender’s authentication. In the case of the keypath this is a single Schnorr signature which in some cases may actually consist of multiple aggregated signatures.

outpoint (txid:vout): 32 vB+4 vB
scriptSig size: 1 vB
nSequence: 4 vB
count witness items: 1 WU
witness item size: 1 WU
signature: 64 WU

32+4+1+4+(1+1+64)/4 = 57.5 vB

Control Blocks

When a fallback to the backup key is necessary, the existence of the Taproot tree must be revealed. If there is only a single leaf, the spender provides only the inner key. Tweaking the inner key with the hashed leaf results in the public key. In sum, the data necessary to prove the existence of a scriptpath is called the control block.

Depth 0 control block

Length of control block: 1 WU
Header byte (script version, sign of output key): 1 WU
Inner key: 32 WU

1+1+32 = 34 WU

In case of two leaves, additionally the first hashing partner for the Merkle path must be revealed:

Depth 1 control block

Length of control block: 1 WU
Header byte: 1 WU
Inner key of root key: 32 WU
Hashing partner in tree: 32 WU

1+ 1+ 32 + 32 = 66 WU

Scriptpath spending cost

The below costs are in addition to the above costs of spending via the keypath.

Scriptpath spend assuming 2-of-2 MuSig leaf (hot backup key)

When the backup key is on a networked system, e.g. an HSM, and can participate in a multi-roundtrip signing process, we can make use of MuSig to aggregate the two public keys.

script size: 1 WU
script <pk> OP_CHECKSIG: 33 WU+1 WU
Depth 1 Control block: 66 WU

57.5+(1+34+66)/4 = 82.75 vB

Construction with 2-of-2 OP_CHECKSIG (cold backup key, no MuSig)

In the case that the backup key is offline and a human would have to make multiple trips employing USB sticks or QR codes, saving roundtrips may take precedence over saving a few bytes. Instead of an aggregated public key, we use a non-interactive multisig construction.

second signature: 1 WU+64 WU
script size: 1 WU
script <pk1> OP_CHECKSIGVERIFY <pk2> OP_CHECKSIG: 33+1+33+1=68 WU
Depth 1 Control block: 66 WU

57.5+(1+64+1+68+66)/4 = 107.5 vB

Discarded approach: single leaf with 2-of-3 script

It turns out that a single 2-of-3 leaf in lieu of the two 2-of-2 leaves is both more costly and less private.

+2nd sig: 1+64 WU
+1 empty witness item: 2 WU
Length of script: 1 WU
Script <pk1> OP_CHECKSIG <pk2> OP_CHECKSIGADD <pk3> OP_CHECKSIGADD 2 OP_EQUAL: 33+1+33+1+33+1+2=104 WU
Depth 0 Control block: 34 WU

57.5+(1+64+2+1+104+34)/4 = 109 vB

Conclusion

Upper bound of input and output sizes for single-sig and 2-of-3 multisig

The described 2-of-3 multisig scheme achieves input sizes of 57.5 vbytes for a keypath spend, 82.75 vbytes for the leaves using a hot backup key, and 107.5 vbytes for non-interactive backup spends. This results in a fee reduction by 45% for 2-of-3 inputs when switching from P2WSH to P2TR spending. In the uncommon case of a recovery transaction, the cost is negligibly increased for cold keys. Single-sig users are also incentivized to switch to P2TR as they save 11 vbytes on each input — the output cost is externalized on the sender.

Excited for Schnorr signatures

Sat, 03 Feb 2018 00:00:00 +0000

This article was originally published on my medium page and hackernoon on 2018-02-03.

If you’re keeping a finger on the pulse of Bitcoin development, you’ve probably already heard about Schnorr signatures and you probably won’t find much new here. You might rather want to check out Pieter Wuille’s recent talk at BPASE18, or Bryan Bishop’s compilation of transcripts of Schnorr signature talks whom this article heavily leans on.

Bitcoin signatures are created using the Elliptic Curve Digital Signing Algorithm (ECDSA). Schnorr signatures are another form of digital signatures. The signatures are based on the same security assumptions as ECDSA and are compatible with the elliptic curve Bitcoin already uses (secp256k1). This means that Schnorr signatures can be created with the same private keys and are compatible with currently used key derivation schemes.

Schnorr signatures are smaller

ECDSA signatures vary in size, but almost all come in at a length of 72 or 71 bytes. A small portion will turn out smaller with a theoretical minimum of 8 bytes. [h/t Greg Maxwell]

Schnorr signatures are more efficient and compact than ECDSA signatures. The maximum length of each signature is 64 bytes. [via Harding]

Bitcoin blocks include thousands of signatures, and I estimate from the top of my head that signatures make up more than a third of the blockchain data. Simply by being more compact, Schnorr signatures would reduce the blockchain data footprint by a few percent.

Schnorr signatures allow for compact multi-signature…

The multi-signature scheme in Bitcoin is straightforward but naïve. You first list the set of public keys of the authorized signers, and then provide a sufficient count of signatures by the former. E.g., in a 2-of-3 multi-signature transaction input, three public keys and two signatures are provided.

Schnorr signatures have a neat mathematical property that allows multiple signatures to be combined into a single signature. The combined signature has the size of a single signature, but provides the authorization of the original separate signatures.

This allows for a more compact multi-signature scheme, where you only list the authorized public keys and provide a single signature. This is even more efficient for bigger multi-signature transactions. For example, a 3-of-15 and a 10-of-15 transaction input could now have the same weight (if you don’t care who signed).

…and aggregated signatures across a whole transaction!

While multi-signature transactions make up a solid portion of the blockspace, the real breakthrough is that signatures can be aggregated across multiple inputs of a transaction. Instead of providing one (or multiple signatures) for each input, a transaction with Schnorr signatures can have a single signature for all inputs.

For example at BitGo, we use 2-of-3 multi-signature transactions. Every transaction input therefore has two signatures. As we’re seeing low fee rates on the network, some of our customers have started consolidating funds from low-value UTXO. With the current signature scheme, a 2-of-3 multi-sig transaction with 200 inputs would need 400 signatures or about 28.5 kB of signature data. With Schnorr signatures the same transaction could be signed with a single 64-byte Schnorr signature.

In his talk at BPASE18, Pieter Wuille estimated that purely from aggregating signatures for each transaction and leaving everything else the same, the Bitcoin blockchain would be between 25% and 30% smaller.

Scriptless Scripts, and… various black crypto magic

There is some interesting work by Andrew Poelstra lately which he calls “Scriptless Scripts” (see e.g. his talk at Real World Crypto 2018). The idea is that you can express conditions in a smart contract by requiring certain signatures to be provided for the payout. By means of the above mentioned signature aggregation, this could be used to compactly encode smart contracts. The terms of the contract would be hidden from other users and only transparent to its participants, yet enforced by the whole Bitcoin network.

You may have heard about the recent cross-chain atomic swaps. The basic idea is that two payments on two different blockchains are linked in a way that they either both go through or neither. This can be used to decentrally trade cryptocurrencies. Hereby, the traders first lock up funds in shared addresses on both chains, and then create two interdependent transactions. The second transaction depends on a hash preimage that is revealed by the first transaction. Either party can back out and wait for the lock to expire to reclaim their funds, but when the first transaction is executed, the other transaction becomes immediately valid.

By means of the same property that allows for the signature aggregation, all of the above cross-chain atomic swap can be expressed in a single Schnorr signature indistinguishable from a regular spending transaction.

A BIP for Schnorr signatures is in the works

The introduction of Schnorr signatures into Bitcoin requires a new OP_CODE for signature verification. Luckily, Segwit gave us versioning for Bitcoin script, so support for Schnorr signatures can be activated with a soft fork. I hear that multiple Bitcoin Improvement Proposals are in the works and forthcoming shortly.

Thanks to Pieter Wuille for review.

Edit: Corrected the length of ECDSA DER-encoded signatures.

A look at an early LN testnet topology

Fri, 29 Dec 2017 00:00:00 +0000

This article originally appeared on my medium site on 2017-12-29.

Let’s take a look together.

A commenter on my Fyookball response: ‘This [tweeted image] shows a lot of centralization around hubs’

We see about ten “supernodes” that stick out by having a lot of channels. We also see that there are some smaller nodes that form channels exclusively with these supernodes. I assume this is what you’re referring to, when you decry “centralization around hubs”.

However, I’d like to point out a few more things in this graph.
If we look at the cluster in the top left for example, we see that every node in that cluster is connected to the supernode, but there are also numerous channels between the other nodes around it. The supernode is connecting to the bigger cluster with a payment channel directly to another supernode as well as through a small bridge node.

If we look at the area around the two yellow highlighted supernodes, we see a large number of smaller nodes that form connections to multiple supernodes as well as other nodes.

The expected topology of LN according to Fyookball

Comparing this to the expected topology from Fyookball’s article, you’ll note that the “Decentralized (with centralized hubs)” topology is distinctly different from what we see in the LN graph. In the suggested “Hub and Spoke” graph, only hubs connect hubs, while nodes exclusively connect to hubs. Obviously, the “Distributed” graph is a showcase, but it’s clear to see that the actually exhibited topology is at least a mixture between the “Decentralized” and “Distributed” topologies.

Let’s take another look, today:

The LN testnet graph on 2017–12–28 via stevenroose.github.io/lightninggraph/

This marvellous jumble we see, makes it clear that we’re not seeing a hub and spoke topology.– If anything, this clutter illustrates how easy it to bypass the “supernodes”.

PSA: Wrong fee rates on block explorers

Tue, 12 Dec 2017 00:00:00 +0000

This article originally appeared on my medium page on 2017-12-12.
Update: Blockchair.com, BTC.com and Smartbit.com.au have corrected their fee rates since my article. Thank you!

We’ve been getting support requests from customers inquiring why their transaction’s fee rate is not matching the parameters they set.

The problem is that popular blockchain explorers don’t get the fee rate right. Let’s look at the following transaction with two P2SH-P2WSH multisig inputs and six outputs: cdeea0d6c37a046d5b7e13a75bc0c9842493f41dd5a97d248df43552ad9e15c8

The customer set a fee rate of 160.20 sat/B. Naturally, they were surprised when they saw what blockchain explorers are telling us:

Tradeblock.com says the fee rate is 216.81 sat/B.

smartbit.com.au says the fee rate is 90.50 sat/B. — Update: smartbit.com.au has fixed their fee rates since this article. Thank you.

It’s pretty clear that the fee is 78,920 satoshis…

…but what does this transaction “weigh“?

Stripped size: 364 bytes
This is the size of a transaction when ignoring the witness.
Without witness each input is 76 bytes long, each of the five P2PKH outputs has 34 bytes, the P2SH output has 32 bytes, and the transaction header is 10 bytes.
2×76+5×34+32+10 = 364 bytes

Fee rate: 78920 satoshis / 364 bytes = 216.8 sat/B

Raw size: 872 bytes
The raw byte length of the transaction.
Each of the two inputs has a raw byte length of 329 bytes, each of the five P2PKH outputs has 34 bytes, the P2SH output has 32 bytes, and the transaction header is 12 bytes long.
2×329+5×34+32+12 = 872 bytes

Fee rate: 78920 satoshis / 872 bytes = 90.5 sat/B

Where are the 160.2 sat/B we are looking for?

The explorers are showing us “fee / stripped transaction size” and “fee / raw transaction size”. While the calculation with the stripped transaction size overestimates the fee rate by considering the transaction smaller than it is, using the raw transaction size underestimates the fee rate by ignoring the witness discount.

However, since Segregated Witness activated, the relevant fee rate is “fee / virtual transaction size” in [satoshis per virtual byte].

Virtual size: 491 vbytes
The virtual size (vsize) of the transaction respective to the portion of a block it occupies, corresponding to the size of non-segwit transactions.
The vsize is calculated as follows:

(3×non-witness size + raw transaction size) / 4
or
(4×non-witness size + witness size) / 4

For non-segwit transactions the vsize is equal to the size. Each of the two inputs weighs 139 vbytes, each of the five P2PKH outputs weighs 34 vbytes, the P2SH output weighs 32 vbytes, and the transaction has a header of 10.5 vbytes.
2×139+5×34+32+11 = 491 vbytes

Fee rate: 78920 satoshis / 491 vbytes = 160.7 sat/vB

At last, BitFlyer gets it right by using scattershot:

BitFlyer lists all: the fee per raw tx size, the fee per WU and the actual fee rate.

Could we pretty, pretty please have more blockexplorers that show us the relevant fee rate?

Appendix: Additional honorary mentions

Blockchair tells us the “fee per kB” is 0.00216813 BTC. — Update: Blockchair has fixed their fee rates since this article. Thank you.

Blockchain.info shows the fee rate over raw byte length and a correct “Fee per weight unit” of 40.2 sat/WU, which however doesn’t compare to the established fee rates.

btc.com shows virtual size, but calculates the fee with the raw byte length: 90.5 sats/B.— Update: BTC.com has fixed their fee rates since this article. Thank you.

[blocktrail.com shows stripped size of 364 bytes and a fee rate of 216.8 sat/B.](blocktrail.com shows stripped size of 364 bytes and a fee rate of 216.8 sat/B.)

blocktrail.com shows stripped size of 364 bytes and a fee rate of 216.8 sat/B.

Some subsidiary points on Lightning Network

Thu, 29 Jun 2017 00:00:00 +0000

This article first appeared on my medium page on 2017-06-29.

I was pondering whether it is a good use of my time to respond a second time to Jonald Fyookball. Since the new article mostly repackages the same misunderstandings, I have decided to only provide some subsidiary information.

Who performs the Routing in Lightning Network?

When a node in Lightning Network wants to initiate a payment the sender picks every hop of the route. To that end, they use “Onion routing” — they construct a packet much like a prank christmas present from your older brother: a multitude of ever smaller boxes, until it is finally revealed that it’s just a pair of alpaca socks.

For illustration, let’s look at a simple example with two hops. Alice wants to send 1mBTC to Carol, and Alice and Carol each have a payment channel with Bob.

Alice sends an encrypted message to Bob with the following content:

A cryptographic contract which commits Alice to signing over 1 mBTC to Bob if he can provide the secret.
Designation of Carol as the next hop.
A cryptographic contract which commits Bob to signing over 1 mBTC to Carol if she can provide the secret.
An encrypted message to be delivered to Carol.

If Bob agrees to the terms of the payment and has sufficient capacity, he may forward the payment request. If he doesn’t, he responds with a failure message, which allows Alice to construct an alternative route. If he doesn’t react, the payment request times out after a fraction of a second. Let’s assume Bob agrees and forwards the payment request. Bob sends a message to Carol with the following content:

A cryptographic contract which commits Bob to signing over 1 mBTC to Carol if she can provide the secret.
The encryted message to Carol.

Carol decrypts the message and finds the invoice identifier, and the secret. She can now use the secret to execute her cryptographic contract with Bob. She gets paid immediately while inextricably providing Bob with the secret. Bob can now use the secret to get back his prepayment from Alice by executing the cryptographic contract with Alice.

Observations

There is no centralized routing queue. Routing is performed by the sender with knowledge of their local neighborhood.
All participants in a multihop payment have to be online to agree on the contract for it to come to pass. Thus, the expected occurrence of unresponsive channels is low. If a node along the route dropped from the network, a failure response is returned.
The recipient is paid immediately. Others are incentivized to settle as quickly as possible. Only the amount locked to a contract is unavailable for transactions, the remaining capacity is free to be used in other payments.
Unresponsive behavior is easy to punish: Nodes that don’t respond to payment requests will not be considered for later payment attempts. Eventually they lose their channel partners for not being useful. The network mends by finding routes around them.
Every participant in a multihop payment learns only about the previous and next hop in the chain. They also cannot tell if a neighboring hop is sender or receiver of the payment.

“Hubs” are just well-connected nodes

In discussion around Lightning Network the term “hub” frequently pops up. This term makes little sense, as there is no clear distinction of “hubs” and other nodes. Each node can create as many payment channels as they wish to commit funds to and find useful. Nodes that are well-connected might be perceived as a more useful payment channel partner, but at some point such nodes would inadvertently arrive at the limit of funds they are willing to commit to payment channels. Besides, if there are fees to be earned for routing payments, it may actually be more profitable to extend routes into remotely connected regions of the network. In any case, there is no restriction on other routes emerging around well-connected nodes.

Responding to Jonald Fyookball's article on "Lightning Network's Infeasibility"

Tue, 27 Jun 2017 00:00:00 +0000

This article was originally published on my medium page on 2017-06-27.

I have just read Jonald Fyookball’s article which claims to provide mathematical proof that the Lightning Network will fail to provide a ‘Decentralized Bitcoin Scaling Solution’.

It starts with a short description of single hop and multihop payments. After that it asserts that it will be infeasible to scale such a network topology to a any significant size.

Before exploring the details, the author already knows what the topology of the Lightning Network will look like (image from the article):

Let’s ignore for a moment the obvious grievance of evidence being circularly derived from preconceived conclusions instead of observations and evidence.

The author correctly recognizes that using LN for a single payment would be pointless as a direct on-chain payment would be more efficient. Next, they present a model of what they think potential routes would look like from the viewpoint of a user (image from the article):

The author muses that “probably only one of these channels will reach the intended recipient at any given time”. This crucial assumption is not further corroborated, and doesn’t make any sense. Simply, the presented tree model is not an accurate representation:
When a LN participant searches for a route, they’re obviously only interested in directed payment capacity. This aspect is correctly represented in the tree. However, the nodes along the route are interconnected. While this could even allow cycles to occur, cycles are not possible in the route construction, as it would allow participants involved multiple times to cut out the cycle when learning the secret for the first time. The resulting graph is what we call a DAG or directed acyclic graph:

Directed acyclic graph [image from wikipedia]

The interesting takeaway is that the assumption of only one route is far-fetched: once a connection is found, any part of the route can be exchanged for a parallel path, allowing for a multitude of possible combinations.

Misrepresentation of payment forwarding as ‘Lending’

The multihop payments in LN are constructed such that forwarding a contract to the next hop is equivalent to committing to the contract. Once the recipient receives the contract through the route, they can use the enclosed secret to immediately pull in the payment from the last forwarder. To get paid, the recipient inextricably trades the secret to the last forwarder, who in turn then can trigger the payment to themselves. Forwarders do not lend money. They trade balance in one payment channel for balance in another. As each hop has to advance the payment before receiving it, they are intrinsically motivated to execute the next hop as soon as possible. This allows to surmise that the recipient will be paid immediately, and in all likelihood all of the hops will be settled within milliseconds after a route has been established.
Once both hops a forwarder is involved in are resolved, all balances are settled and free to be used in other payments. It is therefore inexplicable what Jonald is getting at, when they suggest that a high amount of routing activity would reduce the availability of a user’s channel.

Assumptions about Routing

The author next collects a bunch of assertions from which they conclude that the network would be deficient.

Random searching of the graph has a low success rate
This point is likely related to the deficient route model, however, it is trivial to do better than randomly searching the graph. An early approach suggested by Rusty Russel for example prescribes each peer learning their local topology. If and when the network grows too large for this, it could be overarched with a landmark approach. The suggestion of randomly generating and trying routes is laughably inefficient and thus can’t be taken serious as a limiting factor for the network.
Channels are exclusively used in one direction
Firstly, by channels being part of various routing attempts in different directions, it is not obvious that channels will not rebalance themselves more frequently, and even if it were true, doubling the channels would not be a solution. Besides, it could for example be feasible to combine onchain payments with a “Lightning Network refund” for more frequent rebalancing of payment channels, in fact, this could relieve users of creating change outputs.
Routing for others unbalances channels and decreases usable channels.
Again, since channels are bidirectional and we’re not in a tree graph but an interconnected graph, it may actually be possible that e.g. A→B→C→D and A→C→B→D both exist, thus even two subsequent payments from A to D may rebalance the channel between B and C.
Wealth disparity limits number of routes that can forward payments.
Obviously channels can only forward to the limit of their own capacity. In conclusion a route can only transfer the capacity of the weakest link. This is in particular why Lightning Network has been described as a scalability solution for micropayments and low-value payments. As low-value payments are least competitive on-chain but more frequent, this is in fact a great proposition. Larger payments are more economical on-chain anyway as the fees will represent a smaller relative portion of the transferred value. It seems obvious, that Lightning Network payment requests would soon be extended to allow payments to be split over multiple routes to mitigate the capacity limitations.
“There always exists a risk of a routing channel becoming unresponsive (either intentionally or unintentionally). This risk also grows exponentially with an increasing number of hops.”
As there are many routes from each node to each other node, the network is self-mending. The channel partner of the offline party can choose to terminate the channel and after conclusion thereof put the funds into another channel that is more reliable. If a peer goes offline while participating in a multihop payment, either the payment fails, or the offline participant will only collect their payment with a delay. Participants with low uptimes will likely be peered less with and have smaller channel capacities. They will also likely not be heavy participants in the payment network anyway.

As the model on which Fyookball is basing their calculations is already far-fetched, I will not bother addressing the math.

Capacity vs Scalability

Sat, 14 Jan 2017 00:00:00 +0000

Let’s talk about terminology.

Capacity: The number of transactions that can be processed on the network.

Scalability: Capability of the network to handle a growing amount of work.

Examples:

A 2MB hardfork is a capacity increase but not a scalability improvement.
Segregated Witness is a capacity increase and a scalability improvement.
Monero has no block size limit and thus a higher capacity than Bitcoin, however its TXO pool is unpruneable, its blockchain would grow faster at same usage, and its transactions take more computational effort to validate, so it doesn’t scale as well as Bitcoin.

The Lightning Network is not a sidechain

Tue, 08 Mar 2016 00:00:00 +0000

The Lightning network is not a sidechain.

A sidechain relies on its own blockchain which is coupled to the Bitcoin blockchain via a two-way peg.

On the other hand, the Lightning Network consists of native Bitcoin 2-of-2 multisig transactions.

When two Lightning nodes open a payment channel, they both send funds to one multisig address and each provides the counterparty with a pre-signed exit transaction. If a party wants to resolve the payment channel, they can unilaterally add their own (the second) signature and send the transaction to the Bitcoin network for confirmation.

However, it is more efficient to update the balance of the payment channel repeatedly by exchanging new exit transactions. The updated exit transactions simultaneously invalidate the previous exit transaction. Whenever anyone wants to get out or tries to cheat (by using an invalid exit transaction), the exit transaction is moved to the Bitcoin blockchain for arbitration. Newer exit transactions then may override older ones¹.

So, while Lightning transactions can happen off-chain, they can be put on-chain anytime.

this is a slight simplification of the mechanism ↩︎