Bitcoin Infrastructure - rules and their enforcement

The first section took us through the directives outlined in the white paper for how nodes should operate the network. When looking closely at these directives, there has to be a way for each node to know that the transactions or blocks which they are receiving from other network peers are valid. This is done with a series of protocol rules which are enforced by the collective participants in the block building process.

Proposed transactions and blocks can be broadcast into the network from anywhere and can originate from a system that is honest or malicious. Transactions represent a request to inscribe new information onto the ledger including both arbitrary data used in higher order systems, as well as updated ownership information over Bitcoin tokens, which can in turn themself, represent higher order tokens and exchangeable objects. 

Blocks represent a timestamping event across a topologically ordered collection of transactions and occur at a rate which is automatically controlled by node operators. Nodes work to ensure that only valid transactions and blocks are accepted onto the network. This is managed through a series of checks to ensure information presented in the protocol conforms to a certain set of rules. Each node applies the rules as part of its role in operating the network.

Broadly, they are broken down into the following rulesets:

• Consensus rules
• • Block consensus rules
  • Transaction consensus rules
  • Script language rules

• Standard local policies:
• • Standard local transaction policies
  • Standard local script language policies
  • Standard local P2P network policies

Each event received by a node must be evaluated against these rule sets before being accepted or propagated to other nodes.

Bitcoin consensus rules

The consensus rules are codified into the Bitcoin node client software system and represent fixed and unchangeable rules applied across the network. These rules must be strictly adhered to in order for a node using the client to actively participate in the network governance process.

The immutable rules are a set of rules that define the format and constraints that transactions and blocks must follow. Making any changes to these rules could cause nodes to disagree on the nature of Bitcoin. All agreements are enforced through application of proof-of-work by nodes in a process known as Nakamoto Consensus.

Since 2009, aspects of the protocol have been either changed or removed. The Bitcoin SV philosophy is that where aspects of these rules have been changed in the past they should be returned to be as close to the original rules as possible and then “set in stone”. In some cases this has resulted in divergence with the original protocol including minor changes to the scripting language.

The only further changes needed are the return to the original difficulty readjustment algorithm and re-enablement of the last remaining script opcodes plus some spare opcodes if the need ever arises to allow changes to protect the security of the network. This could be things such as the addition of a new hash function or a more secure digital signature algorithm.

This section details the role of nodes in the network and also how nodes resolve any conflicts in agreement on the state of the Ledger. Nodes vote to enforce the rules used to build the blocks that make up the Bitcoin public ledger and define an honest node as a node that seeks out the longest valid chain of blocks and applies proof-of-work to extend that chain.

Bitcoin block consensus rules

Bitcoin block size rule

When a block is found, there is an economic limit applied to the block size, which is imposed by nodes on the network. This allows nodes to reach consensus on behavioural limits of the network. This limit is set to a large multiple of typical demand.

The size of a block is the size in bytes of the serialised form of the block, including the block header and all of the transactions it includes.

This rule is a configurable consensus rule meaning that node operators set the limit. As a group, they are expected to reach consensus on this value and configure it manually.

There is no default value for this setting, and the current BitcoinSV node client software will not start without the operator setting a value. Nodes that are configured with different values to the rest of the network risk having their blocks rejected, or rejecting blocks that other nodes accept, leaving them stuck working on a nonviable chain-tip.

Bitcoin block subsidy rule

The block subsidy will drop by half at a scheduled rate of every 210,000 blocks, starting with a subsidy of 5,000,000,000 satoshis per block from the genesis block, rounding down when sub-integer fractions are encountered.

This rule defines the rate at which Bitcoins are released onto the network from the original issuance. This rule means that nodes who create blocks are rewarded Bitcoins which have not been previously circulated. It is the means by which the entire supply of Bitcoin is released onto the ledger. The subsidy distribution rate was codified in the first ever version of the Bitcoin client.

Nodes who try to issue extra bitcoins to themselves outside of this so-called ‘inflation schedule’ violate the rule that defines how many bitcoins are in existence and as such their blocks must be rejected as invalid. Nodes must adhere to this rule, or their blocks will be judged as invalid by the other participants in the network.

Interestingly, blocks which fail to reward tokens from the subsidy or tokens paid by transaction creators as fees to the winning miner are still considered valid, and the satoshi tokens which aren’t included in the final reward are withheld from circulation. While this is uncommon, it has happened in the past, collectively resulting in a cumulative loss of over 1000 Bitcoins since the network was first formed.

Proof-of-work target adjustment rule

The network will adjust the target for the difficulty of the proof-of-work needed for a valid block to maintain an approximate 10 minute block discovery rate.

This rule defines how the network adjusts the target difficulty rate as the available resources being applied to proof-of-work increases over time. The discovery of a block is a pseudorandom process in which nodes compete to check as many combinations of nonce and block header to try to find a value which is equal to or less than a difficulty value that is algorithmically determined by the network. 

The system makes periodic adjustments to re-define the difficulty rate based on the period of discovery of previous blocks. In the original release of Bitcoin this value was calculated once every 2016 blocks, or approximately every 2 weeks. However, in 2017 the method of calculation was changed to one which recalculates the difficulty target after every block.

There is a commitment from node operators to revert the network back to the 2016-block adjustment cycle from the original implementation. Because this is an immutable rule, this will require a network wide upgrade to take place in which all nodes will accept the modified consensus conditions simultaneously.

Bitcoin Genesis block rule

Only blocks that add to the block chain formed by building upon the Genesis block are valid.

This rule states that blocks must be added to an unbroken chain of proof-of-work leading back to the genesis block, which has its block hash hard coded into the Bitcoin node client’s software. 

This prevents a malicious party somehow creating a new chain in order to perpetrate a malicious redirection of hashpower or economic activity, and is an important aspect of Simplified Payment Verification, allowing users to check they are using the correct chain of blocks with minimal overhead.

The genesis block is identified using its block hash which is 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f. By ensuring all users of the network are aware of this block hash, there can be certainty around which chain of blocks is valid. 

If a user or node connects to a chain of blocks which leads back to a point of origin that is not the genesis block, it can know immediately that it has connected to the wrong network.

Bitcoin transaction consensus rules

Transaction consensus rules are the rules that define how transactions are interpreted by the nodes collectively, so it is of utmost importance that consensus be reached and maintained.

Maximum transaction size rule

This is an economic limit imposed by miners in consensus.

The length is the number of bytes in the serialised transaction. This rule allows miners to collectively set a practical limit for users on the size of a transaction. As the capabilities of nodes and node software improve this limit will rise, providing added utility to all network users.

Use of nLockTime and nSequence

The nSequence fields of every transaction input and the nLockTime field of the transaction collectively determine the “finality” of a transaction. If a transaction is “non-final” then it can not be valid but it can become “final” at a later time. If a transaction is “final” then it can be included in a block.

The interpreter will evaluate nSequence and nLocktime as per the following:

• If the value of nSequence of a transaction input is 0xFFFFFFFF then that input is a “final input”.
• If the value of nSequence of a transaction input is not 0xFFFFFFFF then that input is a “non-final input”.
• If all of the inputs of a transaction are “final inputs” then the transaction is “final”, irrespective of the value of the nLockTime field.
• If one or more of the inputs of a transaction are “non-final inputs” then: 
• • If the value of the transaction’s nLockTime field is less than 500,000,000 then the field represents a block height.
    • If the node is working on a block whose height is greater or equal to the value of this field, then the transaction is “final”.
    • Otherwise the transaction is “non-final”.
• • If the value of the transaction’s nLockTime field is greater or equal to 500,000,000 then the field represents a UNIX epoch timestamp.
    • If the median time passed of the last 11 blocks is greater or equal to the value of this field, then the transaction is “final”.
    • Otherwise, the transaction is “non-final”.

Only a “final” transaction may be confirmed in a block.

A new transaction must replace a prior “non-final” transaction if it has the same inputs in the same order, every sequence number for every input in the new transaction is not less than the sequence number for the corresponding input in the prior transaction, and the sequence number of at least one input in the new transaction is greater than the sequence number for the corresponding input in the prior transaction.

If a new transaction is detected which does not fulfil all of these requirements then it must be rejected.

If a new transaction is detected which has inputs that conflict with the inputs of a “non-final” transaction, but which are not identical to the inputs of the “non-final” transaction, then the “non-final” transaction is the “first seen” transaction and takes priority over the new transaction.

These rules form the basis of payment channels, which we shall in a future article.

Σ input values ≥ Σ output values

The sum of the value of the inputs of a transaction must be greater than or equal to the sum of the values of the outputs.

When transactions are created in Bitcoin, they must spend what are known as ‘Unspent Transaction Outputs’ (UTXOs). UTXOs are the live coins available to be spent on the network. The ledger represents the cumulative transaction history of those coins from their distribution as part of the block reward. Every node has a set of UTXOs it manages which it curates according to the operator’s chosen local policies.

Each UTXO holds a quantity of bitcoin satoshi tokens locked in a script. When the locking script is successfully executed in the script engine using an unlocking script, the satoshi tokens are released to be spent. 

To be successful, the full script must terminate with a single non-zero value remaining on the stack. The transaction then re-allocates the satoshis to new outputs which themselves become UTXOs, replacing their predecessors in the current UTXO set across the network.

If a transaction tried to create outputs that cumulatively represent more value than the inputs it would be spending, it would be creating new satoshi tokens which is expressly forbidden by the rules. In this way the number of tokens usable on the network remains fixed over time, with tokens only distributed on to the ledger as a reward to node operators in the initial bootstrap phase of the network.

Coinbase maturity rule

Nodes may not spend the outputs of a Coinbase transaction in a block that is less than 100 blocks higher than the one the Coinbase appears in.

This coinbase maturity rule ensures that a few things happen:

1. Node operators are prevented from using outputs from blocks that might be involved in orphan races to buy goods or services.
2. Nodes are incentivised to hash based on a 24 hour average price, reducing volatility on the hashrate being applied and keeping block discovery rates more constant.

This is an example of a constant rule which was applied to the system from the beginning. This is one of several such rules included by the creator which are held stable through consensus.

Bitcoin transaction format rule

Transactions must conform to the data formatting rules of the Bitcoin protocol, including respecting the sizes of certain fields and their encoding schemas.

The currently accepted versions of the Bitcoin transaction serialisation rules specify the format in a very specific manner, as outlined below.

1. Transaction version - A value defining the version of the protocol that should be used to evaluate the transaction (4 bytes).
2. The number of inputs being spent (VARINT 1, 3, 5 or 9 Bytes).
3. Each of the inputs themselves which are each comprised of the following:

1. 1. TXID of the transaction containing the UTXO being spent (32 Bytes).
   2. The VOUT index of the UTXO (4 Bytes).
   3. A field defining the length of the unlocking script, or scriptSig (VARINT, 1, 3, 5 or 9 Bytes).
   4. The scriptSig needed to spend the input.
   5. The scriptSig’s sequence number, which is used in the creation and use of payment channels (4 Bytes).

4. The number of outputs being created (VARINT, 1, 3, 5 or 9 Bytes).
5. The outputs themselves which are each comprised of the following:

1. 1. The value in satoshis being locked into the output script (8 Bytes).
   2. A field defining the length of the scriptPubKey (VARINT, 1, 3, 5 or 9 Bytes).
   3. The output scriptPubKey or locking script.

6. nLockTime which is the time at which the transaction outputs can be spent. If the transaction has an input with a non-final sequence number and an nLockTime in the future, it is considered to be inside a payment channel. These transactions cannot be included in a block until the sequence number is final or nLockTime expires, either of which closes the payment channel. The transaction can be updated by submitting a new valid version with an increased nSequence value for at least one of the non-final inputs

While these rules are rigid and uniformly applied across all current transaction versions, there is significant flexibility within the rules. The hard limits imposed by the protocol itself allow for a transaction to create up to 2^64 outputs, with each output able to express a length field large enough to insert up to 18 Exabytes of data. These limits are theoretical of course, with limitations on actual usage being framed by the economic reality faced by nodes in the operation of the system.

Bitcoin script language rules

The Bitcoin scripting language and its specification

One of the aspects of Bitcoin which gives it such a broad range of applicability and function is its scripting language. Based on Forth, the language is stack based and uses Reverse Polish Notation as the means to enter and process data. While the language appears simple, when used properly it can provide a Turing complete environment within which complex and diverse applications can be built.

The scripting language is made up of a set of 186 opcodes which each instruct operations on the processing stack. Any node that wishes to process transactions on the Bitcoin network must ensure that their node client implementation is processing each of the opcodes in a way that exactly matches the outcomes expected by every other node on the network at that time, including whether or not those opcodes are enabled or not.

Even a minor change in the way opcodes are processed can result in transactions that were committed to the ledger being rendered unspendable and causing irreparable damage to the system’s integrity and usability. It is because of these reasons that it is of vital importance that every node processes each opcode in the script in exactly the same manner.

Interestingly, this also means that bugs that existed in the execution algorithms must also be upheld. Notably, a bug in the OP_CHECKMULTISIG opcode requires that an extra data item be added to the stack before the first signature or the opcode will fail to execute properly. For this reason, anyone spending an output with OP_CHECKMULTISIG in it must add one extra data item to their script. Additionally, any node clients that did not take this known bug into account would incorrectly validate scripts that use the opcode, potentially causing them to reject transactions or blocks that should be considered valid, or accepting transactions and blocks which the rest of the network considers invalid.

The following rules are applied for all transactions

Bitcoin data types

All data items in Bitcoin Script are a byte sequence. Some operations interpret their parameters as numeric or boolean values and require the item to fulfil the specifications of those types. Some operations produce items on the stack which are valid numeric or boolean values.

A byte sequence has a length and a value. The length of the byte sequence must be an integer greater or equal to zero and less than or equal to 232−12^{32-1} 232−1 (UINT32_MAX).

The byte sequence of length zero is called the “null value”.

Any data item can be interpreted as a boolean value. If the data item consists entirely of bytes with value zero, or the data item is the null value, then the boolean value of the item is false. Otherwise, the boolean value of the item is true.

A data item can be interpreted as a numeric value. The numeric value is encoded in a byte sequence using little-endian notation. When script items are processed using opcodes that perform mathematical functions, the node will treat any byte sequence of up to 7500 bytes long as a numeric value, allowing for 'bignum' calculations to be performed in script.

Formal grammar for Bitcoin script

The Formal Grammar for Bitcoin Script is set by node operators. This contains the full set of approved opcodes and their exact spelling and function.

It’s also worth highlighting the following features of this formal grammar:

• The complete script consists of two sections, the unlocking script (scriptSig) and the locking script (scriptPubKey). The locking script is from the transaction output that is being spent, while the unlocking script is included in the transaction input that is spending the output.

• Current consensus rules state that an unlocking script can only contain the first 96 opcodes, which allow constants and data to be pushed onto the stack. This requirement is a part of the Validity of Script Consensus Rule, defined later.

• A branching operator (OP_IF or OP_NOTIF) must have a matching OP_ENDIF.

• An OP_ELSE can only be included between a branching operator and OP_ENDIF pair. There can only be at most one OP_ELSE between a branching operator and an OP_ENDIF.

• OP_RETURN may appear at any location in a valid script. The functionality of OP_RETURN has been restored and is defined later in the section OP_RETURN Functionality. Grammatically, any bytes after an OP_RETURN that is not in a branch block are not evaluated and there are no grammatical requirements for those bytes.

Note that disabled operations are part of this grammar. A disabled operation is grammatically correct but will produce a failure if executed.

Validity of Bitcoin script rule

The locking and unlocking scripts for every input of a transaction must be grammatically valid, as defined by the formal grammar rules from the previous page.

Also note that the scripts must be grammatically valid when they are spent. It is not required that the output scripts of a transaction are grammatically valid although it is highly recommended that client software implement this restriction as a policy.

The unlocking scripts used in transaction inputs may only contain PUSHDATA operations, as defined by the formal grammar above.

Currently, the following 5 opcodes are disabled: OP_2MUL, OP_2DIV, OP_VER, OP_VERIF, OP_VERNOTIF.

Numeric value size rule

For a byte sequence to validly represent a numeric value, the length of the byte sequence must be less than or equal to 750,000 bytes. A byte sequence that is larger than this is a valid byte sequence but is not a valid numeric value.

Note that while some operations require parameters to be valid numeric values, they may produce byte sequences which are not valid numeric values (for example, OP_MUL may produce a byte sequence which is too large to validly represent a numeric value).

Bitcoin stack memory usage rule

The stack memory usage consensus rule limits the amount of memory that can be used on the stacks. This rule is evaluated against the sum of the memory used by the stack and the alt-stack.

If the execution of the unlocking and locking script for an input uses more memory than defined in this rule, then the transaction is invalid.

The memory usage of a stack is calculated using a specific formula, so that it can be coordinated across software implementations. The formula for calculating the memory usage of a stack is:

usage = sum of (for each element: 32 + size in bytes of element)

where "size in bytes of element" is the size in bytes of the element when serialised in the Bitcoin Serialisation Format.

This is a configurable consensus rule, with a default value that is formally unlimited but will in practice depend on the capabilities of the system that is evaluating the script.

Miners are expected to reach consensus on this value and configure it manually.

Other Bitcoin rules

There are some other less important, but equally set-in-stone rules which are checked. These include the sunsetting of Pay to Script Hash (P2SH), the return of the original OP_RETURN functionality and the removal of many limits and restrictions that have previously been applied at the protocol level. 

Where required, node limits have been tuned to numbers only bounded by the hardware/software implementation of the node client, with configuration settings allowing each node operator to manage the in-use settings themselves. This is generally in loose agreement with the rest of the network. 

Node operators are free to use their own configurations, and to reach out to other known node enterprise organisations to ensure their blocks can be managed.

Standard local policies

Policies are settings that are configured by software operators. These settings are generally required by software implementations. It is expected that as implementations of the software mature that these settings will be adjusted.

Policies control which transactions the software will propagate across the P2P network or include in a block. However, policies are not Bitcoin Rules or Consensus Rules and are not used to determine the validity of blocks or the transactions confirmed by a block.

Standard local transaction policies

Maximum Acceptable Transaction Size Policy

The maximum acceptable transaction size is a standard policy that configures the largest transactions that the software will propagate across the P2P network or include in a block.

The default value for this policy setting is 10MB (10,000,000 bytes).

Transaction evaluation timeout

The transaction evaluation timeout is a standard policy that defines the maximum amount of time that the software will allow for the evaluation of a transaction before terminating the evaluation and rejecting the transaction. This setting is always defined with a time unit and the default value is 1 second.

This policy applies to transactions that are received by the software and its processes for determining whether the transaction should be propagated across the P2P network or included in a block. The policy does not apply to the evaluation of transactions in a confirmed block.

Note that:

• if the evaluation of a transaction is terminated due to exceeding the timeout limit then the software has not determined whether the transaction is valid or invalid, it has merely decided not to make that determination;
• the time taken to evaluate a transaction by a particular system will vary based on the software used, the resources available, and other dynamic factors such as the load on the system. It is not an exact measurement.

Minimum fee policies

The minimum fee rule specifies a minimum fee for which the node will accept and/or relay unconfirmed transactions.

This rule is set by the node and determines the minimum fee that a node will accept for mining a transaction into a block.

Currently fee rates are set by determining the size of the transaction in kilobytes and applying a ‘cost per kilobyte’ in Satoshis for the acceptance or relay of that transaction.

The transaction minimum fee rates are set individually by nodes. This means that nodes are free to mine or cache transactions as befits their business model. Mining transactions that other nodes reject however, comes with a much higher risk of nodes losing blocks. 

Nodes attempting to mine many transactions which are not being cached by other nodes must be exceptionally well connected to the network in order to minimise the time needed to transfer the block announcement and those transactions to other nodes.

Over time it is expected that fee calculation will become more complex, providing node operators many opportunities to value-add. This will be something that evolves over time, and techniques such as MinerID and MerchantAPI will allow node operators to create, advertise and manage their own unique business models.

"Dust" policies

The dust rule is a setting that nodes use to discourage Bitcoin users from creating outputs that contain fewer satoshis than it costs to on-spend them at the minimum fee rate. The goal of these policies is to prevent large numbers of low value outputs being used as an attack on the Bitcoin network.

Since early in the history of Bitcoin the dust rate has been a standard policy across the network.

The current policy implementation is defined such that smallest value a transaction’s output may contain is equal to (3x the size of the output script + 142) * the minRelayTxFee parameter.

In addition, transaction outputs are assessed against the dustRelayFee parameter. The current policy implementation is defined such that an output must be equal to (3x the size of the output + 142) * the dustRelayFee parameter in order for the node to propagate the transaction.

Typically these values are set to the same amount to minimise the complexity of generating small value outputs.

It is likely that the system will trend towards having minimal rules around the generation of transaction outputs based on the number of satoshis they contain.

Standard local script language policies

Script Language Policies apply to the execution of script.

Minimal encoding policy

The Minimal Encoding Policy mandates that any push operations done within a script must use the most efficient pushdata opcode to do so. Bitcoin has individual pushdata opcodes to push data items up to 75 bytes in length using a single byte instruction. 

For data items which are between 76 and 255 bytes, the sequence OP_PUSHDATA1 must be used, where the data length is defined using an 8 bit unsigned integer value. For data items between 256 and 65,535 bytes long, the sequence OP_PUSHDATA2 must be used, where the data length is defined using a 16 bit unsigned integer value. 

For data items between 65,536 and 4,294,967,295 bytes long, the sequence OP_PUSHDATA4 must be used, where the data length is defined using a 32 bit unsigned integer value. Data items larger than 4,294,967,295 bytes must be split in order to be pushed into a transaction.

Numeric Value Length

The length of numeric value policy defines the maximum length of a byte sequence to be considered a valid numeric value. The default value for this policy is 250,000 bytes.

Transactions which contain scripts that consume numeric values that are larger than this policy setting will be rejected; they will not be propagated across the P2P network or be included in a block (Genesis upgrade specification).

Stack Memory Usage Policy

The stack memory usage policy limits the amount of memory that can be used on the stacks. This policy is evaluated against the sum of the memory used by the stack and the memory used by the alt-stack.

If the execution of the unlocking and locking script for an input uses more stack memory than defined in this policy, then the transaction is rejected.

The memory usage of a stack is calculated using the same formula described in the Stack Memory Usage Consensus Rule (above).

The default value for this policy is 100MB (100,000,000 bytes). The value of this policy must be less than or equal to the value of the Stack Memory Usage Consensus Rule.

Standard local P2P network policies

P2P Network Connection Policies

Nodes on the Bitcoin network are free to connect or disconnect to any device, user or service that the incentive structure of the network encourages them to connect to. Certain nodes will operate with very different interconnection policies than other nodes, and nodes are free to reject or shut down connections as they see fit.

Each node’s settings are unique and the Bitcoin node client also contains a robust set of conditional monitoring systems which will cause disruptive nodes to be automatically disconnected if they propagate invalid blocks or transactions.

Propagation of non-standard transactions

The local policy on the propagation of non-standard transactions controls whether the software will propagate non-standard transactions to other peers on the P2P network.

Before Genesis activation, the default setting for this policy was that non-standard transactions will not be propagated across the P2P Network.

After Genesis activation, the default setting for this policy is that non-standard transactions will be propagated across the P2P Network.

Note that even though non-standard transactions will be propagated across the P2P network after Genesis activation, other policies such as the Maximum Acceptable Transaction Size Policy remain in effect.