Bitcoin Notes

Welcome!

Bitcoin Notes is an updated list of hyperlinks, notes, and text I found to be useful while learning about Bitcoin.

A lot of these notes were written quickly or copy/pasted from sources. I am working on adding footnotes.

If you find any mistakes, Twitter DM is best way to reach me.

Table of Contents

1 Bitcoin

1.2 Notes from Mastering Bitcoin

Some quick notes from aantonop's Mastering Bitcoin book.

1.2.1 Glossary

  • Full client
    • A full client, or "full node," is a client that stores the entire history of bitcoin transactions (every transaction by every user, ever), manages the users' wallets, and can initiate transactions directly on the bitcoin network.
      • This is similar to a standalone email server, in that it handles all aspects of the protocol without relying on any other servers or third-party services.
  • Lightweight client
    • A lightweight client stores the user’s wallet but relies on third-party–owned servers for access to the bitcoin transactions and network. The light client does not store a full copy of all transactions and therefore must trust the third-party servers for transaction validation.
  • Web client
    • Web clients are accessed through a web browser and store the user’s wallet on a server owned by a third party.
  • wallet
    • A wallet is simply a collection of addresses and the keys that unlock the funds within. You can increase privacy by using a different address for every transaction. Practically no limit to # of addresses a user can create.

1.2.2 Transactions

  • When spending bitcoin, the current bitcoin owner presents her public key and a signature (different each time, but created from the same private key) in a transaction to spend those bitcoin.
  • The bitcoin network can transaction in fractional values down to 1/100,000,000th of a bitcoin (satoshi)
  • Transactions are like lines in a double-entry bookkeeping ledger.
    • each transaction contains one or more “inputs” which are debits against a bitcoin account
    • The other side of the transaction, there are one or more “outputs” which are credits added to a bitcoin account
    • outputs add up slightly less than inputs. The difference represents an implied transaction fee collected by the miner
    • Transactions move value from transaction inputs to transaction outputs. An input is where the coin value is coming from, usually a previous transaction’s output. A transaction output assigns a new owner to the value by associating it with a key. The destination key is called an encumbrance. It imposes a requirement for a signature for the funds to be redeemed in future transactions. Outputs from one transaction can be used as inputs in a new transaction, thus creating a chain of ownership as the value is moved from address to address
    • “Spending” is signing a transaction that transfers value from a previous transaction over to a new owner identified by a bitcoin address
  • common transaction
    • one input, two outputs
    • One address to another, including one output back to original address as “change”

*aggregate transaction

  • multiple inputs, one output
  • Real-work equivalent of exchanging a pile of coins and current notes for a single larger note.
  • distributing transaction
    • one input and multiple outputs
    • Common transaction for commercial payrolls
  • construction a transaction:
    • Alice’s wallet application contains all the logic for selecting appropriate inputs and outputs to build a transaction to Alice’s specification. Alice only needs to specify a destination and an amount and the rest happens in the wallet application without her seeing the details. Importantly, a wallet application can construct transactions even if it is completely offline. Like writing a check at home and later sending it to the bank in an envelope, the transaction does not need to be constructed and signed while connected to the bitcoin network. It only has to be sent to the network eventually for it to be executed.
      • "This output is payable to whoever can present a signature from the key corresponding to Bob’s public address."
        • Because only Bob has the wallet with the keys corresponding to that address, only Bob’s wallet can present such a signature to redeem this output. Alice will therefore "encumber" the output value with a demand for a signature from Bob.
  • A common misconception about bitcoin transactions is that they must be "confirmed" by waiting 10 minutes for a new block, or up to 60 minutes for a full six confirmations. Although confirmations ensure the transaction has been accepted by the whole network, such a delay is unnecessary for small-value items such as a cup of coffee. A merchant may accept a valid small-value transaction with no confirmations, with no more risk than a credit card payment made without an ID or a signature, as merchants routinely accept today.
  • Every 10 minutes on average, miners generate a new block that contains all the transactions since the last block.
    • As miners build a new block, they add unverified transactions from this pool to a new block and then attempt to solve a very hard problem (a.k.a., proof of work) to prove the validity of that new block.
  • Scalability of bitcoin is measured by tps (transactions per second)
  • Transactions are prioritized by the highest-fee transactions and a few other criteria.
  • Over time, as the hight in blocks increases, so dose the computation difficulty for each block and chain as a whole.
    • By convention, any block with six confirmations is considered irrevocable

1.2.3 Keys, Addresses, Wallets

  • Digital keys in a user’s wallet are completely independent of the bitcoin protocol and can be generated and managed by the user’s wallet software without reference to the blockchain or access to the internet.
  • Most bitcoin transactions requires a valid digital signature to be included in the blockchain. Can only be generated with a secret key. The digital signature used to spend funds is also referred to as a witness, a term used in cryptography. Witness data in a transaction testifies to the true ownership of the funds being spent.
  • When spending bitcoin, the current bitcoin owner presents her public key and a signature (different each time, but created from the same private key) in a transaction to spend those bitcoin. Through the presentation of the public key and signature, everyone in the bitcoin network can verify and accept the transaction as valid, confirming that the person transferring the bitcoin owned them at the time of the transfer.
  • Public Key Cryptography and CryptoCurrency
    • Bitcoin uses elliptic curve multiplication for public key crypto
    • Public key crypto creates a key pair that controls access to bitcoins.
    • Key pair consists of a private key and -derived from it - a unique public key. Public key is used to receive bitcoins, private key is used to sign transactions
    • Private key (k) is a number usually picked at random. From the private key, we use elliptic curve multiplication, a one-way crypto function to generate a public key (K).
    • From the public key (K), we use a one way crypto hash function to generate a bitcoin address (A)
    • Private key:
      • random - pick a number between 1 and 2(256)
      • 256 binary digits shown as 64 hexadecimal digits, each 4 bits
      • Bitcoin’s private key space is 2256, approximately 1077 in decimal. visible universe contains 1080 atoms…

1.3 Wallet Design

Wallet Design concerns all things related to wallet functionality. This mostly is application level logic.

1.3.1 BIP32

Wallets use HD Trees:
Deep Dive on Extended Keys

1.3.2 SegWit

Since SegWit, couple of changes to wallets were needed:
SegWit wallet dev guide

One of the immediate problems that SegWit solves is mitigating transaction malleability.

1.3.3 Vaults

vaults w/o covenants
more by bishop

proof of reserves - blockstream

BIP-127 proposes a standard way to do proof of reserves using a PSBT extension.
link to bip

There's rust implementation of a Proof-of-Reserves Client. link to reserves

custody protocols using bitcoin vaults

1.3.4 Batching

Payment batching, more here, is including multiple payments inside a single transaction.

Variables to consider are # of inputs and # of outputs. Better to have a single input and many outputs.
It is also nice to have a lower fee for the entire transaction.

Goal of batching is to lower vbytes per payment. Marginal improvmenent after 1 input and 5 outputs.

1.3.5 Coin Selection

Challenges of coin selection by lopp
iohk on coinsel
what is coinsel?
murch transcript at scaling bitcoin
edge++ transcript

The naive approach would be to simply look for the smallest output that is larger than the amount you want to spend and use it, otherwise start adding the next largest outputs until you have enough outputs to meet the spend target. However, this leads to fracturing of outputs until the wallet becomes littered with unspendable “dust.”

“Our idea is to have the user the option (either global or per account or
per transaction) to choose between "maximize privacy" or "minimize fees"
(or even maybe "minimize UTXO"

”Dust” refers to transaction outputs that are less valuable than three times the mininum transaction fee and are therefore expensive to spend.

A transaction output is labeled as dust when its value is similar to the cost of spending it. Precisely, Bitcoin Core sets the dust limit to a value where spending an 2.3. Transactions 7 output would exceed 1/3 of its value. This calculation is based on the minimum relay transaction fee, a node setting that causes transactions that don’t at least include this lower bound of fee to be dropped from the memory pool, and not relayed to other nodes. With the default for the minimum relay transaction fee set to 1 000 satoshi per kilobyte, and the sizes of a P2PKH input being 148 bytes, and an output being 34 bytes, this computes to all outputs smaller or equal to 546 satoshis being considered dust by Bitcoin Core [Erha15].

utxo mgmt for enterprise wallets

1.3.6 Bitcoin Core Wallet

Bitcoin Core's wallet is always evolving. Some changes to the Bitcoin Core wallet:
Wallet Class Structure Changes
Sipa describing wallet changes
Wallet Architecture transcripts

1.3.8 Script

(https://en.bitcoin.it/wiki/Contract)
Each transaction input has a sequence number. In a normal transaction that just moves value around, the sequence numbers are all UINTMAX and the lock time is zero. If the lock time has not yet been reached, but all the sequence numbers are UINTMAX, the transaction is also considered final.

Sequence numbers can be used to issue new versions of a transaction without invalidating other inputs signatures, e.g., in the case where each input on a transaction comes from a different party, each input may start with a sequence number of zero, and those numbers can be incremented independently.

Signature checking is flexible because the form of transaction that is signed can be controlled through the use of SIGHASH flags, which are stuck on the end of a signature. In this way, contracts can be constructed in which each party signs only a part of it, allowing other parts to be changed without their involvement. The SIGHASH flags have two parts, a mode and the ANYONECANPAY modifier:

  1. SIGHASHALL: This is the default. It indicates that everything about the transaction is signed, except for the input scripts. Signing the input scripts as well would obviously make it impossible to construct a transaction, so they are always blanked out. Note, though, that other properties of the input, like the connected output and sequence numbers, are signed; it's only the scripts that are not. Intuitively, it means "I agree to put my money in, if everyone puts their money in and the outputs are this".
  2. SIGHASHNONE: The outputs are not signed and can be anything. Use this to indicate "I agree to put my money in, as long as everyone puts their money in, but I don't care what's done with the output". This mode allows others to update the transaction by changing their inputs sequence numbers.
  3. SIGHASHSINGLE: Like SIGHASHNONE, the inputs are signed, but the sequence numbers are blanked, so others can create new versions of the transaction. However, the only output that is signed is the one at the same position as the input. Use this to indicate "I agree, as long as my output is what I want; I don't care about the others".

There are two general patterns for safely creating contracts:

  1. Transactions are passed around outside of the P2P network, in partially-complete or invalid forms.
  2. Two transactions are used: one (the contract) is created and signed but not broadcast right away. Instead, the other transaction (the payment) is broadcast after the contract is agreed to lock in the money, and then the contract is broadcast.

This is to ensure that people always know what they are agreeing to.
Together, these features let us build interesting new financial tools on top of the block chain.

It may even be that people find themselves working for the programs because they need the money, rather than programs working for the people.

old oracle services…
https://docs.oraclize.it/#home
http://orisi.org/
http://earlytemple.com/
https://en.bitcoin.it/wiki/Contract#Example_4:_Using_external_state

  1. Scriptless Scripts

1.3.9 Fee Estimation

lopp on fee estimation

Fee estimation is the process of estimating a particular fee rate to use for a transaction in order to incentivize block inclusion at a particular block target.

Supply (blocks) and demand (txns) are unpredicable.

John Newbery's intro to Bitcoin Core Fee Estimation
pt2

  1. Outline of Newbery's post

    At broadcast, the transaction is not going to get into the next block. But rather likely the next block in 10 minutes. Block production follows Poisson distribution.

    As a result, the fee rate should be competitive not only of the current mempool but the likely mempool in ten minutes.

    Looking only at mempool does not consider lucky block runs.

  2. Bitcoin Core's Fee Estimation

    High level desc of Bitcoin Core's fee estimation algorithm
    code
    Bitcoin core groups transaction fee rates into buckets. Each buck is a range of fee rates. A track of block targets from 1 block to 1008 blocks is kept.

    Also, the following is recorded:
    (A) number of transactions that entered the mempool in each fee rate bucket.
    (B) for each bucket-target pair, the number of transactions that were included in a block within the target number of blocks.

    For any target-bucket pair, Bitcoin Core can find the probability that a transaction with the fee rate can be included. This is B/A.

    Additional overview

  3. Mempool File Format

    Mempool File Format can be useful for fee estimation..
    talk by kalle
    Time series of a txn lifecyle until block inclusion in a small file format.

    https://github.com/kallewoof/mff

1.4 Taproot/Schnorr

Notes on BIP340-342. All things concerning Schnorr, Taproot, and Tapscript.

1.4.1 Introduction

Slides from Sipa talk at SF Bitcoin Devs.
Taproot Workshop
Taproot Review

immediate benefit of taproot: "if you lose this key, your funds are gone" to "if you lose this key, you'll have to recover 3 of your 5 backup keys that you sent to trusted friends, and pay a little more, but you won't have lost your funds"" - anthony towns

Notation to be used throughout (from BIP):

  • hashtag(x) notation to refer to SHA256(SHA256(tag) || SHA256(tag) || x)
  • q is taproot output key
  • p is taproot internal key

1.4.2 BIP340

Link to BIP340
Why only x-pubkey?

Proposes standard for 64-byte Schnorr signatures.

Why Schnorr?

  • Provable security
    Security of ECDSA rely on stronger assumptions. Schnorr signatures are provably secure.
  • Non-malleability
    Schnorr signatures are implied to be non-malleable given SUF-CMA security. ECDSA sigs are inherently malleable.
  • Linearity

Encoding: Instead of DER, use fixed 64-byte format.
Public Key Encoding: Instead of compressed 33-byte key, use 32 bytes.

Interestingly, the aim of the BIP is to have the Schnorr spec completely specified. In the past, different ECDSA implementations caused issues.

SuredBits' intro to Schnorr

"tweaking" involves hiding/obfuscation

security of blind discrete log signatures
generalized bday problem

x-only pubkeys
lattice attacks against weak ECDSA

A schnorr signature is defined as the following:
S = R + H(x(R)|P|m) * P
where R is the Nonce point (k*G)

To save 32 bytes, only the x value of R is provided by the signer.
The verifier can computer the y-value.

One of the y-coordinates is even while the other is odd.

Proposal constraints k such that y-value of R is quadratic residue module SECP256K1FIELDSIZE.
Quadaratic residue is having a square root modulo the field size.

If a randomly generated nonce k does not yield a valid nonce point R, then the signer can negate k to obtain a valid nonce.

1.4.3 BIP341

Link to BIP341

Proposes new SegWit v1 output type with spending rules based on Taproot, Schnorr, and merkle branches.

BIP claims no new security assumptions are added.

The aims of the output type is to improve privacy, efficiency, and flexibility of Bitcoin script. This is especially useful in minimizing how much information is shown on the blockchain regarding the spendability conditions. Additionally, a few bug fixes are included.

The BIP is very selective in the technologies that are included. Many are swept for later review in order to reduce complexity of review as well as prevent immature technology from weighind down ready technology.

From the BIP document, the following technologies compose the proposal:

  • Merkle Branches: Reveal the actual executed part of the script.
  • Taproot: Merge pay-to-pubkey and pay-to-scripthash policies making outputs spendable by either indistiguishable.
    As long as key-based spending path is used for spending, it is not revealed whether a script path was permitted as well.
    An assumption is made that most outputs can be spent by all parties agreeing. Schnorr permits key aggregation1.

Key aggregation allows a public key to be constructed from multiple participant keys. Indistinguishable from single-party.

  • Batch validation is permited with schnorr signatures.
  • Every merkle tree has an associated version allowing for new script versions to be introduced via soft fork. Unused 'annex' in the witness can also be used.
  • New Signature Hashing Algorithm includes amount and ScriptPubKey in message. And uses tagged hashes.
  • The public key is directly included in the output.

BIP can be informally summarized in the following way:

a new witness version is added (version 1), whose programs consist of 32-byte encodings of points Q. Q is computed as P + hash(P||m)G for a public key P, and the root m of a Merkle tree whose leaves consist of a version number and a script. These outputs can be spent directly by providing a signature for Q, or indirectly by revealing P, the script and leaf version, inputs that satisfy the script, and a Merkle path that proves Q committed to that leaf. All hashes in this construction (the hash for computing Q from P, the hashes inside the Merkle tree's inner nodes, and the signature hashes used) are tagged to guarantee domain separation.

A taproot output is a native SegWit output with version number 1 and a 32-byte witness program.

Every taproot output corresponds to a combination of a single public key condition (internal key), and zero or more general conditions encoded in scripts in a tree.

General guidelines for construction and spending Taproot outputs:

  • Better to split scripts with conditionls (OPIF) into multiple scripts in the tree…each corresponding to one execution path.
  • When a single condition requires signautres from multiple keys, key aggregation MuSig can be used.
  • Most likely key to be used should be the internal key. If no such condition exists, worthwhile adding one that consists of an aggregation of all keys. This is an "everyone agrees" branch. Else just pick an internal key using a point wi unknown discrete logarithm. See BIP for example.
  • If no script conditions needed, an output key should commit to an unspendable script path instead. See BIP for how to achieve this.
  • Remaining scripts should be organized into leaves. Huffman tree.
  • Binary tree leaves are (leafversion, script) tuples.

Q=P+H(P,m)*G
where P is public key and m is merkle root of a MAST.

switchable scripting

1.4.4 BIP342

Link to BIP342

Proposes semantics of the scripting system described in BIP341.

Includes improvements to schnorr signatures, batch validation, and signature hash.

OPCHECKSIG and OPCHECKSIGVERIFY are modified to verify schnorr signatures.
OPCODESEPARATOR simplified.

OPCHECKMULITSIG and OPCHECKMULTISIGVERIFY are disabled. OPCHECKSIGADD is introduced to make multisigs batch-verifiable.

A potential malleability vector is eleminated by requiring MINIMALIF. Using a non-standard represetentation of true for OPIF is now considered invalid as a violation of consensus rules.

OPSUCCESS opcodes allows introducing new opcodes cleanly than through OPNOP.

Tapscript can be upgraded through soft forks by defining unknown key types. For example, adding a new hashtypes or signature algorithms.

1.4.5 MuSig

  1. MuSig2

    Exchanging nonce commitments is the subject of the MuSig-DN paper.

    Nonce commitment exchange can be removed by generating the nonce deterministically from the signers' public keys and message.
    Providing a non-interactive zk proof that the nonce was generated deterministically along with the nonce.

    The MuSig2 scheme has a two round signing protocol w/o the need for a sk proof.
    Also, the first round of the nonce exchange is done at key setup time.

    Therefore, there are two variants: interactive setup and non-interactive setup.

    BitcionOps explains MuSig2

    MuSig2

1.4.6 SIGHASHANYPREVOUT

proposed bip

a new type of public key for tapscript (bip-tapscript) transactions. It allows signatures for these public keys to not commit to the exact output being spent. This enables dynamic binding of transactions to different UTXOs, provided they have compatible scripts.

Allows dynamic rebinding of a signed transaction to another previous output of the same value

1.5 Mining

All things Bitcoin mining.

1.5.1 Introduction

Excellent podcast on mining

cgminer is open source miner for ASIC/FPGA miner. Lots of companies forked off this original miner.
https://github.com/ckolivas/

channel payouts in mining

1.5.2 GetBlockTemplate

Getblocktemplate: bitcoin core <-> pool server

1.5.3 Stratum

Stratum: pool server <-> asic controller
Stratum Protocol documentation
The design of the Stratum protocol requires pool operators to build and distribute block templates to their clients.

1.5.4 StratumV2

1.5.5 Betterhash

  • Work protocol: bitcoin
  • core <-> mining proxy
  • Work protocol: mining proxy/bitcoin core <-> asic controller
  • Pool protocol: pool server <-> mining proxy

link to bip
betterhash overview

1.5.6 Compact Blocks

faq for compact blocks
Compact block relay, BIP152, is a method of reducing the amount of bandwidth used to propagate new blocks to full nodes.

Using simple techniques it is possible to reduce the amount of bandwidth necessary to propagate new blocks to full nodes when they already share much of the same mempool contents. Peers send compact block “sketches” to receiving peers.

1.6 P2P

P2P layer of Bitcoin.
For the Bitcoin network to remain in consensus, the network of nodes must not be partitioned. So for an individual node to remain in consensus with the network, it must have at least one connection to that network of peers that share its consensus rules.

partition resistance

1.7 Lightning Network

Lightning Network and related off-chain protocols/
Master Lightning Book
ln overview
ln zero to hero
t-bast's notes
Gist: LN keeping most transactions off-chain and leveraging the security of the underlying blockchain as an arbitration layer.
payment channels - two parties commit funds and pay each other by updating the balance redeemable by either party in the channel.
The process is instant and saves users from having to wait for block confirmations before they can render goods or services.

Hash Time-Locked Contracts (HTLCs) allow transactions to be sent between parties who do not have a direct channels by routing it through multiple hops, so anyone connected to the Lightning Network is part of a single, interconnected global financial system.

Payment channels are the main workhorse of the Lightning Network. They allow multiple transactions to be aggregated into just a few on-chain transactions. In the vast majority of cases, someone only needs to broadcast the first and last transaction in the channel.

  • The Funding Transaction creates the channel. During this stage, funds are sent into a multisig address controlled by both Alice and Bob, the counterparties to the channel. This address can be funded as a single-payer channel or by both Alice and Bob.
  • The Closing Transaction closes the channel. When broadcast, the multisig address spends the funds back to Alice and Bob according to their agreed-upon channel amount.

channel updates

  • In between the opening and closing transactions broadcast to the blockchain, Alice and Bob can create a near infinite number of intermediate closing transactions that gives different amounts to the two parties.
  • For example, if the initial state of the channel credits both Alice and Bob with 5BTC out of the 10BTC total contained in the multisig address, Alice can make a 1BTC payment to Bob by updating the closing transaction to pay 4BTC/6BTC, where Alice is credited with 4BTC and Bob with 6BTC. Alice will give the signed transaction to Bob, which is equivalent to payment, because Bob can broadcast it at any time to claim his portion of the funds. 
    • To prevent an attack where Alice voids her payment by broadcasting the initial state of 5BTC/5BTC, there needs to be a way to revoke prior closing transactions. Payment revocation roughly works like the following.
    • Alice must wait 3 days after broadcasting the closing transaction before she can redeem her funds. During this time, Bob is given a chance to reveal a secret that will allow him to sweep Alice’s funds immediately. Alice can thus revoke her claim to the money in some state by giving Bob the secret to the closing transaction. This allows Bob to take all of Alice’s money, but only if Alice attest to this old state by broadcasting the corresponding closing transaction to the blockchain.

Payment channels & revocable transactions
great graphical overview

txn:
Bob’s signature and a relative timelock (Bob’s spend branch); or
Alice’s signature and a secret revocation hash provided by Bob (Alice’s revocation branch).

usually have multiple utxos. Once bob reveals his secret, alice can collect her spend TXO and rTXO.

revocable transaction scriptpubkey:
OPIF # Bob's spend branch - after the revocation timeout duration, Bob can spend with just his signature
<TXO revocation timeout duration> OPCHECKSEQUENCEVERIFY OPDROP
<Bob's public key>
OPELSE # Revocation branch - once the revocation pre-image is revealed, Alice can spend immediately with her signature
OPHASH160 <h(rev)> OPEQUALVERIFY OPDROP
<Alice's public key>
OPENDIF
OPCHECKSIG

recovcation keys used base points and blinding key. similar to bip32, keys derived using base key.

revocable transactions
HTLCs

enterprise lightning presentation

1.7.1 BOLTs

great overview of BOLT by Jim Posen
how onion routing works with HTLCs

presentation by Rene

BOLT is the Basics of Lightning Technology.

The BOLT repo found here describes the specification for the Lightning Network.

  1. BOLT #0

    Provides a basic glossary defining terminology that is used throughout the rest of the specification.

  2. BOLT #1

    Describes the base message protocol including the TLV format and the setup messages.

    TLV is Type-Length-Value.

    Funny enough, the unicode code point for lightning is 0x2607. In decimal, 9735 which is also the default TCP port.

  3. BOLT #2

    Contains peer channel protocol lifecycle.

    A channelid is used to identify a channel. channelid = XOR(fundingtxid, fundingoutputindex)

    Before a channel is created, a temporarychannelid is used which acts a nonce. This nonce is local and can be duplicate across the rest of the protocol.

    1. Channel Establishment

      ------- -------

        –(1)— openchannel –—>  
        <-(2)– acceptchannel -----  
           
      A –(3)– fundingcreated —> B
        <-(4)– fundingsigned -----  
           
        –(5)— fundinglocked -—>  
        <-(6)— fundinglocked -----  

      ------- -------

      • where node A is 'funder' and node B is 'fundee'

      An outpoint is provided to B at step 3.

    2. Channel Close

      ------- -------

        –(1)–— shutdown --–—>  
        <-(2)–— shutdown --------  
           
        <complete all pending HTLCs>  
      A B
           
        –(3)– closingsigned F1—>  
        <-(4)– closingsigned F2----  
         
        –(?)– closingsigned Fn—>  
        <-(?)– closingsigned Fn----  

      ------- -------

    3. Normal Operation

      Once both nodes have exchanged fundinglocked, the channel is used to make payments with HTLCs.

  4. BOLT #3

    Describes transaction and script formats.

  5. BOLT #4
  6. BOLT #5

    Channels can end with a mutual close, unilateral close, or a revoked transaction close.

    In a mutual close, local and remote nodes agree to close. They generate a closing transaction.

    In a unilateral close, one side publishes its latest commitment transaction.

    In a revoked transaction close, one party is cheating and publishes an oudated commitment transaction.

    A commitment transaction has up to six types of outputs:

    1. local node's main output: Zero or one output, to pay to the local node's delayedpubkey.
    2. remote node's main output: Zero or one output, to pay to the remote node's delayedpubkey.
    3. local node's anchor output: one output paying to the local node's fundingpubkey.
    4. remote node's anchor output: one output paying to the remote node's fundingpubkey.
    5. local node's offered HTLCs: Zero or more pending payments (HTLCs), to pay the remote node in return for a payment preimage.
    6. remote node's offered HTLCs: Zero or more pending payments (HTLCs), to pay the local node in return for a payment preimage.

    If the local node publishes its commitment transaction, it will have to wait to claim its own funds, whereas the remote node will have immediate access to its own funds.

  7. BOLT #7

    P2P

  8. BOLT #8
  9. BOLT #9
  10. BOLT #10
  11. BOLT #11

    Invoice spec.

1.7.2 anecdotal example

Suppose Alice has a channel with Bob, who has a channel with Carol, who has a channel with Dave: A<->B<->C<->D. How can Alice pay Dave?
Alice first notifies Dave that she wants to send him some money.
In order for Dave to accept this payment, he must generate a random number R. He keeps R secret, but hashes it and gives the hash H to Alice.

Alice tells Bob: “I will pay you if you can produce the preimage of H within 3 days.” In particular, she signs a transaction where for the first three days after it is broadcast, only Bob can redeem it with knowledge of R, and afterwards it is redeemable only by Alice. This transaction is called a Hash Time-Locked Contract (HTLC) and allows Alice to make a conditional promise to Bob while ensuring that her funds will not be accidentally burned if Bob never learns what R is. She gives this signed transaction to Bob, but neither of them broadcast it, because they are expecting to clear it out later.
Bob, knowing that he can pull funds from Alice if he knows R, now has no issue telling Carol: “I will pay you if you can produce the preimage of H within 2 days.”
Carol does the same, making an HTLC that will pay Dave if Dave can produce R within 1 day. However, Dave does in fact know R. Because Dave is able to pull the desired amount from Carol, Dave can consider the payment from Alice completed. Now, he has no problem telling R to Carol and Bob so that they are able to collect their funds as well.

Alice knows that Bob can pull funds from her since he has R, so she tells Bob: “I’ll pay you, regardless of R, and in doing so we’ll terminate the HTLC so we can forget about R.” Bob does the same with Carol, and Carol with Dave.

Now, what if Dave is uncooperative and refuses to give R to Bob and Carol? Note that Dave must broadcast the transaction from Carol within 1 day, and in doing so must reveal R in order to redeem the funds. Bob and Carol can simply look at the blockchain to determine what R is and settle off-chain as well.

1.7.3 Lightning Conf 2019 Berlin

electrum slides on lightning
Circular routes: send to self.
Suggestions

  • do not accept random peers
  • disallow invoices to blacklisted pubkeys

Command line tools

  • LNDmanage by @bitromortac
  • Balance of Satoshis by @alexbosworth
  • Rebalance-LND by @C-Otto

Make Me an Offer (Bolt 12) introduced.

LSAT

  • Macaroon - cryptographic bearer credential
  • Delegation possible
  • Chained HMAC construction
    • Secret root used to derive all others
  • Fine grained permission

Hedging the Chain

  • Bitcoin fee market
  • “Every biz using the blockchain is inherently short blockchain fees”
  • Derivatives traditionally used as a hedge
  • Corn farmers inherently long corn
  • They short corn futures as a hedge

Liquidity

  • No pairwise trades
  • different sources of liquidity is not the same
  • Set outbound liquidity to the same fee
  • Varied inbound liquidity
  • Make liquidity a pairwise market
  • External settlement mechanisms
  • Circular rebalancing

Attacks

  • Set min chan size …too many channels causes performance issues
  • Create a bunch of hold invoices and drain balance
  • Stealing free fees, someone sets up intermediate node between invoice and collects fees.

htlcs are harmful

1.7.4 Discrete Log Contracts

1.7.5 Security

  1. LSAT

    Lightning Service Authentication Token
    lsat talk

    using macaroon based bearer API credential with lightning network payment

  2. Key management

    talk on key mgmt
    need onchain hot wallet to open channels (only need once)

    1 of 2 keys must be hot for the funding transaction.
    If counterparty gets key, funds are lossed. If 3rd party gets it, they must collude.

    Commitment secret: must be hot.
    Used to generate "localpubkey" and "remotepubkey"
    Used to derive subsequent secrets and public keys.
    If leaked, peer can steal all money in commitment txn.

    Revocation basepoint secret: can be cold.
    Used to claim peer funds if they try to cheat.
    Can be cold if accessible before "toselfdelay"

    If your counterparty gets access to this key, they can claim their funds in their tolocal output immediately by circumventing the locktime

    Payment basepoint secret: claim money from the "toremote" output on peer commitment txn.
    can be cold
    if peer gets access to this key, all funds can be taken in the "toremote".

    Delayed Payment Basepoint Secret: claim money on "tolocal" output of commitment txn. can be cold.

    HTLC Basepoint Secret: secret needed to sign for HTLCs. must be hot.

    hosted channels
    gist on hosted channels
    interesting idea but need to look more into security assumptions..

1.7.6 Routing

Routing is generally constructed for a specified payment amount.
Other considerations, however, includes value of open channels, decision to make new channels, re-balancing decisions, multi-path payments or multi-part payments (MMP, formerly AMP).

amount independent routing

Rendezvous routing
rendezvous mechanism on top of sphinx

1.7.7 Trampoline Payments

Lightning network currently relies on source routing where sender calculates the route. Sender needs to maintain graph state.

Trampoline payments is a new suggested way of outsourcing that aims at having lite clients outsourcing the route computation to trampoline nodes, nodes of higher Memory, bandwidth and computation power.

design decisions on trampoline routing

1.7.8 HTLCs

HTLCs..Hashed Time Lock Contracts.

The initiator of a Lightning channel pays the closing fee. Lots of HTLCs = large fee. See thread.

Thread on free HTLC forwarding

An interesting idea to handling the edge cases around HTLCs is to have a firewall. An example.

1.7.9 PTLCs

1.7.10 Static/Send/Spontaneous/Push Payments

Wow, lots of names for an overlapping concept.

Offers
Static Payments
Push Invoices

1.7.12 revocationsecretderivation

TODO

1.8 Privacy

Privacy and techniques used in chain-analysis.
privacy wiki

snowball presentation at ldn bitdevs

common input hueristic: “different public keys used as inputs to a transaction as being controlled by the same user”
original paper on blockchain analysis

coin join wiki

1.8.1 CoinJoins

"So a world where "basically everyone uses CoinJoin" is cool for privacy, but could end up pretty bad for scalability, because these transactions are in addition to the normal payments." - waxwing

  1. PayJoin

    payjoin by waxwing
    PayJoin is coinjoin + payment

    "Let Bob do a CoinJoin with his customer Alice - he'll provide at least one utxo as input, and that/those utxos will be consumed, meaning that in net, he will have no more utxos after the transaction than before, and an obfuscation of ownership of the inputs will have happened without it looking different from an ordinary payment."

    "the main point is with PayJoin - we break the heuristic without flagging to the external observer that the breakage has occurred." … unlike coinjoins

    "snowball effect" … payjoin/p2ep reduces utxo set and receiver's utxo gets bigger after each payment txn.

    who pays for the fee?
    "every payment to the merchant creates a utxo, and every one of those must be paid for in fees when consumed in some transaction. "

    real world implementation is samourai wallet

    join market

  2. Pay To EndPoint (P2EP)

    p2ep blockstream
    "The basic premise of P2EP is that both Sender and Receiver contribute inputs to a transaction via interactions coordinated by an endpoint the Receiver presents using a BIP 21 compliant URI."

    Steps:

    1. Receiver generates a BIP 21 formatted URI with an additional parameter that specifies their P2EP endpoint.
    2. The Sender initiates interaction with the Receiver by confirming that the endpoint provided is available. If not, the transaction is broadcast normally, paying to the Receiver’s BIP 21 regular Bitcoin address. If the Receiver’s endpoint is available, the Sender provides a signed transaction to the Receiver as proof of UTXO ownership.
    3. The Receiver then sends a number of transactions to the Sender for them to sign. Out of these transactions, only one includes a UTXO that is actually the owned by the Receiver, the rest can be selected from the pool of spendable UTXOs.
    4. Receiver obtains a signed transaction that corresponds to their UTXO they can sign and broadcast the transaction, which will now contain inputs from both the Sender and the Receiver.

    Example:
    If Alice wants to pay Bob 1 BTC:

    1. Alice inputs 3 BTC to a transaction.
    2. Bob inputs 5 BTC to the same transaction.
    3. Alice receives 2 BTC (as her change).
    4. Bob receives 6 BTC (as his change, plus the 1 BTC payment from Alice).

    Disadvantages:
    Receiver and Sender must be online. Interactive.
    More Cons/Pros listed in blogpost.

1.8.2 BIP-79

1.8.3 CoinSwaps

maxwell on coinswaps
waxwing on coinswaps

"We can use a cryptographic commitment scheme to create atomicity that binds two, independent Bitcoin transactions"

Make a random x, hash it. Make a p2sh output that is spendable with proving hash(x) is hash in scriptpubkey and pubkey owns output.

Other party can see x and then solve for their p2sh with their pubkey.

great explainer on cross-chain swaps

problem here is that x is revealed and a connection exists between both parties.

HTLCs with presigned transactions can help avoid revealing x.
htlcs wiki

"An advantage of Coinswap over Coinjoin is a potentially bigger anonymity set (a lot more could be said)"

visual guide

implementation

new coinswap implementation

1.8.4 TumbleBit

waxwing on tumblebit
original paper

"A blind signature is allows a central authority to sign data which is hidden from them"

"Chaumian cash" is a central mint authorised to blind-sign transfers of this cash

" At a very high level, it's using commitments - I promise to have X data, by passing over a hashed or encrypted version, but I'm not yet giving it to you - and interactivity - two-way messaging, in particular allowing commitments to occur in both directions."

blind signatures

1.8.5 SNICKER

link to bip

SNICKER (Simple Non-Interactive Coinjoin with Keys for Encryption Reused)

allowing the creation of a two party coinjoin without any synchronisation or interaction between the participants.

1.8.6 PaySwap

1.8.7 More Cryptography

  1. Adaptor Signatures

    explainer using atomic swaps
    "An "adaptor signature" is a not a full, valid signature on a message with your key, but functions as a kind of "promise" that a signature you agree to publish will reveal a secret, or equivalently, allows creation of a valid signature on your key for anyone possessing that secret."

  2. Schnorr
  3. Ring Signatures

1.8.8 Chain Analysis

Peel chains are strings of transactions commonly used for money laundering, in which entities send funds through several wallets in quick succession, usually breaking off small amounts to cash out at each step and sending the majority on to the next wallet.

1.9 Security

1.9.1 Hardware Wallets

1.10 Bitcoin Core

Notes on Bitcoin Core architecture and development.

1.10.1 Debugging

debug wiki

LogPrintf("")
cat debug.log | grep @@@

lldb src/bitcoind

unit tests in src/test/ using BOOST lib test framework.

Run just one test file: src/test/testbitcoin –loglevel=all –runtest=getargtests
Run just one test: src/test/testbitcoin –loglevel=all –runtest=*/theonetest

Logging from unit tests…
BOOSTTESTMESSAGE("@@@");

functional tests in test/functional using python
–loglevel=debug
self.log.debug("bar")

Use –tracerpc to see the log outputs from the RPCs of the different nodes running in the functional test in std::out.

on tests
on unit tests
on functional tests

core review tools

1.10.2 Architecture

1.10.3 Bitcoin PR Review

  1. #17487

    pr
    UTXO cache is responsible for maintaining a view of the spendable coins based upon the txns in blocks. Major bottleneck during block validation.

    UTXO set currently is more than 8 GB.

    For this reason, UTXO cache is across several layes: on-disk and in-memory.

    The -dbcache param controls how much memory we allocate to the in-memory portion. As we validate blocks, we pull coins and we look up from disk into mem until we run out of memory.

    We completely empty the UTXO cache by writing to disk by calling CCoinsViewCache::Flush()

    We periodically flush the coins cache to avoid having to replay blocks if we shutdown improperly.

    Once we flush the cache, we are forced to read from and write to disk for all UTXO operations, which can be notably slower depending on the underlying disk. For this reason, separating the emptying of the cache from the writing to disk might allow us to ensure durability without losing the performance benefits of maintaining the cache.

    Another case that requires writing to disk without necessarily emptying the cache can be found in the assumeutxo project. When loading a UTXO set from a serialized snapshot, it’s preferable to write out the newly constructed chainstate immediately after load to avoid having to reload the snapshot once again after a bad shutdown.

    "the main benefit of this cache is to reduce the number of unnecessary writes, i.e. when a coin is created and then destroyed we save 2 disk writes. But when we flush, even without deleting the coins from RAM, we expect 1 write if the coin is spent before the tip, otherwise no write." - sjors

    "For some reason (why?) you need to flush at the end of loading the snapshot, which normally means no coins are in RAM. This PR changes that last flush to keep stuff around."

  2. #17428

    pr

1.10.4 Wallet

wallet dev presentation by John Newbery
CPubKey - a public key, used to verify signatures. A point on the secp256k1 curve.
CKey - an encapsulated private key. Used to sign data.
CKeyID - a key identifier, which is the RIPEMD160(SHA256(pubkey))
CTxDestination - a txout script template with a specific destination. Stored as a varint variable

  • CNoDestination: no destination set
  • CKeyID: P2PKH
  • CScriptID: P2SH
  • WitnessV0ScriptHash
  • WitnessV0KeyHash
  • WitnessUnknown

Wallet component is intialized through the WalletInitInterface.
For builds with wallet, the interface is overrridden in src/wallet/init.cpp

For –disable-wallet, there is DummyWalletInit

initiation interface methods are called during node initialization

During loading… WalletInit::Construct() adds a client interface to the wallet.
Node then tells wallet to load/start/stop/etc through the ChainClient interface in src/interfaces/wallet.cpp
Most methods in that interface call through to functions in src/wallet/load.cpp

Node <> Wallet Interface
Node holds a WAlletImpl interface to call functions on the wallet.
Wallet holds a ChainImpl interface to call functions on the node.
Notifications handler
Node notifies the wallet about new transactions and blocks through the CValidationInterface

Identifying Transactions
When a transaction is added to the mempool or block is "connected", the wallet is notified through CValidationInterface.
SyncTransaction() … calls AddToWalletIfInvolvingMe()
IsMine() : takes the scriptPubKey, interprets it as a Destination type, and then checks whether we have the key(s) to watch/spend.

Generate Keys
Originally a collection of unrelated private keys.
Keypools introduced in 2010 by Satoshi. Cache 100 private keys. When a new key is needed, draw it from keypool and refresh.
HD wallets introduced to Bitcoin Core in 2016. Keypool essentially became an address lock-ahead pool. It is used to implement a 'gap limit'.

Constructing Transactions
sendtoaddress
sendtomany
{create,fund,sign,send}rawtransaction
The address is decoded into a CDestination.
Other parameters can be added for finer control (RBF, fees, etc).
Wallet creates the transaction in CreateTransaction().

Coin Selection
By default, coin selection is automatic.
Logic starts in CWallet:SelectCoins().
By preference, we choose coins with more confirmations.
Manual coin selection (coin control) is possible in CCoinControl.

Signing Inputs
Last step in CreateTransaction()
CWallet is an implementation of SigningProvider interface.
Signing logic for the SigningProvider is all in src/script/sign.cpp.

Sending Transactions
Wallet saves and broadcats the wallet in CommitTransaction()
submitToMemoryPool(), relayTransaction()

1.11 CheckTemplateVerify Workshop

1.12 History

Bitcoin history and misc. trivia items.
history of segwit activation

1.13 Other

Notes that do not fit neatly in the other categories.

Merkelized Abstract Syntax Trees are a general concept: when bitcoin developers talk about it, they’re talking about reworking bitcoin scripts into a series of “OR” branches, and instead of the output committing to the whole script, you commit to the head of the tree.  To spend it, you only need to provide the branch of the script you’re using, and the hashes of the other branches. This can improve privacy, and also shrink the total size of large scripts, particularly if there’s a short, common case, and a long, complex rare case. Note that each key is 33 bytes and each signature about 72 bytes, and each merkle branch only 32 bytes.

Sidechains are based on cross-chain consensus validation through SPV and reorganization proofs (an idea that dates back to my P2PTradeX protocol), while drivechains are based on miners being consensus proxies.

The idea behind JoinMarket is to help create a special kind of bitcoin transaction called a CoinJoin transaction. It's aim is to improve the confidentiality and privacy of bitcoin transactions, as well as improve the capacity of the blockchain therefore reduce costs. The concept has enormous potential, but had not seen much usage despite the multiple projects that implement it. This is probably because the incentive structure was not right.
A CoinJoin transaction requires other people to take part. The right resources (coins) have to be in the right place, at the right time, in the right quantity. This isn't a software or tech problem, its an economic problem. JoinMarket works by creating a new kind of market that would allocate these resources in the best way.

Merged mining is the act of using work done on another block chain (the Parent) on one or more Auxiliary block chains and to accept it as valid on its own chain, using Auxiliary Proof-of-Work (AuxPoW), which is the relationship between two block chains for one to trust the other's work as their own. The Parent block chain does not need to be aware of the AuxPoW logic as blocks submitted to it are still valid blocks. 

1.13.1 Future directions of bitcoin

transcript
Schnorr Signature Scheme

  • Has security proof, EDCSA does not.
  • Has linear property, sum of sigs is sum of keys.

SIGHASHNOINPUT - sign scripts, not txid

Taproot
basic idea-> tweak pubkey Q = P+H(P,S)G
Q in output
key spend sign(Q)
script spend: P,S, inputs

Graftroot
if a key exists to represent everyone
use delegation instead of merkle tree
inherently interactive key setup

1.13.2 Utreexo

1.13.3 Graftroof

The idea of graftroot is that in every contract there is a superset of people that can spend the money.
In graftroot, if all the participants agree, then they can just spend. So they can do pubkey aggregation on P

Taproot: P = c + H(c || script) G

Graftroot: sigp(script)

graftroot vs taproot

1.13.4 AssumeUTXO

You get a serialized UTXO set snapshot obtained by a peer. This all hinges on a content-based hash of the UTXO set. The peer gets headers chain, ensures base of snapshot in chain, load snapshot. They want to verify the base of the snapshot or the blockhash is in the header chain. We load the snapshot which deserializes a bunch of coins and loads it into memory. Then we fake a blockchain; we have a chainstate but no blocks on disk, so it's almost like a big pruned chain. We then validate that the hash of the UTXO set matches what we expected through some hardcoded assumeutxo. This is a compiled parameter value, it can't be specified at runtime by the user which is very important. At that point, we sync the tip and that will be a similar delta to what assumevalid would be now, maybe more frequent because that would be nice. Crucially, we start background verification using a separate chainstate where we do regular initial block download, bnackfill that up to the base of the snapshot, and we compare that to the hash of the start of the snapshot and we verify that.
talk on assumeutxo

bitcoin-dev email

The initializing node syncs the headers chain from the network, then obtains and loads one of these UTXO snapshots (i.e. a serialized version of the UTXO set bundled with the block header indicating its "base" and some other metadata).

hardcoded hashs exist in software ..hash(utxoset). similar to assumevalid.

snapshots can obtained in same manner as block download. Doesn't matter about source cuz of content hash.

1.13.5 CoinWitness

Applications of ZK Snarks… ". Instead of embedding the rules that govern an output inside the blockchain, you'd instead embed a proof that the rules were followed. Instead of everyone checking that a transaction was permitted to be spent, they'd instead check that you checked." - Maxwell

coin witness

"You write down a small program which verifies the faithfulness of one of these transcripts for your chosen verifiable off-chain system. The program requires that the last transaction in the transcript is special in that it pays to a Bitcoin scrippubkey/p2sh. The same address must also be provided as a public input to the program. We call this program a "witness" because it will witness the transcript and accept if and only if the transcript is valid.

You then use the SCIP proof system to convert the program into a verifying key. When someone wants to create a Bitcoin in an off-chain system, they pay that coin to the hash of that verifying key. People then transact in the off-chain system as they wish. To be confident that the system works faithfully they could repeat the computationally-expensive verifying key generation process to confirm that it corresponds to the transaction rules they are expecting.

When a user of one of these coins wants to exit the system (to compact its history, to move to another system, to spend plain Bitcoins, or for any other reason), they form a final transaction paying to a Bitcoin address, and run the witness on their transcript under SCIP and produce a proof. They create a Bitcoin transaction redeeming the coin providing the proof in their script (but not the transcript, thats kept private), and the Bitcoin network validates the proof and the transaction output. The public learns nothing about the intermediate transactions, improving fungibility, but unlike other ideas which improve fungibility this idea has the potential to both improve Bitcoin's scalability and securely integrate new and innovative alternative transaction methods and expand Bitcoin's zero-trust nature to more types of transactions."

1.13.6 Covenants

A covenant in its most general sense and historical sense, is a solemn promise to engage in or refrain from a specified action.
maxwell on covenants

scaling bitcoin
"Covenants can be recursively enforced down the chain for as long as you need to reinforce them. "

"Covenants can be used to break fungibility."
whitepaper

Bitcoin Covenants: Three Ways to Control the Future

1.13.7 Zero Knowledge Contigent Payment

zero knowledge payment
ZKCP

swapping information for value

Footnotes:

Author: Philip Glazman

Created: 2020-11-22 Sun 18:39

Validate