Written by: Bob Feldbauer (@bobfeldbauer)
Edited by: Hugh Brown (@saintaardvark)
Since Bitcoin was introduced in 2009, cryptocurrencies have steadily increased in popularity and thousands of digital currencies have been created. The current market capitalization of the top ~500 digital currencies, including Bitcoin and other coins (often called “altcoins”), has reached $5.9 billion1. In 2014, more than $300 million in venture capital was invested in startups focused on digital currencies2. As popularity and use cases for digital currencies grow, gaining a solid understanding of the core concepts and best practices involved is increasingly important for devops professionals.
Before we dive into practical implementation details, management, and risks, let’s review the basic concepts of the blockchain, mining, and wallets.
Mining the Blockchain
The blockchain3 is widely considered to be the core innovation of Bitcoin, and it acts as a permanent public ledger of transactions. Digital currencies use the principles of public-key (or asymmetric) cryptography4, relying on a public and private key to transfer coins from one address to another (which is called a “transaction”). Transactions are grouped together and encoded into blocks, approximately every 10 minutes although it is worth noting that’s just an average. Mining is the process by which blocks are created, and the fundamental principle of digital currency networks is that mining power acts in a distributed, P2P fashion to verify and “vote” on the validity of new blocks being created.
When a new valid block is agreed on by a majority of miners, it is added to the blockchain. Each new block references a hash of the previous block, which is how the blockchain is formed. There are two common types of methods for miners to generate blocks: the most common method is called “proof-of-work”5 (or “POW”), which is used in Bitcoin, Litecoin, etc. The second method, called “proof-of-stake”6 (or “POS”), is used by Peercoin, Blackcoin, and many newer altcoins. New blocks with proof-of-stake are generated by holding coins within a wallet.
For in-depth details about the technical structure of Bitcoin blocks and transactions, see Bitcoin for Mere Mortals and Bitcoins the Hard Way.
Wallets
From Dogecoin to Bitcoin, each digital currency has its own form of wallet software but they tend to provide most of the same core functions. Wallets primarily exist to store your public and private keys, and interact with the blockchain by creating transactions. Most coins have similar basic functionality and you usually interact with them in pretty standard ways – both from a devops perspective and as a user.
Interestingly, many of the innovations from altcoins have been primarily wallet-based, with new features including things like anonymous transactions, traditional point-of-sale functionality, smart contracts, encrypted messaging services, etc.
Another job that wallets can perform, for coins that utilize proof-of-stake, is to generate new blocks and verify transactions on the blockchain based on the coins held in the wallet instead of traditional computationally intensive mining power.
Service Implementation
From a devops perspective, interacting with most wallets is accomplished by using an JSON RPC API that the wallet can be configured to expose. You can do that the hard way by writing your own JSON calls, or use one of the many popular libraries available in most languages.
To get started, here’s a list of API calls and the API reference with examples in various languages. Most altcoins will use the same core API calls, although they may extend the API with their own functionality.
There are numerous libraries out there for interacting with the JSON RPC API in most languages. Some of the more popular ones include bitcoin-pythonlib, bitcoin-ruby, and bitcoin-php.
Unless you’re talking about an extreme volume of transactions, you won’t likely need to build your own custom wallet implementation. Some tuning of wallet configuration settings, hardware, etc. may be required to improve performance though.
Wallet Configuration
Beyond setting the usual options like user, password, and port, if you’re building a service that interacts with the wallet you’ll also want to restrict access to the RPC interface, increase the number of connections, and bump up the thread count.
Coin wallets normally use a simple configuration file for their settings. Here’s an example configuration to get you started, which should work for Bitcoin and the vast majority of altcoins:
rpcuser=YourSecureUser
rpcpassword=AnyRandomSecurePassword
rpcport=4567
server=1
listen=1
gen=0
testnet=0
daemon=1
rpcthreads=100
maxconnections=200
rpcallowip=127.0.0.1
The first three options (rpcuser, rpcpassword, and rpcport) should be obvious enough - they define the user access control for RPC interface. The next five (server, listen, gen, testnet, and daemon) are just to tell the wallet you want to start a daemon, not try to uselessly mine new blocks with your server’s CPU, and operate on the main blockchain for the coin instead of the test network.
Setting rpcthreads and maxconnections is required to improve performance when you’re building services so you don’t get bottlenecked by the default limits. And finally, rpcallowip is set to restrict the wallet’s JSON RPC interface from being accessed by any outside address. Do note that securing your wallets via normal firewall rules, appropriate user permissions, etc. is still important and security will be discussed a bit more later.
*Note for Proof-of-Stake Coins
If you’re building services that interact with a coin that uses proof-of-stake to generate blocks, you’ll want to disable staking to avoid complications. Some proof-of-stake coins’ wallets won’t respect one setting or the other to disable staking, so you’ll want to set both of these options:
reservedbalance=999999999
staking=0
(Here, reservedbalance is a number larger than the number of coins you’ll have in the wallet at any time.)
Efficient Transactions
When sending transactions, you’ll normally need to pay a transaction fee to get miners to include your transaction in a block. To improve performance and reduce the amount you spend on transaction fees, you’ll want to avoid sending a single transaction for each transaction your service requires, if possible. You can typically reduce your transaction fees by at least 60–80% by batching transactions using the sendmany RPC call to send coins to multiple addresses in a single transaction, instead of using the standard sendtoaddress call.
Accounting
Don’t rely on built-in wallet “accounts” for your accounting, although they may initially appear useful. Balances reported by the built-in account system should not be considered reliable7 or accurate, and the “accounts” functionality is being deprecated entirely in new versions of Bitcoin. Track incoming and outgoing transactions by transaction id, make sure to verify the number of confirmations meets your requirements, and handle your own accounting in a database of your choice.
Also, be sure to regularly audit your expected balance against what the wallet reports the balance as in a getinfo RPC call. If you spot a discrepancy, your wallet may be corrupted and you’ll need to work on repairing it.
Repairing a Corrupted Wallet
Like anything else, backups are important. Be sure to make regular backups and test restores regularly, before you have to deal with a corrupted wallet. Hopefully you already have good backups of your wallet’s .dat file (normally just called wallet.dat) that you could restore, but the basic process for wallet repair under most coins looks like this:
Make a note of any pending transactions that you’ve sent but haven’t reached at least 1 confirmation on the blockchain yet. This is accomplished with a “listtransactions” commandline or JSON RPC call and checking the number of confirmations listed for recent transactions.
Stop the wallet, then make a backup of the entire wallet directory.
Run the wallet from the commandline with -rescan to see if your wallet has simply missed some transaction[s]. If that fixes your balance discrepancy, you’re done.
If -rescan didn’t help, run -salvagewallet. If that doesn’t help, you’re stuck trying to recover the private keys and importing them into a new wallet…
To accomplish the export/import, without a previous good backup, you’ll likely end up using pywallet.
Redundancy
You can duplicate your wallet and have it running on multiple servers at the same time. Active-passive and active-active setups are both possible. From an infrastructure perspective, the key point is that each running wallet functions as an independent node on the coin network. Running the same wallet on multiple servers means that the public/private keys are present and transactions can be generated anywhere.
Security
Always keep in mind that your wallet holds your public/private keys, which are all that is needed to spend coins. Proper systems security is beyond the scope of this post, but there are a few cryptocurrency-specific items to be aware of which we’ll discuss.
Encryption
The first layer of protection on a wallet is to use the wallet’s built in encryption with a strong password. When you spend coins, you’ll need to enter the password, which adds at least a reasonable basic layer of protection.
Backups
As with any form of valuable data, you’ll want to keep multiple encrypted, secured backups of your digital currency wallets. If you lose your wallet’s private keys, you’ve lost the ability to spend your coins.
“Hot” and “Cold” wallets
These are just fancy terms for what most of us would define as “online” and “offline”. Hot wallets are where you store frequently accessed funds or a sufficient supply for normal transaction volume if you’re running a service. Having large amounts of coins in a hot wallet is unwise because of the risk of loss should your wallet be compromised.
Cold wallets store your coins offline so they can’t be accessed without physical action, whether that’s in the form of a physical coin with a private key attached, a piece of paper in a safe, an encrypted hardware wallet, a USB drive, or otherwise. See https://en.bitcoin.it/wiki/Cold_storage for more information on cold storage.
For additional security on your cold storage, you can use multi-signature protocols to ensure a single compromised source couldn’t spend your coins. “Multisig”8 transactions work by requiring multiple private keys to sign a transaction for it to be able to be spent.
Conclusion
As Bitcoin and altcoins grow in popularity, DevOps professionals will need to understand the issues involved in developing, managing, and securing digital currencies. Although this article is not a comprehensive guide, hopefully you’ve learned enough to get started. Further reading can be found in the “Additional Resources” section.
No comments:
Post a Comment