GitLab

- In wallet and GUI code LOCK cs_main as well as cs_wallet when
  necessary
- In main.cpp SendMessages move the TRY_LOCK(cs_main) up, to encompass the call
  to IsInitialBlockDownload.
- Make ActivateBestChain, AddToBlockIndex, IsInitialBlockDownload,
  InitBlockIndex acquire the cs_main lock

Fixes #3997

All functions that use ChainActive but do not aquire the cs_main
lock themselves, need to be called with the cs_main lock held.

This commit adds assertions to all externally callable functions
that use chainActive or chainMostWork.

This will flag usages when built with -DDEBUG_LOCKORDER.

PrintBlockTree output was broken starting from e010af70.
Everything appears on one line.

PrintWallet() added the newline after a block, but this functionality
was removed and no newline was added.

Seemingly, no one noticed. Add a newline after the block information
to fix this.

Forces the block reward to zero when right shift in GetBlockValue() is
undefined, after 64 reward halvings (block height 13,440,000).

This resolves a case in which a mismatch could be used to bloat up the
mempool by sending transactions that pay enough fee to relay, but not
to be mined, with the default policies.

Use the spent outpoint multimap to figure out which wallet transaction
outputs are unspent, instead of a vfSpent array that is saved
to disk.

Amend to d5f1e727. It turns out that BerkelyDB was including inttypes.h
indirectly, so we cannot fix this with just macros.

Trivial commit: apply the following script to all .cpp and .h files:

    # Middle
    sed -i 's/"PRIx64"/x/g' "$1"
    sed -i 's/"PRIu64"/u/g' "$1"
    sed -i 's/"PRId64"/d/g' "$1"
    # Initial
    sed -i 's/PRIx64"/"x/g' "$1"
    sed -i 's/PRIu64"/"u/g' "$1"
    sed -i 's/PRId64"/"d/g' "$1"
    # Trailing
    sed -i 's/"PRIx64/x"/g' "$1"
    sed -i 's/"PRIu64/u"/g' "$1"
    sed -i 's/"PRId64/d"/g' "$1"

After this commit, `git grep` for PRI.64 should turn up nothing except
the defines in util.h.

As the tinyformat-based formatting system (introduced in b77dfdc9) is
type-safe, no special format characters are needed to specify sizes.

Tinyformat can support (ignore) the C99 prefixes such as "ll" but
chokes on MSVC's inttypes.h defines prefixes such as "I64X". So don't
include inttypes.h and define our own for compatibility.

(an alternative would be to sweep the entire codebase using sed -i to
get rid of the size specifiers but this has less diff impact)

Extend CMerkleTx::GetDepthInMainChain with the concept of
a "conflicted" transaction-- a transaction generated by the wallet
that is not in the main chain or in the mempool, and, therefore,
will likely never be confirmed.

GetDepthInMainChain() now returns -1 for conflicted transactions
(0 for unconfirmed-but-in-the-mempool, and >1 for confirmed).

This makes getbalance, getbalance '*', and listunspent all agree when there are
mutated transactions in the wallet.

Before:
 listunspent: one 49BTC output
 getbalance: 96 BTC (change counted twice)
 getbalance '*': 46 BTC (spends counted twice)

After: all agree, 49 BTC available to spend.

contrib/devtools/fix-copyright-headers.py script to be able to perform this maintenance task with ease during the rest of the year, every year. Modifications to contrib/devtools/README.md to document what fix-copyright-headers.py does.

Keep track of which block is being requested (and to be requested) from
each peer, and limit the number of blocks in-flight per peer. In addition,
detect stalled downloads, and disconnect if they persist for too long.

This means blocks are never requested twice, and should eliminate duplicate
downloads during synchronization.

In case the total number of orphan blocks in memory exceeds a limit
(currently set to 750), a random orphan block (which is not
depended on by another orphan block) is dropped. This means it will
need to be downloaded again, but it won't consume memory until then.

This changes the block processing logic from "try to atomically switch
to a new block" to a continuous "(dis)connect a block, aiming for the
assumed best chain".

This means the smallest atomic operations on the chainstate become
individual block connections or disconnections, instead of entire
reorganizations. It may mean that we try to reorganize to one block,
fail, and rereorganize again to the old block. This is slower, but
doesn't require unbounded RAM.

It also means that a ConnectBlock which fails may be no longer called
from the ProcessBlock which knows which node sent it. To deal with that,
a mapBlockSource is kept, and invalid blocks cause asynchronous "reject"
messages and banning (if necessary).

Previously CreateNewBlock() didn't take into account the fact that
IsFinalTx() without any arguments tests if the transaction is considered
final in the *current* block, when both those functions really needed to
know if the transaction would be final in the *next* block.

Additionally the UI had a similar misunderstanding.

Also adds some basic tests to check that CreateNewBlock() is in fact
mining nLockTime-using transactions correctly.

Thanks to Wladimir J. van der Laan for rebase.

After the tinyformat switch sprintf() family functions support passing
actual std::string objects.

Remove unnecessary c_str calls (236 of them) in logging and formatting.

Switch to tinyformat-based formatting.

Tinyformat is a typesafe drop-in replacement for C99 printf functions:
https://github.com/c42f/tinyformat

This makes it useable for non-global locks such as the wallet and
keystore locks.

Transaction fee is only used by the wallet.
No need for it to be in main.cpp.

There were quite a few places where assert() was used with side effects,
 making operation with NDEBUG non-functional.  This commit fixes all the
 cases I know about, but also adds an  #error on NDEBUG because the code
 is untested without assertions and may still have vulnerabilities if
 used without assert.

Also changes mempool to store CTxMemPoolEntries
to keep track of when they enter/exit the pool.

GetValueIn makes more sense as a CTransaction member.

This dead code can be resurrected from git history if
transaction replacement is ever implemented. Keeping
dead code in the source is a bad idea, because it implies
it was tested and worked at some point, which is not true.