GitLab

Summary

Node stopped minting blocks, fell off the MN list.

Steps to reproduce

Unknown.

Expected behavior

Nodes should run normally, minting blocks as expected.

Problematic behavior

MN fell off the network.

Crown-core environment info

Ubuntu 16.04

Crown-core application info

Crown version v0.14.0.0-Emerald

Relevant logs, dumps and/or screenshots

2020-03-29 03:09:58 ProcessNewBlock : ACCEPTED
2020-03-29 03:10:01 socket recv error Connection reset by peer (104) (78.46.234.248:62236)
2020-03-29 03:10:03 CActiveMasternode::ManageStatus() - Begin
2020-03-29 03:10:03 CActiveMasternode::SendMasternodePing() - Relay Masternode Ping vin = CTxIn(COutPoint(c018367fac6fb417d9c6d684afcb7415c798b2bf2aca1075d97676f857b32677, 1), scriptSig=)
2020-03-29 03:10:28 CreateCoinStake: Found valid kernel for mn/sn collateral COutPoint(c018367fac6fb417d9c6d684afcb7415c798b2bf2aca1075d97676f857b32677, 1)
2020-03-29 03:10:28 CreateCoinStake: OutPoint: dcbec51e0f23f5c9d595cd03e105994f9fa9dffbe13d2266407cd1051acdb2eb:1 Modifier=5510e38df933c014364245c1a6ebd1689a59f480cfb4e4245910c03b2b7e9701 timeblockfrom=1585440050 time=1585451436 amount=10000
2020-03-29 03:10:28 Masternode payment to CRWXXL66zgtZQDMtqCX8wsZsM7e7eVnjad2n
2020-03-29 03:10:28 Systemnode payment to CRWJPkaFXoKduwz56Mw7Ss51bt1PzsMEYYAY
2020-03-29 03:10:28 CreateNewBlock(): total size 1000
2020-03-29 03:10:28 CheckProofOfStake : OutPoint: dcbec51e0f23f5c9d595cd03e105994f9fa9dffbe13d2266407cd1051acdb2eb:1 Modifier=5510e38df933c014364245c1a6ebd1689a59f480cfb4e4245910c03b2b7e9701 timeblockfrom=1585440050 time=1585451436 amount=10000
2020-03-29 03:10:28 CBlock(hash=0f4351301724b43fa86094e0f672ecae322741e0aeb61105966b6a2bd6ac8ca3, ver=1442306, hashPrevBlock=1c500da68a69a1627410b11cb0667deabf9bd6c0048b75dc041f8ea06cb056c9, hashMerkleRoot=6abea58d57f8b003387938e34272a3f63d48f88e6fc2705710789cf099295439, nTime=1585451436, nBits=1d072645, nNonce=0, vtx=2)
  CTransaction(hash=7962c2c1dc, ver=1, type=0, vin.size=1, vout.size=3, nLockTime=0, extraPayload.size=0)
    CTxIn(COutPoint(0000000000000000000000000000000000000000000000000000000000000000, 4294967295), coinbase 0374e22a00)
    CTxOut(nValue=0.00000000, scriptPubKey=)
    CTxOut(nValue=1.85000000, scriptPubKey=OP_DUP OP_HASH160 d68833c3e9dc)
    CTxOut(nValue=0.40000000, scriptPubKey=OP_DUP OP_HASH160 467fadba15ab)

  CTransaction(hash=5cd5d47d1b, ver=1, type=0, vin.size=1, vout.size=1, nLockTime=0, extraPayload.size=0)
    CTxIn(COutPoint(0000000000000000000000000000000000000000000000000000000000000000, 4294967295), coinbase c00374e22a00)
    CTxOut(nValue=1.50000000, scriptPubKey=OP_DUP OP_HASH160 8cfa3b9f26bb)

  vMerkleTree:  7962c2c1dc57c3b4c905531b8fe808bef1df05b43d86e6663b62792c413adb62 5cd5d47d1ba3d0b9d80d5a0cf673832a69632019cd7f579bbbd522407a4ebc0f 6abea58d57f8b003387938e34272a3f63d48f88e6fc2705710789cf099295439

2020-03-29 03:10:28 generated 0.00
2020-03-29 03:10:28 CheckProofOfStake : OutPoint: 36216f5a0f34eb5b5ae640e5e333813d515d12567c25d94f2c5dc8e6e3d7bd86:1 Modifier=2b7c6242a9e89c0d4ec589593b13ce2192f863020327f1470fa73c065116a81c timeblockfrom=1585424117 time=1585451444 amount=10000
2020-03-29 03:10:28 CheckProofOfStake : OutPoint: 36216f5a0f34eb5b5ae640e5e333813d515d12567c25d94f2c5dc8e6e3d7bd86:1 Modifier=2b7c6242a9e89c0d4ec589593b13ce2192f863020327f1470fa73c065116a81c timeblockfrom=1585424117 time=1585451444 amount=10000
2020-03-29 03:10:28 UpdateTip: new best=01e2efde8da754876e8ab818ddb170785630b3eb484040364fb2d9d9a310796f  height=2810484  log2_work=86.425817  tx=3554312  date=2020-03-29 03:10:44 progress=1.000000  cache=0.7MiB(2441tx)
2020-03-29 03:10:28 CBudgetManager::NewBlock - incremental sync started
2020-03-29 03:10:28 CBudgetManager::Sync - sent 0 items
...
2020-03-29 03:10:28 CBudgetManager::Sync - sent 0 items
2020-03-29 03:10:28 CBudgetManager::Sync - sent 0 items
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - mapBudgetDrafts cleanup - size: 69
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Cannot find masternode : CTxIn(COutPoint(efdc4f0f7208586b3d4243490a38b79d195ae795b86162890c3681c631e734a7, 0), scriptSig=)
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Older than current blockHeight
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Cannot find masternode : CTxIn(COutPoint(c16fa23196b7cc976027e1b22908838b1c395fb5a03532d9f287583fc20aa3a6, 0), scriptSig=)
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Older than current blockHeight
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Masternode signature is not valid : e9994260ac44fe4e9c09c7a820cb7b5d9126f0f500081be23ab48f1ed40f0c05
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Cannot find masternode : CTxIn(COutPoint(5fc46e707d21623ac78fe195ebf048eb5a6a6c0d8d899bfdb9ba4a6eddc7c307, 0), scriptSig=)
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Older than current blockHeight
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Older than current blockHeight
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Older than current blockHeight
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Older than current blockHeight
...
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Older than current blockHeight
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Cannot find masternode : CTxIn(COutPoint(32da4691adecd1fff7423cc55f1feedf87bac0075eaf5ceffcbeeb2a903be534, 1), scriptSig=)
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Cannot find masternode : CTxIn(COutPoint(e3e449724c21c8c7a749255efc0195a86a9d589072a4f7161b78390e19b5e231, 7), scriptSig=)
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - pbudgetDraft->IsValid - strError: Older than current blockHeight
2020-03-29 03:10:28 CBudgetManager::CheckAndRemove - mapProposals cleanup - size: 108
2020-03-29 03:10:28 keypool keep 504
2020-03-29 03:15:04 CActiveMasternode::ManageStatus() - Begin
2020-03-29 03:15:04 CActiveMasternode::SendMasternodePing() - Relay Masternode Ping vin = CTxIn(COutPoint(c018367fac6fb417d9c6d684afcb7415c798b2bf2aca1075d97676f857b32677, 1), scriptSig=)
...
2020-03-29 04:10:08 CActiveMasternode::ManageStatus() - Begin
2020-03-29 04:10:08 CActiveMasternode::SendMasternodePing() - Relay Masternode Ping vin = CTxIn(COutPoint(c018367fac6fb417d9c6d684afcb7415c798b2bf2aca1075d97676f857b32677, 1), scriptSig=)
2020-03-29 04:15:08 CActiveMasternode::ManageStatus() - Begin
2020-03-29 04:15:08 CActiveMasternode::SendMasternodePing() - Relay Masternode Ping vin = CTxIn(COutPoint(c018367fac6fb417d9c6d684afcb7415c798b2bf2aca1075d97676f857b32677, 1), scriptSig=)
2020-03-29 04:15:29 CSystemnodeSync::Process() - tick 1571736 RequestedSystemnodeAssets 0
2020-03-29 04:15:34 CSystemnodeSync::Process() - tick 1571741 RequestedSystemnodeAssets 1
2020-03-29 04:15:39 CSystemnodeSync::Process() - tick 1571746 RequestedSystemnodeAssets 1
2020-03-29 04:15:44 CSystemnodeSync::Process() - tick 1571751 RequestedSystemnodeAssets 2
2020-03-29 04:15:44 CSystemnodeSync::Process() - lastSystemnodeList 0 (GetTime() - SYSTEMNODE_SYNC_TIMEOUT) 1585455339
2020-03-29 04:15:49 CSystemnodeSync::Process() - tick 1571756 RequestedSystemnodeAssets 2
2020-03-29 04:15:49 CSystemnodeSync::Process() - lastSystemnodeList 0 (GetTime() - SYSTEMNODE_SYNC_TIMEOUT) 1585455344
2020-03-29 04:15:49 CSystemnodeSync::Process() - lastSystemnodeList 0 (GetTime() - SYSTEMNODE_SYNC_TIMEOUT) 1585455344
2020-03-29 04:15:55 CSystemnodeSync::Process() - tick 1571761 RequestedSystemnodeAssets 2
2020-03-29 04:15:55 CSystemnodeSync::Process() - lastSystemnodeList 0 (GetTime() - SYSTEMNODE_SYNC_TIMEOUT) 1585455350
2020-03-29 04:15:55 CSystemnodeSync::Process() - lastSystemnodeList 0 (GetTime() - SYSTEMNODE_SYNC_TIMEOUT) 1585455350
2020-03-29 04:15:55 CSystemnodeSync::Process() - lastSystemnodeList 0 (GetTime() - SYSTEMNODE_SYNC_TIMEOUT) 1585455350
2020-03-29 04:15:55 socket send error Broken pipe (32)

Drop full logs & dumps here: https://nextcloud.crownplatform.com/index.php/s/Q6H8enXNmJsQYCD

Possible fixes

(Any comments on what you think might be responsible for the problem, if you have particular insight.)

/cc @artem