Informe Técnico Nº 01, de 23 de Dezembro de 2022
Este informe detalha o procedimento para atualização de um peer b-Cadastros para as últimas versões homologadas em Dez/2022 e deve ser aplicado apenas em peers instalados antes de 10/12/2022.
O procedimento atualiza o Hyperledger Fabric para a versão 2.4.7, o CouchDB para v3.2.2 e o Nginx para v1.22, com ganhos em segurança, desempenho e funcionalidades. Reestrutura ainda os scripts de inicialização do peer para melhor gerenciamento das atualizações futuras.
Se realizado corretamente, este procedimento acarreta na indisponibilidade do serviço por apenas alguns segundos. O único impacto previsto é a interrupção do acesso às bases do peer b-Cadastros por até 1 min, o que permite que o procedimento seja realizado em janelas curtas de manutenção. Mesmo assim sugerimos a reserva de pelo menos 1 hora, para possibilitar testes posteriores à atualização e, caso necessário, a realização de um procedimento de rollback conforme descrito ao final deste informe.
Atenção
É importante que seja realizada a leitura completa deste procedimento antes de realizá-lo passo a passo.
Qualquer dúvida deve ser encaminhada via solicitação de suporte.
Pré-requisitos
Antes de prosseguir com a atualização é importante garantir que o peer possua um backup realizado nas últimas 24h.
Durante o procedimento é realizado o download de imagens Docker hospedadas em hub.docker.com:443, download de scripts hospedados em s3.i02.estaleiro.serpro.gov.br:443 e envio de telemetria para bcadastros-pushgateway-prod.blockchain.estaleiro.serpro.gov.br:443. Portanto, é pré-requisito que o peer possa realizar o acesso via Internet para esses domínios, pelo menos durante a execução deste procedimento.
O serviço do peer será parado por alguns segundos durante o processo de atualização, portanto é necessário avaliar os impactos dessa indisponibilidade nas integrações que porventura existam com as bases do seu peer.
Todos os comandos a seguir devem ser realizados em um terminal do peer (via SSH, por exemplo). O usuário deve ser um administrador do sistema, com conhecimentos de Linux e permissões de superusuário (sudo).
Verificação prévia do peer
Antes de executar a atualização, verifique se o peer está corretamente em execução com o comando:
O resultado do comando acima deve ser algo semelhante a:
IMAGE STATUS NAMES
nginx:1.19 Up 4 months ago nginx
hyperledger/fabric-peer:2.3.2 Up 4 months ago <nome_do_peer>
couchdb:3.1.1 Up 4 months ago couchdb
hyperledger/fabric-tools:2.3.2 Up 4 months ago cli
Verifique os logs do contêiner do peer com o comando:
As mensagens devem estar no padrão abaixo, indicando que o peer está recebendo os blocos normalmente:
2022-12-09 04:04:22.607 UTC [gossip.privdata] StoreBlock -> INFO 53593 Received block [332966] from buffer channel=chcpf
2022-12-09 04:04:22.618 UTC [committer.txvalidator] Validate -> INFO 53594 [chcpf] Validated block [332966] in 11ms
2022-12-09 04:04:24.009 UTC [kvledger] commit -> INFO 53595 [chcpf] Committed block [332966] with 12 transaction(s) in 1388ms (state_validation=21ms block_and_pvtdata_commit=8ms state_commit=1359ms) commitHash=[9597cefd808154028b3b1c720c4a1dd95e75c943e395683f19013bc70f49ebf6]
2022-12-09 04:04:25.525 UTC [gossip.privdata] StoreBlock -> INFO 53596 Received block [332967] from buffer channel=chcpf
2022-12-09 04:04:25.541 UTC [committer.txvalidator] Validate -> INFO 53597 [chcpf] Validated block [332967] in 15ms
2022-12-09 04:04:26.852 UTC [kvledger] commit -> INFO 53598 [chcpf] Committed block [332967] with 11 transaction(s) in 1309ms (state_validation=17ms block_and_pvtdata_commit=7ms state_commit=1284ms) commitHash=[7bd4cc3abaca14ff159c68582180e4b038f372b39609cb07c86924d6c9cbae11]
2022-12-09 04:04:27.605 UTC [gossip.privdata] StoreBlock -> INFO 53599 Received block [332968] from buffer channel=chcpf
2022-12-09 04:04:27.615 UTC [committer.txvalidator] Validate -> INFO 5359a [chcpf] Validated block [332968] in 9ms
2022-12-09 04:04:28.788 UTC [kvledger] commit -> INFO 5359b [chcpf] Committed block [332968] with 9 transaction(s) in 1169ms (state_validation=14ms block_and_pvtdata_commit=6ms state_commit=1147ms) commitHash=[09b65fd248a86b086d98a07dd1511ab236282c5e4f3b9f459e8c5c46665efd5d]
2022-12-09 04:04:30.369 UTC [gossip.privdata] StoreBlock -> INFO 5359c Received block [332969] from buffer channel=chcpf
2022-12-09 04:04:30.383 UTC [committer.txvalidator] Validate -> INFO 5359d [chcpf] Validated block [332969] in 13ms
2022-12-09 04:04:31.695 UTC [kvledger] commit -> INFO 5359e [chcpf] Committed block [332969] with 12 transaction(s) in 1308ms (state_validation=19ms block_and_pvtdata_commit=7ms state_commit=1280ms) commitHash=[3249405423f0bef1803aa152becf34897e7ae72bebb9dc67390baad4e63da2c0]
2022-12-09 04:04:32.384 UTC [gossip.privdata] StoreBlock -> INFO 5359f Received block [332970] from buffer channel=chcpf
2022-12-09 04:04:32.401 UTC [committer.txvalidator] Validate -> INFO 535a0 [chcpf] Validated block [332970] in 16ms
2022-12-09 04:04:33.372 UTC [kvledger] commit -> INFO 535a1 [chcpf] Committed block [332970] with 8 transaction(s) in 969ms (state_validation=22ms block_and_pvtdata_commit=7ms state_commit=938ms) commitHash=[6c9ba46c9de5135cd14a495f5d60af5757e9c06b9c9a2383f9356fa193f390ac]
Execução do procedimento de atualização
Por meio de um terminal do peer realize o download e execute o script de atualização:
wget https://s3.i02.estaleiro.serpro.gov.br/bcad-prod-publico/atualizacoes/2022-01/update-bcad-2022-01.sh
sudo bash update-bcad-2022-01.sh
O script de atualização realizará o download das novas versões das imagens Docker e atualizará os scripts de gerenciamento do peer automaticamente. Ao final da execução os contêineres do peer serão reiniciados. Todos os passos são informados durante o processo e caso algo saia errado proceda conforme a mensagem emitida na saída do script. Caso tudo corra bem o script emitirá a seguinte mensagem:
Verificação da atualização do peer
Por fim, verifique se os contêineres estão em execução com as novas versões definidas com o comando:
Os status dos contêineres devem ser "Up" e as versões das imagens conforme a seguir:
IMAGE STATUS NAMES
nginx:1.22 Up 4 seconds nginx
hyperledger/fabric-peer:2.4.7 Up 4 seconds <nome_do_peer>
couchdb:3.2.2 Up 4 seconds couchdb
hyperledger/fabric-tools:2.4.7 Up 4 seconds cli
Verifique o log do peer com o comando:
Devem aparecer linhas semelhantes a:
2022-12-09 15:01:06.807 UTC 0001 INFO [nodeCmd] serve -> Starting peer:
Version: 2.4.7
Commit SHA: df9c661
Go version: go1.18.7
OS/Arch: linux/amd64
Chaincode:
Base Docker Label: org.hyperledger.fabric
Docker Namespace: hyperledger
2022-12-09 15:01:06.808 UTC 0002 INFO [peer] getLocalAddress -> Auto-detected peer address: 172.20.0.5:7051
2022-12-09 15:01:06.808 UTC 0003 INFO [peer] getLocalAddress -> Host is 0.0.0.0 , falling back to auto-detected address: 172.20.0.5:7051
2022-12-09 15:01:06.810 UTC 0004 INFO [nodeCmd] initGrpcSemaphores -> concurrency limit for endorser service is 2500
2022-12-09 15:01:06.810 UTC 0005 INFO [nodeCmd] initGrpcSemaphores -> concurrency limit for deliver service is 2500
2022-12-09 15:01:06.810 UTC 0006 INFO [nodeCmd] initGrpcSemaphores -> concurrency limit for gateway service is 500
2022-12-09 15:01:06.810 UTC 0007 INFO [nodeCmd] serve -> Starting peer with TLS enabled
2022-12-09 15:01:06.849 UTC 0008 INFO [certmonitor] trackCertExpiration -> The enrollment certificate will expire on 2032-12-06 12:24:15 +0000 UTC
2022-12-09 15:01:06.849 UTC 0009 INFO [certmonitor] trackCertExpiration -> The server TLS certificate will expire on 2025-09-07 23:37:29 +0000 UTC
2022-12-09 15:01:06.849 UTC 000a INFO [ledgermgmt] NewLedgerMgr -> Initializing LedgerMgr
2022-12-09 15:01:07.121 UTC 000b INFO [ledgermgmt] NewLedgerMgr -> Initialized LedgerMgr
2022-12-09 15:01:07.123 UTC 000c WARN [gossip.gossip] New -> External endpoint is empty, peer will not be accessible outside of its organization
2022-12-09 15:01:07.123 UTC 000d INFO [lifecycle] InitializeLocalChaincodes -> Initialized lifecycle cache with 0 already installed chaincodes
2022-12-09 15:01:07.124 UTC 000e INFO [nodeCmd] computeChaincodeEndpoint -> Entering computeChaincodeEndpoint with peerHostname: 172.20.0.5
2022-12-09 15:01:07.124 UTC 000f INFO [nodeCmd] computeChaincodeEndpoint -> Exit with ccEndpoint: 172.20.0.5:7052
2022-12-09 15:01:07.125 UTC 0010 INFO [sccapi] DeploySysCC -> deploying system chaincode 'lscc'
2022-12-09 15:01:07.126 UTC 0011 INFO [sccapi] DeploySysCC -> deploying system chaincode 'cscc'
2022-12-09 15:01:07.126 UTC 0012 INFO [sccapi] DeploySysCC -> deploying system chaincode 'qscc'
2022-12-09 15:01:07.126 UTC 0013 INFO [sccapi] DeploySysCC -> deploying system chaincode '_lifecycle'
2022-12-09 15:01:07.126 UTC 0014 INFO [nodeCmd] serve -> Deployed system chaincodes
2022-12-09 15:01:07.126 UTC 0015 INFO [peer] Initialize -> Loading chain chcaepf
2022-12-09 15:01:07.126 UTC 0016 INFO [ledgermgmt] OpenLedger -> Opening ledger with id = chcaepf
2022-12-09 15:01:07.189 UTC 0017 INFO [lifecycle] update -> Updating cached definition for chaincode 'bcadastros' on channel 'chcaepf'
2022-12-09 15:01:07.232 UTC 0018 INFO [ledgermgmt] OpenLedger -> Opened ledger with id = chcaepf
2022-12-09 15:01:07.264 UTC 0019 WARN [peer.orderers] Update -> Config defines both orderer org specific endpoints and global endpoints, global endpoints will be ignored channel=chcaepf
2022-12-09 15:01:07.265 UTC 001a INFO [deliveryClient] StartDeliverForChannel -> This peer will retrieve blocks from ordering service (will not disseminate them to other peers in the organization) channel=chcaepf
2022-12-09 15:01:07.265 UTC 001b INFO [peer] Initialize -> Loading chain chcno
2022-12-09 15:01:07.265 UTC 001c INFO [ledgermgmt] OpenLedger -> Opening ledger with id = chcno
2022-12-09 15:01:07.317 UTC 001d INFO [lifecycle] update -> Updating cached definition for chaincode 'bcadastros' on channel 'chcno'
2022-12-09 15:01:07.362 UTC 001e INFO [ledgermgmt] OpenLedger -> Opened ledger with id = chcno
2022-12-09 15:01:07.385 UTC 001f INFO [peer.blocksprovider] DeliverBlocks -> Pulling next blocks from ordering service channel=chcaepf orderer-address=hom-orderer0.bcadastros.serpro.gov.br:443 nextBlock=680
2022-12-09 15:01:07.393 UTC 0020 WARN [peer.orderers] Update -> Config defines both orderer org specific endpoints and global endpoints, global endpoints will be ignored channel=chcno
2022-12-09 15:01:07.396 UTC 0021 INFO [deliveryClient] StartDeliverForChannel -> This peer will retrieve blocks from ordering service (will not disseminate them to other peers in the organization) channel=chcno
2022-12-09 15:01:07.396 UTC 0022 WARN [nodeCmd] serve -> Discovery service must be enabled for embedded gateway
2022-12-09 15:01:07.396 UTC 0023 INFO [nodeCmd] serve -> Starting peer with ID=[peer.bcadastros.orgao.gov.br], network ID=[dev], address=[172.20.0.5:7051]
2022-12-09 15:01:07.396 UTC 0024 INFO [nodeCmd] serve -> Started peer with ID=[peer.bcadastros.orgao.gov.br], network ID=[dev], address=[172.20.0.5:7051]
2022-12-09 15:01:07.396 UTC 0025 INFO [kvledger] LoadPreResetHeight -> Loading prereset height from path [/var/hyperledger/production/ledgersData/chains]
2022-12-09 15:01:07.501 UTC 0026 INFO [peer.blocksprovider] DeliverBlocks -> Pulling next blocks from ordering service channel=chcno orderer-address=hom-orderer2.bcadastros.serpro.gov.br:443 nextBlock=1158
Reversão em caso de falhas (rollback)
Por precaução, o script salva cópias de segurança dos scripts em /etc/hyperledger/fabric/peer/backups, que podem ser restauradas para a pasta /etc/hyperledger/fabric/peer. O próprio script de atualização tenta restaurar esse backup caso algo dê errado e reinicia o peer com as versões antigas das imagens.
Porém, caso isso falhe e a execução do script tenha levado o peer a um estado de mal funcionamento, os comandos abaixo retornam o peer para a versão anterior à atualização. Esses comandos podem ser copiados e colados como um bloco só, pois devem ser executados em sequência:
Atenção!
Possíveis erros nas saídas dos comandos docker-compose devem ser ignorados.
sudo /usr/local/bin/docker-compose -f /etc/hyperledger/fabric/peer/docker-compose-cli.yml down --remove-orphans
sudo /usr/local/bin/docker-compose -f /etc/hyperledger/fabric/peer/docker-compose-peer.yml down --remove-orphans
sudo /usr/local/bin/docker-compose -f /etc/hyperledger/fabric/peer/docker-compose.yml down --remove-orphans
cd /etc/hyperledger/fabric/peer
sudo command cp -f backups/$(ls backups/ | head -n1)/* .
sudo rm -f docker-compose.yml .env
sudo /etc/hyperledger/fabric/peer/start.sh
cd -
Verifique se os contêineres estão em execução:
O resultado do comando acima deve ser algo semelhante ao trecho abaixo. Atente para o número das versões das imagens:
IMAGE STATUS NAMES
nginx:1.19 Up 4 seconds nginx
hyperledger/fabric-peer:2.3.2 Up 4 seconds <nome_do_peer>
couchdb:3.1.1 Up 4 seconds couchdb
hyperledger/fabric-tools:2.3.2 Up 4 seconds cli
Verifique o log de inicialização do peer com o comando:
Devem aparecer linhas semelhantes a:
2022-12-09 15:39:29.537 UTC [nodeCmd] serve -> INFO 001 Starting peer:
Version: 2.3.2
Commit SHA: 0022e8f
Go version: go1.15.7
OS/Arch: linux/amd64
Chaincode:
Base Docker Label: org.hyperledger.fabric
Docker Namespace: hyperledger
2022-12-09 15:39:29.537 UTC [peer] getLocalAddress -> INFO 002 Auto-detected peer address: 172.20.0.4:7051
2022-12-09 15:39:29.537 UTC [peer] getLocalAddress -> INFO 003 Auto-detect flag is set, returning 172.20.0.4:7051
2022-12-09 15:39:29.540 UTC [nodeCmd] initGrpcSemaphores -> INFO 004 concurrency limit for endorser service is 2500
2022-12-09 15:39:29.540 UTC [nodeCmd] initGrpcSemaphores -> INFO 005 concurrency limit for deliver service is 2500
2022-12-09 15:39:29.540 UTC [nodeCmd] serve -> INFO 006 Starting peer with TLS enabled
2022-12-09 15:39:29.564 UTC [certmonitor] trackCertExpiration -> INFO 007 The enrollment certificate will expire on 2032-12-06 12:24:15 +0000 UTC
2022-12-09 15:39:29.564 UTC [certmonitor] trackCertExpiration -> INFO 008 The server TLS certificate will expire on 2025-09-07 23:37:29 +0000 UTC
2022-12-09 15:39:29.565 UTC [ledgermgmt] NewLedgerMgr -> INFO 009 Initializing LedgerMgr
2022-12-09 15:39:29.688 UTC [couchdb] handleRequest -> WARN 00a Attempt 1 of 11 returned error: Get "http://couchdb:5984/": dial tcp 172.20.0.2:5984: connect: connection refused. Retrying couchdb request in 125ms
2022-12-09 15:39:29.814 UTC [couchdb] handleRequest -> WARN 00b Attempt 2 of 11 returned error: Get "http://couchdb:5984/": dial tcp 172.20.0.2:5984: connect: connection refused. Retrying couchdb request in 250ms
2022-12-09 15:39:30.065 UTC [couchdb] handleRequest -> WARN 00c Attempt 3 of 11 returned error: Get "http://couchdb:5984/": dial tcp 172.20.0.2:5984: connect: connection refused. Retrying couchdb request in 500ms
2022-12-09 15:39:30.566 UTC [couchdb] handleRequest -> WARN 00d Attempt 4 of 11 returned error: Get "http://couchdb:5984/": dial tcp 172.20.0.2:5984: connect: connection refused. Retrying couchdb request in 1s
2022-12-09 15:39:31.738 UTC [ledgermgmt] NewLedgerMgr -> INFO 00e Initialized LedgerMgr
2022-12-09 15:39:31.741 UTC [gossip.gossip] New -> WARN 00f External endpoint is empty, peer will not be accessible outside of its organization
2022-12-09 15:39:31.741 UTC [lifecycle] InitializeLocalChaincodes -> INFO 010 Initialized lifecycle cache with 0 already installed chaincodes
2022-12-09 15:39:31.742 UTC [nodeCmd] computeChaincodeEndpoint -> INFO 011 Entering computeChaincodeEndpoint with peerHostname: 172.20.0.4
2022-12-09 15:39:31.742 UTC [nodeCmd] computeChaincodeEndpoint -> INFO 012 Exit with ccEndpoint: peer.bcadastros.orgao.gov.br:7052
2022-12-09 15:39:31.743 UTC [sccapi] DeploySysCC -> INFO 013 deploying system chaincode 'lscc'
2022-12-09 15:39:31.743 UTC [sccapi] DeploySysCC -> INFO 014 deploying system chaincode 'cscc'
2022-12-09 15:39:31.743 UTC [sccapi] DeploySysCC -> INFO 015 deploying system chaincode 'qscc'
2022-12-09 15:39:31.743 UTC [sccapi] DeploySysCC -> INFO 016 deploying system chaincode '_lifecycle'
2022-12-09 15:39:31.743 UTC [nodeCmd] serve -> INFO 017 Deployed system chaincodes
2022-12-09 15:39:31.743 UTC [peer] Initialize -> INFO 018 Loading chain chcaepf
2022-12-09 15:39:31.743 UTC [ledgermgmt] OpenLedger -> INFO 019 Opening ledger with id = chcaepf
2022-12-09 15:39:31.796 UTC [lifecycle] update -> INFO 01a Updating cached definition for chaincode 'bcadastros' on channel 'chcaepf'
2022-12-09 15:39:31.837 UTC [ledgermgmt] OpenLedger -> INFO 01b Opened ledger with id = chcaepf
2022-12-09 15:39:31.867 UTC [peer.orderers] Update -> WARN 01c Config defines both orderer org specific endpoints and global endpoints, global endpoints will be ignored channel=chcaepf
2022-12-09 15:39:31.870 UTC [deliveryClient] StartDeliverForChannel -> INFO 01d This peer will retrieve blocks from ordering service and disseminate to other peers in the organization for channel chcaepf
2022-12-09 15:39:31.870 UTC [peer] Initialize -> INFO 01e Loading chain chcno
2022-12-09 15:39:31.870 UTC [ledgermgmt] OpenLedger -> INFO 01f Opening ledger with id = chcno
2022-12-09 15:39:31.914 UTC [lifecycle] update -> INFO 020 Updating cached definition for chaincode 'bcadastros' on channel 'chcno'
2022-12-09 15:39:31.951 UTC [ledgermgmt] OpenLedger -> INFO 021 Opened ledger with id = chcno
2022-12-09 15:39:31.982 UTC [peer.orderers] Update -> WARN 022 Config defines both orderer org specific endpoints and global endpoints, global endpoints will be ignored channel=chcno
2022-12-09 15:39:31.986 UTC [deliveryClient] StartDeliverForChannel -> INFO 023 This peer will retrieve blocks from ordering service and disseminate to other peers in the organization for channel chcno
2022-12-09 15:39:31.986 UTC [nodeCmd] serve -> INFO 024 Starting peer with ID=[peer.bcadastros.orgao.gov.br], network ID=[dev], address=[172.20.0.4:7051]
2022-12-09 15:39:31.986 UTC [nodeCmd] serve -> INFO 025 Started peer with ID=[peer.bcadastros.orgao.gov.br], network ID=[dev], address=[172.20.0.4:7051]
2022-12-09 15:39:31.986 UTC [kvledger] LoadPreResetHeight -> INFO 026 Loading prereset height from path [/var/hyperledger/production/ledgersData/chains]
Por fim, caso o peer ainda esteja apresentando mal funcionamento, acione o suporte do b-Cadastros.