Sunday, September 25, 2022
  • The network had a problem with Heimdall, one of its two polygon PoS chain layers
  • The deployment of Polygon on OpenSea was affected, so users could not complete NFT transactions

Ethereum’s speed and cost-oriented layer 2 scaling solution, Polygon, recently suffered a breakdown. The network was down for more than 13 hours, which worried users and wondered why the developers took so long to fix the said bug.

problem

The Polygon team informed users yesterday at 16:20 UTC that the network was expected to go down from about 17:50 UTC due to the need to maintain one of its three consensus layers. It noted that the proof of stake users would be the victims.

Since Polygon had a major bug last time, $24 billion in user funds were at risk, and the Polygon team was keen to reassure the community that all user holdings remain safe.

“The team is currently investigating a problem with the Tendermint implementation used by one of the two layers of Polygon’s PoS chain. All user funds and data remain completely secure, but there is a potential risk that Polygon PoS users will experience sporadic downtime“, write the Polygon developers on Twitter.

cause

The Polygon developers said that the problem was due to a recent update affecting Heimdall, a PoS-verified layer on the network. Heimdall is essentially a bunch of PoS nodes participating in network consensus, and in this case a bug had actually plagued the Heimdall validators, causing them to start using different chain versions.

“We suspect that there may have been a bug in the update that affected the consensus and caused different Heimdall validators to be on different versions of the chain, as a result of which a 2/3 consensus was not reached”, the developers said.

As expected, the network was down at 17:55 UTC, but as time passed and Polygon did not work again, users became unsettled. Words of concern spread throughout the community, and users then said the downtime made them feel more than just a pinch.

Action

Polygon did not bring an update until at least 7 hours after the failure of the network. The update at 1:45 am more informed users about the fact that the developers issued a temporary hotfix, while the team was working on a long-term solution. Even then, the provision of the “fix” took longer. Polygon scan data shows that the network is secured with new validated blocks.