Ethereum Reorg Monitoring: Event Recovery and Release Decisions

Choose which actions may follow a provisional event

Permit a provisional event to open an investigation. Require stronger evidence before it releases an irreversible benefit.

That is a policy decision, not a subscription setting. A security dashboard can retract a warning after its block leaves the canonical chain. A service that has already delivered an external benefit may have no equivalent undo operation. Define which situation the application is in before choosing its event-processing behavior.

Ethereum reorg monitoring connects observations of changing block history with the application decisions that depend on that history. It includes detecting a replaced block, revising the affected event projection and deciding what may proceed while recovery is incomplete. A live connection establishes none of those outcomes on its own.

The Geth subscription documentation states that notifications cover current events, not past events. It also describes removal notices for logs on an old chain. Those mechanisms help a connected consumer maintain a view. They do not promise to repair every observation missed during a disconnect.

Separate speed from completeness. Receiving the newest head promptly does not show that every intermediate block was processed. Geth documents that a catch-up burst can emit only the last new-head notification. The consumer therefore needs a history-recovery procedure even when the connection appears healthy again.

Our smart contract monitoring guide covers rule selection and response. This guide narrows the operational question to what happens when the block behind a signal changes, or when the application cannot establish which observations it missed.

Put the release condition in a named product requirement. An engineer should be able to distinguish an event being seen, the local projection being reconciled and the business action being authorized. Calling all of them confirmed makes incident review harder precisely when the distinction matters.

Original replay comparison across 27 traces

Method

We executed a finite Python replay exercise using the notification semantics in the opened primary references. It compares application policies, not Ethereum clients. The canonical reference is supplied to the exercise; no live node was asked to discover or verify it.

Reference scenario

Constructed block versions A and B occupy the same height and contain the same transaction and log index. Their symbolic hashes and model payload annotations differ. The expected final projection contains only B.

Complete input domain

Old addition, old removal and replacement addition each have 0, 1 or 2 observed copies. Source order is preserved. The full Cartesian product gives 27 traces, with none sampled out.

Meaning of copies

Missing copies model missed observations; repeated copies model consumer replay. These are constructed inputs, not measured notification probabilities or a claim that Geth retries every message.

Compared policies

An append-only collector ignores removals. A versioned projection upserts by chain, block hash and log index, then deletes the matching occurrence on removal.

Additional controls

Each trace also receives a successful canonical-reference replacement control and an unavailable-query control. Replacement is assumed atomic; unavailable results retain the provisional view and mark it unknown.

Provenance

Primary pages fetched on November 2, 2026. seo/research/ethereum-reorg-monitoring-2026-11-02/measurement.py, measurement.json and trace-summary.csv retain the exercise. These repository artifacts are not public downloads.

Results

The unavailable-query control marks all 27 traces unknown and retains their provisional projections. It does not turn an unavailable response into an empty successful result. This checks the chosen error policy; it does not prove that an actual provider will return a complete canonical answer.

Limits

• No live RPC, consensus process or database transaction executed. The numbers describe the declared application model.
• Only a single replacement scenario and source-ordered observations are included. Deeper forks, reordered workers and multiple events per transaction require additional tests.
• Canonical data and atomic replacement are assumptions in the successful control. Dishonest endpoints, crash recovery and finality verification were not tested.
• These counts establish neither reorg frequency nor production reliability. Extending the input domain can change every outcome count.

Use the mismatching rows to select integration tests. The fact that the better projection still mismatches in 13 cases is a reason to define reconciliation, not a measured failure rate for a deployed indexer.

Keep block identity with every event

Retain the occurrence's block version. A transaction hash identifies a transaction, while a block hash identifies the block containing an observed occurrence. Geth explicitly allows a log subscription to emit logs for the same transaction multiple times during reorganizations.

Occurrence identity

For mined logs, record the chain context, block hash and log index. Keep the transaction hash as related evidence rather than treating it as the entire event key.

Chain continuity

Store block number, block hash and parent hash together. A height alone cannot distinguish two competing blocks at that height.

Projection status

Separate provisional, removed and reconciled observations. Keep an audit record of why the visible application state changed.

Filter identity

Retain the address and topic selection whose history was scanned. Changing the filter changes what a completed scan establishes.

In the exercise, observing the old addition once, missing its removal and observing the replacement once leaves both occurrences in the versioned projection. The keys prevent them from overwriting each other, but the missing removal still leaves obsolete state. Correct identity is necessary for a useful audit trail; it is not a substitute for canonical reconciliation.

Another retained trace observes the old addition once, its removal once and the replacement twice. An append-only collector retains the obsolete occurrence and both replacement copies. The versioned projection ends with the replacement only. This demonstrates idempotent upsert and matching removal under the declared ordering, without claiming that all real delivery orderings were tested.

Preserve a raw observation journal beside the current projection. The journal explains what the consumer saw, including duplicates and removals. The projection describes what the application currently accepts. Deleting all historical observations when repairing a view can erase the evidence needed to explain an earlier alert or release decision.

Application-level deduplication needs its own key as well. A business action may depend on several occurrences or on a separate order identifier. Retrying a reconciliation must not repeat an external fulfillment merely because the block-version record was rebuilt. The replay exercise does not implement that external action, so its evidence stops at the projection.

Write down how competing endpoint observations are handled. A load-balanced connection can expose a different current view after a retry. Record the endpoint and requested block identity so that a later investigation can distinguish changed chain evidence from an application parser or storage problem.

Reconcile the gap before advancing the cursor

A reconnect is the start of recovery. Restore a contiguous, identified history before allowing the scan cursor to represent that range as complete. A cursor that advances merely because a request was attempted hides the exact gap the system needs to repair.

EIP-234 explains the difficult case: a client disconnects, a reorganization occurs and the client reconnects without receiving the old removal notices. It also explains why querying by block number can race with a replacement. An empty result alone may not identify which block the answer described.

1. Load the last accepted block identity and the saved filter definition.
2. Compare the relevant stored hashes and parent continuity with the endpoint's current chain view. On disagreement, locate the shared history before rebuilding the affected projection.
3. Retrieve the required block and log range with explicit identities. Validate successful responses and coverage before marking the range complete.
4. Commit the repaired projection and its cursor under the intended transaction boundary. If recovery cannot establish completeness, retain an unknown state and hold dependent releases.

Define the retained recovery window before an outage. EIP-234 describes keeping a local block history, but the useful depth is an application constraint. If the shared history cannot be established inside the stored evidence, escalate to a broader rebuild or an explicit hold. Do not invent an ancestor merely to resume the cursor. The recovery owner should know which records and endpoint capabilities are needed to extend that search.

A hash-pinned log request makes the requested block explicit. It does not establish that the block is canonical or finalized. EIP-234 permits retrieval for a specified block whether or not it belongs to the current main chain. Treat identity, canonicality and release eligibility as separate checks.

The Ethereum JSON-RPC reference describes blockHash log filtering and excludes combining it with fromBlock or toBlock. Test the actual endpoint's support and error behavior. The exercise did not make a provider-compatibility request.

Keep state reads distinct from log filters. EIP-1898 defines hash-based block parameters and requireCanonical for the state-query methods it names. That option is not an interchangeable key to add to an eth_getLogs filter. A coherent recovery implementation may need both kinds of read, with their respective contracts.

Set release rules around chain evidence

Choose the chain-evidence condition for the action being authorized. A fixed wait for more observed heads and a reported finalized checkpoint answer different questions. Neither should silently replace the condition approved by the product owner.

The Ethereum proof-of-stake overview describes finality through checkpoint voting and an economic security condition. It does not turn a monitoring product into a fixed-duration settlement promise. An application waiting for finality also needs a policy for the period when the required finalized evidence has not advanced.

Latest observation

Suitable for provisional awareness when the application can revise or retract its interpretation. Keep the affected action reversible under the selected policy.

Reconciled projection

History has been repaired to the declared scan target and filter. This is an application completeness claim, not automatically a finality claim.

Release evidence

The chosen chain condition and application acceptance conditions both hold for the relevant occurrence. Record the evidence used at the decision point.

Observation stage

Preserve what arrived without presenting the observation as irrevocable.

Successful recovery

Evaluate the application decision against the identified chain history and the approved release rule.

Unresolved recovery

Keep the exception visible and prevent it from becoming an accidental authorization.

A timeout needs an outcome too. If the required chain evidence stops advancing, a waiting period should lead to the approved escalation or service restriction. It should not silently weaken the release condition. Specify who may approve an exception, what evidence they must retain and which affected actions remain reversible. This is a proposed product policy, not a claim about a universal Ethereum timeout.

Record exceptions explicitly. An operator may be permitted to investigate or restrict service while a signal remains provisional. That does not imply permission to release an irreversible benefit under the same evidence. Escalation authority and release authority can belong to different roles.

This scope is Ethereum execution-log handling with Ethereum chain evidence. Rollup confirmation levels and their relationship to a parent chain need a chain-specific review. Reusing a tag name does not establish identical operational guarantees across networks.

Assign ownership and rehearse reversal

Make recovery a deliverable with an owner. A dashboard showing a new head is weak acceptance evidence if nobody can demonstrate how the application retracts obsolete effects and repairs a gap.

Rehearse partial failure between projection storage and cursor storage. The Python replacement control assumes atomicity, so it provides no crash-safety evidence. A production test must show what happens when the process stops at that boundary and starts again.

Exercise a crash after an external action as a separate case. Rebuilding the projection can replay an occurrence that already triggered fulfillment. The action record must let the application recognize that earlier outcome under its business key. A database cursor and a successful network call are different pieces of evidence. The finite replay exercise does not establish exactly-once fulfillment or any cross-system transaction guarantee.

Retain an ordinary positive control alongside each rejection or hold. A system that permanently withholds every action can appear safe in a narrow test while failing the product's intended operation. Demonstrate that a complete accepted history reaches the permitted outcome as well as that incomplete evidence holds it.

Use the incident response guide to connect a retracted or revised signal with the people who must act. A retrospective correction should state which earlier interpretation changed, what evidence replaced it and which actions remain affected.

The security provider directory and Pharos Production member profile are starting points for defining an external review. Name the off-chain consumer, recovery cursor and dependent business action in the engagement scope. Reviewing only the emitting smart contract does not establish those operational properties.

Acceptance turns on whether the implementation demonstrates its declared policy on retained evidence. Keep the 27 replay traces as an inspectable starting set, add the deployment's missing conditions and preserve unknown outcomes until the responsible owner resolves them.