Release Notes¶
Trinity is moving fast. Read up on all the latest improvements.
Trinity 0.1.0-alpha.37 (2021-03-24)¶
Features¶
- During Beam Sync, Trinity now collects account data that is not touched by active blocks. We often call this “state backfill”. Obtaining the full state is the best way to make sure that your node cannot be knocked offline with a Denial-of-Service attack that accesses a bunch of state in a series of blocks. (#854)
- Keep up with the tip of the chain a bit better. (By also listening to
NewBlockHashes
) (#1791) - Make the JSON-RPC API HTTP listen address configurable via
--http-listen-address
(#1795) - Rename
--rpcport
to--http-port
to better align with related flags (#1803) - Only print JSON HTTP API request access logs when Trinity runs with
DEBUG2
logs. (#1804) - Implement backfill of historical blocks during Beam Sync.
The backfill automatically pauses if Beam Sync starts lagging behind and resumes
as soon as it catches up again. It can also be disabled altogether
via the
--disable-backfill
flag. (#1819) - Implement minimal viable
eth_sendRawTransaction
JSON-RPC API (#1855) - Show state backfill progress as info logs (#1872)
- Make
eth_getTransactionCount
work during Beam Sync by requesting state on-demand from peers. (#1873) - Implement pause/resume for historical header backfill. The backfill will pause if Beam Sync starts lagging too far behind and will resume as it catches up again. (#1879)
- Trinity on DappNode: Ensure sync progress is displayed on the dashboard. (#1909)
- Make it configurable which JSON-RPC modules are exposed via HTTP.
Prior to this change, every JSON-RPC module was exposed via HTTP if Trinity was
instructed to run with the HTTP server enabled. With this change, the HTTP
server is enabled via the
--enable-http-apis
flag which takes a string value of either"*"
to expose every module via HTTP or a comma seperated list of module names e.g."eth, net"
. (#1911) - Track block and header db health in metrics. (#2022)
- Report Beam Sync pivot rate to metrics for visualization on grafana dashboard. (#2040)
- Implement a preferred node component to support preferred nodes with discovery disabled. (#2051)
- Introduce NewBlockComponent to be a better p2p participant. By rebroadcasting blocks as they are received from peers, and after they have been imported. (#2058)
- Ensure Clique checkpoints align to epoch length when resolving via etherscan. (#2072)
- Add support for the
eth_chainId
RPC endpoint. (#2076) - Support
admin_removePeer
RPC endpoint. (#2077) - Add support for Berlin, from py-evm v0.4.0-alpha.1 through v0.4.0-alpha.3
(#2126)
- More changes imported from py-evm v0.3.0-alpha.18 (#1818)
- More changes imported from py-evm v0.3.0-alpha.19 (#2004)
Bugfixes¶
- Beam Sync hung indefinitely if your best peer gave you an empty response for state data, and it was your only peer. Now, Beam Sync recovers gracefully and the best peer is asked for state data again, after a brief pause. (#1811)
- Syncing seemed to race ahead and halt and race ahead. Header and block body buffering was reworked to run a bit smoother. (#1820)
- Fix for when a connection is lost while sending
Pong
:p2p.exceptions.PeerConnectionLost: Attempted to send msg with cmd id 3 to ...
(#1821) - Apply UPnP to TCP as well. (#1829)
- Crash-fix, when Beam Sync’s target block is numbered 2 through 7. (#1830)
- Only return unique trie nodes in NodeData responses to peers. Before this fix, Trinity<->Trinity connections where one sends a GetNodeData with duplicate hashes would cause the receiving end to never get the response. (it failed a validation check, and couldn’t be certain the response was even intended for the given request) (#1834)
- Catch and resolve PeerConnectionLost during stream_transport_messages() (#1846)
- Ensure we do not relay transactions with mismatching chain id (#1855)
- Log a user-friendly message on JSON-RPC API errors (#1870)
- Catch exception and present user friendly error when a request for an receipt
to an unknown transaction is send to the
eth_getTransactionReceipt
API. (#1871) - Ensure JSON-RPC API do work with hexadecimal block identifier during Beam Sync (#1873)
- Fix
ConnectionBusy
exception taking whole node down. The fix makes it so that additionalGetBlockHeaders
request (for historical backfill) are allowed to be queued up. (#1879) - Handle error (don’t crash Trinity) when metrics can’t be written to the InfluxDB because of any connection errors. (#1903)
- Make logs a little cleaner by not including stack traces in warnings that are generated by hitting missing RPC endpoints. (#1913)
- Fix a crash when a NEIGHBORS packet contains invalid node records. We also no longer process unsolicited NEIGHBORS packets. (#1995)
- Fix the QueueFull error raised occasionally during sync. (#2027)
- No longer skip preferred peers from list of connection candidates (#2035)
- Use asks library for asyncio service http requests. (#2069)
- Handle error in system metrics collecting for dead processes. (#2083)
- Update EIP1085 validation utils to strictly enforce address formatting. (#2086)
- Update header validation strategy in NewBlock component to support multiple consensus types. (#2089)
- Fix crash when Trinity was launched with a checkpoint too close from tip in Beam Sync mode (#2091)
Performance improvements¶
- Speed up block import during Beam Sync using “Spread Beam”. It asks multiple peers for the trie node data that’s on the critical path for importing the current block. (#1101)
- Speed up Beam Sync a bit by only asking for critical state data from our fastest peer, instead of bundling in the state data needed for pending blocks. (#1947)
- Stop holding the event loop for so long when downloading state for Beam Sync. It was causing performance issues. (#1984)
- Transmit the logs to the logging aggregator process in a separate thread. This helps avoid holding the async event loop for too long. (#1989)
- Move the last of the Beam state download database calls out of the event loop. (#1990)
- Move command decoding out of the event loop. (#1991)
- More cooperative asyncio coroutines in a few places. (#1992)
Improved Documentation¶
- Mention Görli support in docs (homepage + cookbook) (#1780)
- Now logs more information during Beam Sync about the intermediate progress in the middle of a block. (#1916)
- Display the JSON-RPC API URL on the DappNode package view (#2009)
- Extend the DappNode guide with a paragraph that explains how to connect Trinity to DappNode’s default Geth node. (#2012)
- Trinity on DappNode:
- Ensure the package uses the timezone of the host machine. This is to ensure the reported logs use the correct time and match the reported metrics.
- Allow HTTP requests from external addresses. This ensures the DappNode dashboard can show syncing progress and users can perform requests against the node. Note that the DappNode itself is shielded from the external world and only allows access via VPN or its own local W-LAN.
- Don’t map Trinity’s source directory as volume because it is preventing us from having clean updates. (#2013)
- Create
make create-dev-dappnode-image
command to make the creation of temporary dappnode packages easier and reduce the chances of human error. (#2030)
Deprecations and Removals¶
- BREAKING CHANGE: The
--enable-http
flag is no longer supported, use--enable-http-apis
instead. Read the “feature” section for more information. (#1911)
Internal Changes - for Trinity Contributors¶
- Make TaskQueue.complete() async (#1997)
- Upgrade some dependencies: geth to v1.10.0, eth-tester to v0.5.0-beta.3, pytest-xdist to v1.34.0, and coincurve to v15. Also fixed a couple flaky tests and removed the geth bootnodes (because of unresolved issues on Trinity’s end, probably partially due to not having an eth/66 implementation). (#2124)
- Refactored transaction handling into an obaque blob. Previousy, code assumed a fixed transaction shape of a list of bytes, which is no longer accurate (as of EIP-2930). Also, don’t default maxfail=10 locally, and squash a couple test warnings. (#2215)
Trinity 0.1.0-alpha.36 (2020-06-18)¶
Features¶
Implement
admin_peers
JSON-RPC API (#1491)Implement basic metrics tracking. With this change Trinity supports collecting and writing metrics to InfluxDB to be further processed and visualized by Grafana.
Run Trinity with
--enable-metrics --metrics-host "mymachine"
and provide the Influx server and password via the following two environment variables:TRINITY_METRICS_INFLUX_DB_SERVER
TRINITY_METRICS_INFLUX_DB_PW
(#1569)
Monitor the total number of incoming and outgoing packets in Influx/Grafana (#1595)
Monitor the number of peers in Grafana (#1599)
The –ethstats flag is now named –enable-ethstats to improve consistency. (#1604)
Don’t let CompressedLengthError escape and be written to the console. (#1606)
Add partial support for eth/65. Recognize the protocol with its new commands and support it across all internal infrastructure. The built-in “transaction pool” does not yet use the
NewPooledTransactionHashes
command yet and answersGetPooledTransactions
with empty responses. (#1627)Allow for Influxdb port & protocol to be configured via cli arguments. (#1680)
Upgrade to py-evm v0.3.0-alpha.15 – gives access to block state witnesses (#1681)
Support reporting of the current head for connected peers to influx / grafana. (#1699)
Rename
--beam-from-checkpoint
to--sync-from-checkpoint
Ensure sync related flags show up under
--sync-mode
section in--help
Add new
--sync-mode header
to only sync the header chain on the ETH network. This does also support launching from a checkpoint via--sync-from-checkpoint
flag. (#1707)Report trinity process and thread count to grafana. (#1713)
Implement backfilling of gaps in the header chain (introduced through checkpointing) for sync modes
header
andbeam
. This background job can be disable via the--disable-backfill
flag. (#1714)Upgraded py-evm to v0.3.0-alpha.16 – See py-evm v0.3.0-alpha.16 release notes (#1761)
Upgraded py-evm to v0.3.0-alpha.17 – See py-evm v0.3.0-alpha.17 release notes (#1768)
Increase the tip distance for etherscan checkpoint resolving from 50 to 5000 blocks. (#1781)
Bugfixes¶
Fix the propagation of transactions which was broken when the transaction pool was moved into an isolated component.
Also the transaction pool tests now do assertions based on what the remote pool actually receives instead of based on what we appear to be sending out. (#1649)
Squash some deprecation warnings, by upgrading to eth-keys v0.3.3, eth-tester v0.4.0-beta.2, and web3.py v5.9.0 (#1724)
Return checksum address for
to
field ineth_getTransactionReceipt
RPC API (#1749)Fix etherscan checkpoint resolving for Ropsten and Görli. (#1782)
Fix Beam Sync crashing when running under Clique consensus (e.g. Görli) (#1786)
Improved Documentation¶
Deprecations and Removals¶
- Drop support for Python 3.6 making Python 3.7 the minimal supported Python version. (#1675)
Internal Changes - for Trinity Contributors¶
Remove test that scans for unhandled errors when the transaction pool is disabled. Disabling the transaction pool is unlikely to increase the chance of error. (#1611)
Upgrade latest lahja 0.16.0
With the new release the event bus raises exceptions if events or requests are send into the void, meaning when no active subscribers exist. This can be explicitly allowed if desired by setting
require_subscriber
toFalse
when broadcasting events or making requests. (#1653)Improve test coverage of Ethereum Wire Protocol making sure that certain tests ran across all supported protocols from ETH/63 to ETH/65. (#1672)
Run RPC API tests against Istanbul fixtures (#1749)
Trinity 0.1.0-alpha.35 (2020-03-03)¶
Features¶
We now “pivot” Beam Sync if we fall too many blocks behind. When a naive (non-pivoting) Beam Sync falls too far behind the tip of the chain, it can get “stuck” because the trie data is no longer available from peers. (#908)
Add support for networks that run under Clique consensus. Add support for syncing the Görli network via –goerli flag. (#1196)
Support
include_transactions
param ineth_getBlockBy*
APIs (#1465)Improve error message for when the parameters to JSON-RPC calls are invalid
Improve error message for
eth_getWork
,eth_submitWork
andeth_submitHashrate
APIs which Trinity does not support as it generally does not support mining operations.Change
eth_hashrate
andeth_coinbase
to also return errors instead of default values. (#1475)Upgrade to py-evm v0.3.0-alpha.14 (#1478)
Implement
admin_nodeInfo
JSON-RPC endpoint (#1503)Implement the
eth/64
networking protocol according to EIP 2124 (#1543)Add support for Etherscan API keys to continue support for the
--beam-from-checkpoint eth://block/byetherscan/latest
flag now that Etherscan has made API keys mandatory. API keys need to be exposed via theTRINITY_ETHERSCAN_API_KEY
environment variable. (#1565)
Bugfixes¶
- The
p2p.abc.TransportAPI.close
andp2p.abc.MultiplexerAPI.close
methods are now coroutines to allowp2p.abc.TransportAPI.close()
to drain the underlyingStreamWriter
before closing it, ensuring that any pending messages are written over the socket before closing it. (#1417) - Ensure validation errors in components are properly handled during bootstrap. (#1486)
- Fix broken API docs on https://trinity-client.readthedocs.io/en/latest/api/ (#1510)
PeerConnectionLost
was escaping during ahighest_td_peer
call, and crashing whatever called it. The exception is now caught. (#1524)- Catch a variety of uncaught exceptions:
asyncio.TimeoutError
,UnknownAPI
,PeerConnectionLost
, OSError,ValidationError
,CorruptTransport
. Also, prevent AttributeError when printing ap2p.kademlia.Node
that has no address. (#1545) - Catch a
PeerConnectionLost
raised when: disconnecting from a peer for sending a malformed message, and another one when finding the fastest peer. Reject handshake attempts when the IP address is unknown. Catch aCorruptTransport
, raised during multiplex. (#1548)
Trinity 0.1.0-alpha.34 (2019-12-23)¶
Features¶
Full rework of
Component
APIs. CLI validation is now done during application initialization. Component lifecycle is well defined and simpler to implement. (#1328)Add support for eth_getTransactionReceipt JSON-RPC API
See: https://github.com/ethereum/wiki/wiki/JSON-RPC#eth_gettransactionreceipt (#1337)
AsyncioIsolatedComponent
no longer uses the standard libarymultiprocessing
for process isolation, in favor of using the more async friendlyasyncio-run-in-process
(#1363)Underlying
web3
module changed from v4 to v5, including in the console. (#1383)End JSON-RPC responses with n to support dopple. (#1388)
Upgrade Py-EVM and add support for Muir Glacier fork (#1409)
Bugfixes¶
Ensure
eth_getStorageAt
pads results to 32 byteSee Spec or Geth Example (#1403)
Catch exception leaking out of the UpnpService and log it as warning. (#1410)
Trinity 0.1.0-alpha.33 (2019-12-12)¶
Bugfixes¶
- Upgraded py-evm to fix the error:
KeyError: (b'\x03', 'key could not be deleted in JournalDB, because it was missing')
, while importing Istanbul blocks. See other py-evm changes from v0.3.0-alpha.11 (#1376)
Internal Changes - for Trinity Contributors¶
Fix flaky interactive web3 console test
Due to the way components cross connect to each other, not all logs of that startup routine are happening deterministically. We were waiting on a log that would sometimes never show up for all valid reason. This test is now based on a log that is guaranteed to show up unless a serious bug happened. (#1382)
Trinity 0.1.0-alpha.32 (2019-12-09)¶
Bugfixes¶
Upgrade py-evm to v0.3.0-alpha.10, for critical Istanbul bugfix, related to net gas metering. (#1893)
Only allow a single connection per peer, even if an outgoing and incoming handshake are initiated simultaneously. Bonus: squashed UnknownAPI log when talking to a peer that is disconnecting. (#1352)
Quiet down a flood of
BrokenPipeError
that occasionally triggers on shutdown during Beam Sync. (#1355)Ensure ThreadPoolExecutor in beam importer is set up with contextmanager
Without contextmanager one needs to manually call shutdown on the executor which we weren’t doing either. This change may fix some warnings during shutdown of the client. (#1371)
Performance improvements¶
- Speed up the TxPool shutdown a bit: it had hanging tasks that we were waiting 5s to force-close. (#1356)
Improved Documentation¶
Add Matomo Tracking to Docs site.
Matomo is an Open Source web analytics platform that allows us to get better insights and optimize for our audience without the negative consequences of other compareable platforms.
Read more: https://matomo.org/why-matomo/ (#1365)
Trinity 0.1.0-alpha.31 (2019-12-04)¶
Features¶
- Upgrade py-evm to v0.3.0-alpha.9, which includes consensus-sensitive Istanbul fix. See the py-evm release notes (#1343)
- Add export and import command that lets one export blocks to a file or import blocks from a file. (#1266)
- Properly handle Ropsten and Goerli when using –beam-from-checkpoint eth://block/byetherscan/latest to resolve a checkpoint. Also, propagate a proper error to the user in case the syntax is used for an unsupported networḱ. (#1269)
- Automatically rotate logfiles across runs (#1294)
- Add support for eth_getTransactionByHash JSON-RPC API (#1329)
- When resuming Beam Sync, prefer to pick up from the canonical tip, if it’s not too far behind. (#1349)
Internal Changes - for Trinity Contributors¶
Trinity 0.1.0-alpha.30 (2019-11-13)¶
Features¶
- Upgrade to Py-EVM
0.3.0a8
adding the planned Istanbul block for mainnet. See all the other changes in the latest py-evm (#1255)
Trinity 0.1.0-alpha.29 (2019-09-30)¶
Features¶
Bugfixes¶
Improved Documentation¶
- Document how to install, run and develop with the Trinity DappNode package. (#1082)
Internal Changes - for Trinity Contributors¶
- Add a
make create-dappnode-image
command to expose Trinity as a DappNode package. The package can be found astrinity.public.dappnode.eth
. (#1082) Plugins
are now calledComponents
. We’ve foundComponents
to be a better term for the provided functionality, especially since it is less loaded with the assumption of being something that optionally extends functionality when in reality Trinity’s core functionality is built out ofComponents
. (#1140)- Add
ConnectionAPI.get_protocol_for_command_type
(#1145) - Add
ConnectionAPI.get_receipt_by_type(receipt_type: Type[ReceiptAPI])
API (#1148) - The
ConnectionAPI
now has a mirrored version ofMultiplexerAPI.has_protocol
viaConnectionAPI.has_protocol
(#1181)
Trinity 0.1.0-alpha.28 (2019-09-12)¶
Features¶
Remove Trinity specific subclass of the
lahja
endpoint in favor of using the coreEndpointAPI
everywhere. The previous functionality from theTrinityEventBusEndpoint
is now handled by a special service designed to manage the endpoint lifecycle. (#672)Allow trinity db-shell to inspect the beacon node (#809)
Expose
NewBlockEvent
on the event bus. (#822)Add
p2p.p2p_proto.P2PProtocol.send_ping
andp2p.p2p_proto.P2PProtocol.send_hello
methods. (#826)Add
p2p.peer.receive_handshake
to encapsulate the logic for handling incoming connections. (#828)The
p2p.p2p_proto.P2PProtocol
class now requires that handshake parameters be passed into thesend_handshake
method. These parameters are now part of thep2p.peer.BasePeerContext
class. (#829)Add a new
p2p.tools.factories.TransportPairFactory
for generating directly connectedp2p.transport.Transport
objects. (#830)Add
p2p.multiplexer.Multiplexer
for combining the commands from different devp2p sub-protocols into a single network write stream, and split the incoming network stream into individually retrievable sub-protocol commands. (#835)Adds
p2p.protocol.get_cmd_offsets
helper function for computing the command id offsets for devp2p protocols (#836)Use the
p2p.multiplexer.Multiplexer
within theBasePeer
to handle the incoming message stream. (#847)Add factories for creating devp2p protocols and commands for testing. (#850)
Beam Sync: parallel execution of blocks. When connected to a peer on a local network, can now keep up with mainnet (assuming a beefy machine). Also added beam stats in the logs. (#855)
Replace
multiprocessing
based database access with a custom implementation that increases database access performance by 1.5-2x (#859)Implement
p2p.handshake
API. This provides a generic interface for performing proper DevP2p handshakes using multiple sub-protocols without needing involvement of theBasePeer
. (#869)Use the new
p2p.handshake
APIs in thep2p.peer.BasePeer
handshake logic. (#887)If Trinity is Beam Sync and a call to eth_getBalance requests data which is not in the local database, Trinity asks for the data over the network. (#894)
Speculative Execution in Beam Sync: split block transactions to run them in parallel, for speedup. (#899)
Allow Beam Sync to start from a trusted checkpoint. Specify a checkpoint via CLI parameter such as:
--beam-from-checkpoint="eth://block/byhash/<hash>?score=<score>"
When given, Beam Sync will use this as a checkpoint to avoid having to download the entire chain of headers first. (#921)
Expose the force-beam-block-number config as a command line parameter. The config is useful for testing to force Beam Sync to activate on a given block number. (#923)
Add
p2p_version
top2p.peer.BasePeerContext
properties and use for handshake. (#931)If eth_getCode is called during Beam Sync but the requested data is not available locally trinity will attempt to fetch the requested data from remote peers. (#944)
Beam Sync: start backfilling data, especially as a way to gather performance data about peers, and improve the performance of Beam Sync importing. (#951)
Add
p2p.service.run_service
which implements a context manager API for running ap2p.service.BaseService
. (#955)Add
p2p.connection.Connection
service which actively manages thep2p.multiplexer.Multiplexer
exposing an API for registering handler callbacks for individuall protocol commands or entire protocols, as well as access to general metadata about the p2p connection. (#956)If eth_getStorageAt is called during Beam Sync but the requested data is not available locally trinity will attempt to fetch the requested data from remote peers. (#957)
p2p.peer.BasePeer
now usesConnectionAPI
for underlying protocol interactions. (#962)Allow Trinity to automatically resolve a checkpoint through the etherscan API using this syntax:
--beam-from-checkpoint="eth://block/byetherscan/latest"
(#963)Fetch missing data from remote peers, if requested over json-rpc during Beam Sync. Requests for data at an old block will fail; remote peers probably don’t have it. (#975)
Expose the
MiningChain
on the db-shell REPL to allow creating blocks on a REPL (#977)Add
ConnectionAPI.get_p2p_receipt
for fetching theHandshakeReceipt
for the basep2p
protocol. (#986)p2p.protocol.Protocol.supports_command
is now aclassmethod
(#987)The
HandlerSubscriptionAPI
now supports a context manager interface, removing/cancelling the subscription when the context exits (#989)Handler functions for
Connection.add_protocol_handler
andConnection.add_command_handler
now expect theConnection
instance as the first argument. (#990)Introduce
p2p.session.Session
which is now used in place of theremote
to identify peers in the peer pool. (#1054)Add
HTTPServer
for JSON-RPC over HTTP APIs. (#1078)Make beam the default sync strategy and remove fast sync. (#1084)
Detect if a checkpoint is too close to the tip and delay sync until we have reached a minimum distance to the tip. (#1107)
Bugfixes¶
Proper cancellation of subtasks upon cancellation of
p2p.service.BaseService
(#809)The recently introduced fix that ensures we do not run multiple concurrent handshakes to the same peer accidentially introduced a (rarely exposed) memory leak. This fix introduces a
ResourceLock
and refactores the code to use it to also fix the previously introduced memory leak. (#811)Fix issue where test state was leaking between tests in
tests/p2p/test_discovery.py
(#839)Beam Sync: Serve node data requests in parallel, instead of series (#857)
Fix for
DEBUG2
logs always being shown irrespective of log level. (#860)Beam Sync stats: Count the extra single node that is sometimes required when downloading the nodes needed to look up an account or storage. (Usually because of a trie reorg) (#877)
Fixes issue with Trinity not shutting down when issues a
CTRL+C
. (#878)Fix
__str__
implementation ofBaseProxyPeer
to properly represent thep2p.kademlia.Node
URI. (#881)Add missing field from to the response of RPC calls eth_getTransactionByBlockHashAndIndex and eth_getTransactionByBlockNumberAndIndex. (#889)
Ensure
--profile
parameter takes profiles of every process (#891)Handle escaping
PeerConnectionLost
exception fromMultiplexer
inBasePeer
(#895)Fix JSON-RPC call eth_getBalance(address, block_number) to return balance at the requested block_number. Earlier it would always return balance at block(0). (#900)
Fix a MissingTrieNode exception when the first imported block has an uncle (#909)
Handles
MalformedMessage
andTimeoutError
exceptions that can occur while multiplexing the devp2p connection (#916)Fix type hints so that
max_headers
is recognized as keyword argument toget_block_headers
. (#921)BootManager
now uses theBasePeer.loop
as well as their cancel token. (#926)Fix a deadlock bug: if you request data from a peer at just the wrong moment, the request would hang forever. Now, it correctly raises an
OperationCancelled
. (#932)ETHHandshakeReceipt
andLESHandshakeReceipt
now properly accept their protocol instances in their constructors. (#934)Pin
lahja==0.14.0
until connection timeout issue is resolved. (#936)Beam Sync: catch the TimeoutError that was escaping, and retry (#939)
Ensure the
BasePeer
negotiates the proper base protocol. (#942)Capture
PeerConnectionLost
in more places, especially sync. (#943)Beam Sync: Sometimes we would get stuck using a bad peer for node retrieval, fixed. Sometimes we would stop asking for predicted trie nodes when we don’t have any immediate nodes to ask for, fixed. (#958)
Fix
p2p.tools.factories.MultiplexerPairFactory
negotiation ofp2p
protocol version. (#964)Add missing exception handling inside of
Connection.run
forPeerConnectionLost
exception that bubbles from multiplexer.Connection
is now responsible for callingMultiplexer.close
on shutdown. Detect a closed connection during handshake. (#992)Fix
P2PProtocol.send_disconnect
to accept enum values fromp2p.disconnect.DisconnectReason
(#994)Instead of the
ProcessPoolExecutor
use aThreadPoolExecutor
to normalize expensive messages. This fixes a bug where Trinity would leave idle processes from theProcessPoolExecutor
behind every time it shuts down after a sync.Performance wise, both methods should be roughly compareable and since many task have already been moved to their own managed processes over time, using a
ThreadPoolExecutor
strikes as a simple solution to fix that bug. (#1004)Fix a bug where trying to start Beam Sync from a checkpoint would throw an error due to an uninitialized var if a request to a peer would raise an error while we are trying to resolve a header from it. (#1005)
Fix for
TrioService.run_task
to ensure that when a background task throws an unhandled exception that it causes full service cancellation and that the exception is propagated. (#1040)Fix issue where Trinity does not recognize and disconnect from ETC peers when it is being used as an ETH client (#1050)
Handle
MalformedMessage
rising out of theTransport
in theConnection
. (#1051)Ensure discovery V4 handles invalid command ids gracefully (#1063)
Fix issue where attempts to establish new peer connections would halt shortly after startup due to missing timeout when attempting to dial a peer. (#1069)
An exception while serving peer requests would crash out the peer pool event server. Now it doesn’t crash, but logs a big red error (and catches innocuous exceptions, early on). (#1074)
An occasional warning “ValidationError: Duplicate tasks detected” was crashing the node. It’s recoverable, so log it, but don’t crash. (#1083)
Fix warning on ethstats.net due to incorrectly reported API version number. (#1094)
Fix warning caused by inappropriate call to
cancel_nowait
. (#99999)
Performance improvements¶
- Immediately insert Beam Sync nodes that are “predicted” (soon to be used during parallel execution) This saves a round trip on live execution, when parallel execution already downloaded a node. Also, more aggressively make predictive requests if no urgent requests are waiting in the queue. (#877)
- Previously, we gave up on predicted nodes that were not returned by a peer. Now we retry them, which helps make sure we aren’t missing any nodes at block import time. (#932)
- During Beam Sync previews, be sure to collect the nodes required to generate the new state root, rather than wait until it’s time to import the block. (#933)
Improved Documentation¶
Deprecations and Removals¶
- Remove
p2p._utils.clamp
in favor of the one frometh-utils>=1.5.2
(#832) - Remove unused
token
argument fromp2p.tools.memory_transport.MemoryTransport
constructor (#838) - Remove legacy tests from core application code. (#882)
- Remove the
FakeAsync...
classes from tests in favor of using the real versions for things like chain and database objects (#949)
Trinity 0.1.0-alpha.27 (2019-07-17)¶
Trinity 0.1.0-alpha.26 (2019-07-16)¶
Features¶
- Expose certain peer pool events and move RequestServer into an isolated plugin (#617)
- Run UPnP Service as an isolated plugin (plus #730 fixup) (#636)
- Log the gap time between the imported header and now; don’t blast screen with logs when regular syncing a lot (#646)
- Make logging config more ergonomic, flexible and consistent (#682)
- In-memory
Transport
for use in testing. (#693) - Expose events for Transaction and NewBlockHashes commands on the EventBus (#696)
- Use
trinity db-shell
to connect to a running process and inspect its database (#728) - Pool management upgrades
- Move TransactionPool into its own process
- Create ProxyPeerPool that partially exposes peer pool functionality to any process (#734)
- Move responsibility for receiving handshake into
p2p.transport.Transport
class. (#755) - Trinity can now autocomplete CLI parameters on
<tab>
. Learn how to activate autocomplete in the docs. (#768) - Implement
p2p.trio_service.Service
abstraction usingtrio
as a loose replacement for the existingasyncio
basedp2p.service.BaseService
. (#790) - trinity attach can now accept path to ipc as parameter Learn more docs. (#796)
Bugfixes¶
- Header syncing is now limited in how far ahead of block sync it will go (#704)
- Prevent
KeyError
exception raised atdel self._dependencies[prune_task_id]
during syncing (#731) - Fix a race condition in Trinity’s event bus announcement ceremony (#763)
- Several very uncommon issues during syncing, more likely during Beam Sync (#772)
- Squashed bug that redownloads block bodies and logs this warning:
ValidationError: Cannot finish prereq BlockImportPrereqs.StoreBlockBodies of task
(#780) - When starting Beam Sync, download previous six block bodies, so that uncle validation can succeed. Import needs to verify that new block imports don’t add uncles that were already added. (#803)
Improved Documentation¶
- Setup towncrier to generate release notes from fragment files to ensure a higher standard for release notes. (#754)
- Cover
trinity.protocol
events in API docs (#766) - Cover
TrinityConfig
,Eth1AppConfig
andBeaconAppConfig
in API docs. (#775) - Improve layout of API docs by grouping classmethods, methods and attributes. (#778)
- In the API docs display class methods, static methods and methods as one group “methods”. While we ideally wish to separate these, Sphinx keeps them all as one group which we’ll be following until we find a better option. (#794)
Deprecations and Removals¶
- Plugin removals
- Remove
BaseAsyncStopPlugin
which isn’t needed anymore now that there is no hardwirednetworking
process anymore. - Remove plugin manager scopes which aren’t needed anymore now that there is only
a single
PluginManager
. (#763)
- Remove
- The
get_chain_config
API was moved from theTrinityConfig
to theEth1AppConfig
(#771)
v0.1.0-alpha.25¶
Released 2019-06-05
- Upgraded py-evm to deal with eth-keys v0.3.0 dependency issue – see commit
v0.1.0-alpha.24¶
Released 2019-05-21
- #637: EVM upgrade: py-evm upgraded to v0.2.0-alpha.43, changes copied here, from the py-evm changelog
- #1778: Feature: Raise custom decorated exceptions when a trie node is missing from the database (plus some bonus logging and performance improvements)
- #1732: Bugfix: squashed an occasional “mix hash mismatch” while syncing
- #1716: Performance: only calculate & persist state root at end of block (post-Byzantium)
- #1735:
- Performance: only calculate & persist storage roots at end of block (post-Byzantium)
- Performance: batch all account trie writes to the database once per block
- #1747:
- Maintenance: Lazily generate VM.block on first access. Enables loading the VM when you don’t have its block body.
- Performance: Fewer DB reads when block is never accessed.
- Performance: speedups on
chain.import_block()
:- #1764: Speed up
is_valid_opcode
check, formerly 7% of total import time! (now less than 1%) - #1765: Reduce logging overhead, ~15% speedup
- #1766: Cache transaction sender, ~3% speedup
- #1770: Faster bytecode iteration, ~2.5% speedup
- #1771: Faster opcode lookup in apply_computation, ~1.5% speedup
- #1772: Faster Journal access of latest data, ~6% speedup
- #1773: Faster stack operations, ~9% speedup
- #1776: Faster Journal record & commit checkpoints, ~7% speedup
- #1777: Faster bytecode navigation, ~7% speedup
- #1764: Speed up
- #1751: Maintenance: Add placeholder for Istanbul fork
- #629: Feature: Peers which disconnect from us too quickly are blacklisted for a short period of time.
- #625: Feature: Peer backend system is now sent full list of connected remotes
- #624: Feature: Better logging and tracking of the reason a peer disconnection occured.
- #612: Feature: Make Python 3.7 the environment of the ethereum/trinity docker images
- #596: Feature:
p2p.PeerPool
now sources peer candidates using an extendable backend system. - #519: Feature: Retain disconnect reason on
BasePeer
when we disconnect. - #555: Feature: Peers who timeout too often in the Request/Response API will be disconnected from and blacklisted for 5 minutes.
- #558: Feature: Peers who are disconnected due to a
bad_protocol
are blacklisted for 10 minutes. - #559: Feature: Peers who send invalid responses are disconnected from using
bad_protocol
. - #569: Feature: Log messages with sequences of block numbers now use a concise representation to reduce message size.
- #571: Feature:
BaseService.uptime
property now exposes integer number of seconds since service was started. - #441: Feature: Run with any custom network ID, as long as you specify a genesis file
- #436: Feature: Connect to preferred nodes even when discovery is disabled
- #518: Feature: Create log directory for you, if data dir is empty
- #630: Bugfix: Proper shutdown of the whole trinity process if the network database is corrupt.
- #618: Bugfix: Can actually connect to other trinity peers now (and syncing peers).
- #595: Bugfix: Error handling for corrupt snappy data
- #591: Bugfix: Catch
RuntimeError
in handshake to prevent crashing the entire node - #469: Bugfix: Fix deprecation warnings from
p2p.ecies
module. - #527: Bugfix:
LESPeer
class now raises proper exceptions for mismatched genesis hash or network id. - #531: Bugfix:
p2p.kademlia.Node
class is now pickleable. - #564: Bugfix: Sub-protocol compatibility matching extracted from
p2p.BasePeer
to make it easier to test. - #565: Bugfix:
p2p.Protocol
andp2p.Command
classes no longer use mutable data structures for class-level properties. - #568: Bugfix: Revert to fixed timeout for Request/Response cycle with peer to mitigate incorrect timeouts when networking conditions change.
- #570: Bugfix: Remove local implementations of humanize utils in favor of
eth-utils
library implementations. - #485: Bugfix: Ensure Trinity shuts down if Discovery crashes unexpectedly
- #400: Bugfix: Respect configuration of individual logger (e.g -l p2p.discovery=ERROR)
- #336: Bugfix: Ensure Trinity shuts down if the process pool dies (fatal error)
- #347: Bugfix: Don’t crash during sync pruning when switching peers
- #446: Bugfix(es): Several reliability improvements to regular sync
- #389: Bugfix: Always return contiguous headers from header syncer
- #493: Performance: Establish peer connections concurrently rather than sequentially.
- #528: Performance: Limit number of concurrent attempts to establish new peer connections.
- #536: Performance: Peer connection tracking is now a plugin in the
trinity
codebase. - #389: Performance: When switching sync to a new lead peer, don’t backtrack to importing old headers
- #556: Performance: Upgrade to lahja 0.13.0 which performs less inter-process communication
- #386: Performance: Slightly reduce eventbus traffic that the peer pool causes
- #483: Performance: Speed up normalization of peer messages
- #608: Maintenance: Enable tests for Constantinople and Petersburg
- #623: Maintenance: Optimise for faster test runs
0.1.0-alpha.23¶
Released 2019-02-28
- #337: Feature: Support for ConstantinopleV2 aka Petersburg aka ConstantinopleFix
- #270: Performance: Persist information on peers between runs
- #268: Maintenance: Add more bootnodes, use all the Geth and Parity bootnodes
- #263: Performance: Upgrade to lahja 0.11.0 and get rid of EventBus coordinator process
- #227: Bugfix: Do not accidentially create many processes that sit idle
- #227: Tests: Cover APIs that also hit the database in trinity attach tests
- #155: Feature: Disable syncing entirely with –sync-mode none
- #155: Feature: Allow running –sync-mode full directly
- #155: Feature: Allow plugins to extend –sync-mode with different strategies
- #236: Performance: Quicker pruning of in-memory headers, was a leading asyncio bottleneck
- #236: Bugfix: Several reliability improvements during sync
0.1.0-alpha.20¶
Released December 13, 2018
- #1579: Feature: Full Constantinople support, with all* tests passing
- #1590: Performance: CodeStream speedup
- #1576: Bugfix: require recent enough py-ecc to avoid busted py-ecc release (see #1572)
- #1577: Maintenance: Show state diffs on all state failures (see #1573)
- #1570: Maintenance: Cleanup sporadic unclean shutdown of peer request
- #1580: Maintenance: The logged delta in expected vs actual account balance was backwards
- #1573: Maintenance: Display state diffs on failing tests, for much easier EVM debugging
- #1567: Performance: Reduce event bus traffic by enabling point-to-point communication
- #1569: Bugfix: Increase Kademlia timeouts to work on high-latency networks
- #1530: Maintenance: Rename logging level from
trace
(reserved for EVM tracing) todebug2
- #1553: Maintenance: Dynamically tune peer timeouts with historical latency (also #1583)
- #1560: Bugfix: Constantinople CREATE2 gas usage
- #1559: Feature: Mainnet configuration now defaults to Constantinople rules at 7080000
- #1557: Docs: Clarify that local plugins must be installed with
-e
- #1538: Maintenance: Variety of dependency resolution warning cleanups
- #1549: Maintenance: Separate Plugin space for
trinity
andtrinity-beacon
- #1554: Maintenance: Enable asynchronous iterators that can be cancelled by a service
- #1523: Maintenance: Much faster testing of valid PoW chains
- #1536: Maintenance: Add
trinity-beacon
command as a placeholder for future Beacon Chain - #1500: Performance: Be smarter about validating the bloom filter, to avoid duplicate hashing
- #1537: Maintenance: Use new event bus feature to avoid the old hack for clean shutdown
- #1544: Docs: Quickstart fix – use
trinity attach
instead of console - #1541: Docs: Simplify and de-duplicate readme
- #1533: Bugfix: Light chain data lookups regressed during genesis file feature. Fixed
- #1524: Bugfix: Validate header chain continuity during light sync
- #1528: Maintenance: Computation code reorg and gas logging bugfix
- #1522: Bugfix: Increase the system recursion limit for EVM requirements, but never decrease it
- #1519: Docs: Document why we must spawn instead of fork on linux (spoiler: asyncio)
- #1516: Maintenance: Add test for
trinity attach
- #1299: Feature: Launch via custom genesis file (See EIP proposal)
- #1496: Bugfix: Regular chain sync crash
- The research team has started adding Beacon Chain code to the underlying py-evm repo. It’s all a work in progress, but for those who like to follow along:
0.1.0-alpha.18,19¶
That sound you make when you burp in the middle of a hiccup. Hiccurp?
0.1.0-alpha.17¶
Released November 20, 2018
- #1488: Bugfix: Bugfix for state sync to limit the number of open files.
- #1478: Maintenance: Improve logging messages during fast sync to include performance metrics
- #1476: Bugfix: Ensure that network connections are properly close when a peer doesn’t successfully complete the handshake.
- #1474: Bugfix: EthStats fix for displaying correct uptime metrics
- #1471: Maintenance: Upgrade
mypy
to0.641
- #1469: Maintenance: Add logging to show when fast sync has completed.
- #1467: Bugfix: Don’t add peers which disconnect during the boot process to the peer pool.
- #1465: Bugfix: Proper handling for when
SIGTERM
is sent to the main Trinity process. - #1463: Bugfix: Better handling for bad server responses by EthStats client.
- #1443: Maintenance: Merge the
--nodekey
and--nodekey-path
flags. - #1438: Bugfix: Remove warnings when printing the ASCII Trinity header
- #1437: Maintenance: Update to use f-strings for string formatting
- #1435: Maintenance: Enable Constantinople fork on Ropsten chain
- #1434: Bugfix: Fix incorrect mainnet genesis parameters.
- #1421: Maintenance: Implement
eth_syncing
JSON-RPC endpoint - #1410: Maintenance: Implement EIP1283 for updated logic for
SSTORE
opcode gas costs. - #1395: Bugfix: Fix gas cost calculations for
CREATE2
opcode - #1386: Maintenance: Trinity now prints a message to make it more clear why Trinity was shutdown.
- #1387: Maintenance: Use colorized output for
WARNING
andERROR
level logging messages. - #1378: Bugfix: Fix address generation for
CREATE2
opcode. - #1374: Maintenance: New
ChainTipMonitor
service to keep track of the highest TD chain tip. - #1371: Maintenance: Upgrade
mypy
to0.630
- #1367: Maintenance: Improve logging output to include more contextual information
- #1361: Maintenance: Remove
HeaderRequestingPeer
in favor ofBaseChainPeer
- #1353: Maintenance: Decouple peer message handling from syncing.
- #1351: Bugfix: Unhandled
DecryptionError
- #1348: Maintenance: Add default server URIs for mainnet and ropsten.
- #1347: Maintenance: Improve code organization within
trinity
module - #1343: Bugfix: Rename
Chain.network_id
to beChain.chain_id
- #1342: Maintenance: Internal rename of
ChainConfig
toTrinityConfig
- #1336: Maintenance: Implement plugin for EthStats reporting.
- #1335: Maintenance: Relax some constraints on the ordered task management constructs.
- #1332: Maintenance: Upgrade
pyrlp
to1.0.3
- #1317: Maintenance: Extract peer selection from the header sync.
- #1312: Maintenance: Turn on warnings by default if in a prerelease
0.1.0-alpha.16¶
Released September 27, 2018
- #1332: Bugfix: Comparing rlp objects across processes used to fail sporadically, because of a changing object hash (fixed by upgrading pyrlp to 1.0.3)
- #1326: Maintenance: Squash a stack trace in the logs when a peer sends us an invalid public key during handshake
- #1325: Bugfix: When switching to a new peer to sync headers, it might have started from too far behind the tip, and get stuck
- #1327: Maintenance: Squash some log warnings from trying to make a request to a peer (or receive a response) while it is shutting down
- #1321: Bugfix: Address a couple race condition exceptions when syncing headers from a new peer, and other downstream processing is in progress
- #1316: Maintenance: Reduce size of images in documentation
- #1313: Maintenance: Remove miscellaneous things that are generating python warnings (eg~ using deprecated methods)
- #1279: Reliability: Atomically persist when storing: a block, a chain of headers, or a cluster of trie nodes
- #1304: Maintenance: Refactor AtomicDB to return an explict database instance to write into
- #1296: Maintenance: Require new AtomicDB in chain and header DB layers
- #1295: Maintenance: New AtomicDB interface to enable a batch of atomic writes (all succeed or all fail)
- #1290: Bugfix: more graceful recovery when re-launching sync on a fork
- #1277: Maintenance: add a cancellable
call_later
to all services - #1226: Performance: enable multiple peer requests to a single fast peer when other peers are slow
- #1254: Bugfix: peer selection when two peers have exactly the same throughput
- #1253: Maintenance: prefer f-string formatting in p2p, trinity code
0.1.0-alpha.15¶
- #1249: Misc bugfixes for fast sync reliability.
- #1245: Improved exception messaging for
BaseService
- #1244: Use
time.perf_counter
ortime.monotonic
overtime.time
- #1242: Bugfix: Unhandled
MalformedMessage
. - #1235: Typo cleanup.
- #1236: Documentation cleanup
- #1237: Code cleanup
- #1232: Bugfix: Correctly enforce timeouts on peer requests and add lock mechanism to support concurrency.
- #1229: CI cleanup
- #1228: Merge
KademliaProtocol
andDiscoveryProtocol
- #1225: Expand peer stats tracking
- #1221: Implement Discovery V5 Protocol
- #1219: Re-organize and document fixture filler tools
- #1214: Implement
BaseService.is_operational
. - #1210: Convert sync to use streaming queue instead of batches.
- #1209: Chain Builder tool
- #1205: Bugfix: ExchangeHandler stats crash
- #1204: Consensus bugfix for uncle validation
- #1151: Change to
import_block
to return chain re-organization data. - #1197: Increase wait time for database IPC socket.
- #1194: Unify
ValidationError
to useeth-utils
exception class. - #1190: Improved testing for peer authentication
- #1189: Detect crashed sub-services and exit
- #1179:
LightNode
now usesServer
for incoming peer connections. - #1182: Convert
fix-unclean-shutdown
CLI command to be a plugin
0.1.0-alpha.14¶
- #1081 #1115 #1116: Reduce logging output during state sync.
- #1063 #1035 #1089 #1131 #1132 #1138 #1149 #1159: Implement round trip request/response API.
- #1094 #1124: Make the node processing during state sync more async friendly.
- #1097: Keep track of which peers are missing trie nodes during state sync.
- #1109 #1135: Python 3.7 testing and experimental support.
- #1136 #1120: Module re-organization in preparation of extracting
p2p
andtrinity
modules. - #1137: Peer subscriber API now supports specifying specific msg types to reduce msg queue traffic.
- #1142 #1165: Implement JSON-RPC endpoints for:
eth_estimateGas
,eth_accounts
,eth_call
- #1150 #1176: Better handling of malformed messages from peers.
- #1157: Use shared pool of workers across all services.
- #1158: Support specifying granular logging levels via CLI.
- #1161: Use a tmpfile based LevelDB database for cache during state sync to reduce memory footprint.
- #1166: Latency and performance tracking for peer requests.
- #1173: Better APIs for background task running for
Service
classes. - #1182: Convert
fix-unclean-shutdown
command to be a plugin.
0.1.0-alpha.13¶
- Remove specified
eth-account
dependency in favor of allowingweb3.py
specify the correct version.
0.1.0-alpha.12¶
- #1058 #1044: Add
fix-unclean-shutdown
CLI command for cleaning up after a dirty shutdown of thetrinity
CLI process. - #1041: Bugfix for ensuring CPU count for process pool is always greater than
0
- #1010: Performance tuning during fast sync. Only check POW on a subset of the received headers.
- #996 Experimental new Plugin API: Both the transaction pool and the
console
andattach
commands are now written as plugins. - #898: New experimental transaction pool. Disabled by default. Enable with
--tx-pool
. (warning: has known issues that effect sync performance) - #935: Protection against eclipse attacks.
- #869: Ensure connected peers are on the same side of the DAO fork.
Minor Changes
- #1081: Reduce
DEBUG
log output during state sync. - #1071: Minor fix for how version string is generated for trinity
- #1070: Easier profiling of
ChainSyncer
- #1068: Optimize
evm.db.chain.ChainDB.persist_block
for common case. - #1057: Additional
DEBUG
logging of peer uptime and msg stats. - #1049: New integration test suite for trinity CLI
- #1045 #1051: Bugfix for generation of block numbers for
GetBlockHeaders
requests. - #1011: Workaround for parity bug parity #8038
- #987: Now serving requests from peers during fast sync.
- #971 #909 #650: Benchmarking test suite.
- #968: When launching
console
andattach
commands, check for presence of IPC socket and log informative message if not found. - #934: Decouple the
Discovery
andPeerPool
services. - #913: Add validation of retrieved contract code when operating in
--light
mode. - #908: Bugfix for transitioning from syncing chain data to state data during fast sync.
- #905: Support for multiple UPNP devices.
0.1.0-alpha.11¶
- Bugfix for
PreferredNodePeerPool
to respectmax_peers
0.1.0-alpha.10¶
- More bugfixes to enforce
--max-peers
inPeerPool._connect_to_nodes
0.1.0-alpha.9¶
- Bugfix to enforce
--max-peers
for incoming connections.
0.1.0-alpha.7¶
- Remove
min_peers
concept fromPeerPool
- Add
--max-peers
and enforcement of maximum peer connections maintained by thePeerPool
.
0.1.0-alpha.6¶
- Respond to
GetBlockHeaders
message during fast sync to prevent being disconnected as a useless peer. - Add
--profile
CLI flag to Trinity to enable profiling viacProfile
- Better error messaging with Trinity cannot determine the appropriate location for the data directory.
- Handle
ListDeserializationError
during handshake. - Add
net_version
JSON-RPC endpoint. - Add
web3_clientVersion
JSON-RPC endpoint. - Handle
rlp.DecodingError
during handshake.