From 15f545fca2b7a38d82c68211ce9e0530ba97310c Mon Sep 17 00:00:00 2001 From: vyzo Date: Mon, 3 Jan 2022 14:15:20 +0200 Subject: [PATCH] REAMDE --- README.md | 234 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 232 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index e5f1626..56e2f7e 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,232 @@ -# go-libp2p-rcmgr -An implmentation of the libp2p network resource manager interface +# The libp2p Network Resource Manager + +This package contains the canonical implementation of the libp2p +Network Resource Manager interface. + +The implementation is based on the concept of Resource Management +Scopes, whereby resource usage is constrained by a DAG of scopes, +accounting for multiple levels of resource constraints. + +## Design Considerations + +- The Resource Manager must account for basic resource usage at all + levels of the stack, from the internals to the application. +- Basic resources include memory, streams, connections, and file + descriptors. These account for both space and time used by + the stack, as each resource has a direct effect on the system + availability and performance. +- The design must support seamless integration for user applications, + which should reap the benefits of resource management without any + changes. That is, existing applications should be oblivious of the + resource manager and transparently obtain limits which protect it + from resource exhaustion and OOM conditions. +- On the same time, the design must support opt-in resource usage + accounting for applications who want to explicitly utilize the + facilities of the system to inform about and constrain their own + resource usage. +- The design must allow the user to set its own limits, which can be + static (fixed) or dynamic. + + +## Basic Resources + +### Memory + +Perhaps the most fundamental resource is memory, and in particular +buffers used for network operations. The system must provide an +interface for compoenents to reserve memory that accounts for buffers +(and possibly other live objects), which is scoped within the component. +Before a new buffer is allocated, the component should try a memory +reservation, which can fail if the resource limit is exceeded. It is +then up to the component to react to the error condition, depending on +the situation. For example, a muxer failing to grow a buffer in +response to a window change should simply retain the old buffer and +operate at perhaps degraded performance. + +### File Descriptors + +File descriptors are an important resource that uses memory (and +computational time) at the system level. They are also a scarce +resource, as typically (unless the user explicitly intervenes) they +are constrained by the system. Exhaustion of file descriptors may +render the application incapable of operating (e.g. because it is +unable to open a file). + +### Connections + +Connections are a higher level concept endemic to libp2p; in order to +communicate with another peer, a connection must first be +established. Connections are an important resource in libp2p, as they +consume memory, goroutines, and possibly file descriptors. + +We distinguish between inbound and outbound connections, as the former +are initiated by remote peers and consume resources in response to +network events and thus need to be tightly controlled in order to +protect the application from overload or attack. Outbound +connections are typically initiated by the application's volition and +don't need to be controlled as tightly. However, outbound connections +still consume resources and may be initiated in response to network +events because of (potentially faulty) application logic, so they +still need to be constrained. + +### Streams + +Streams are the fundamental object of interaction in libp2p; all +protocol interactions happen through a stream that goes over some +connection. Streams are a fundamental resource in libp2p, as they +consume memory and goroutines at all levels of the stack. + +Streams always belong to a peer, specify a protocol and they may +belong to some service in the system. Hence, this suggests that apart +from global limits, we can constrain stream usage at finer +granularity, at the protocol and service level. + +Once again, we disinguish between inbound and outbound streams. +Inbound streams are initiated by remote peers and consume resources in +response to network events; controlling inbound stream usage is again +paramount for protecting the system from overload or attack. +Outbound streams are normally initiated by the application or some +service in the system in order to effect some protocol +interaction. However, they can also be initiated in response to +network events because of application or service logic, so we still +need to constrain them. + + +## Resource Scopes + +The Resource Manager is based on the concept of resource +scopes. Resource Scopes account for resource usage that is temporally +delimited for the span of the scope. Resource Scopes conceptually +form a DAG, providing us with a mechanism to enforce multiresolution +resource accounting. Downstream resource usage is aggregated at scopes +higher up the graph. + +The following diagram depicts the canonical scope graph: +``` +System + +------------> Transient.............+................+ + | . . + +------------> Service------------- . ----------+ . + | . | . + +-------------> Protocol----------- . ----------+ . + | . | . + +-------------->* Peer \/ | . + +------------> Connection | . + | \/ \/ + +---------------------------> Stream +``` + +### The System Scope + +The system scope is the top level scope that accounts for global +resource usage at all levels of the system. This scope constrains all +other scopes and institutes global hard limits. + +### The Transient Scope + +The transient scope accounts for resources that are in the process of +full establishment. For instance, a new connection prior to the +handshake does not belong to any peer, but it still needs to be +constrained as this opens an avenue for attacks in transient resource +usage. Similarly, a stream that has not negotiated a protocol yet is +constrained by the transient scope. + +### Service Scopes + +The system is typically organized across services, which may be +ambient and provide basic functionality to the system (e.g. identify, +autonat, relay, etc). Alternatively, services may be explicitly +instantiated by the application, and provide core components of its +functionality (e.g. pubsub, the DHT, etc). + +Services consume resources such as memory and may directly own streams +that implement their protocol flow. Services typically have some +stream handler, so they are subject to inbound stream creation and +resource usage in response to network events. As such, the system +explicitly models them allowing for isolated resource usage that can +be tuned by the user. + +### Protocol Scopes + +Protocol Scopes account for resources at the protocol level. They are +an intermediate resource scope which can constrain streams which may +not have a service associated or for resource control within a +service. + +For instance, a service that is not aware of the resource manager and +has not be ported to mark its streams, may still gain limits +transparently without any programmer intervention. Furthermore, the +protocol scope can constrain resource usage for services that +implement multiple protocols for the shake of backwards +compatibility. A tighter limit in some older protocol can protect the +application from resource consumption caused by legacy clients or +potential attacks. + +For a concrete example, consider pubsub with the gossipsub router: the +service also understands the floodsub protocol for backwards +compatibility and support for unsophisticated clients that are lagging +in the implementation effort. By specifying a lower limit for the +floodsub protocol, we can can constrain the service level for legacy +clients using an inefficient protocol. + +### Peer Scopes + +The peer scope accounts for resource usage by an individual peer. This +constrains connections and streams and limits the blast radius of +resource consumption by a single remote peer. + +### Connection Scopes + +The connection scope is delimited to the duration of a connection and +constrains resource usage by a single connection. The scope is a leaf +in the DAG, with a span that begins when a connection is established +and ends when the connection is closed. Its resources are aggregated +to the resource usage of a peer. + +### Stream Scopes + +The stream scope is delimited to the duration of a stream, and +constrains resource usage by a single stream. This scope is also a +leaf in the DAG, with span that begins when a stream is created and +ends when the stream is closed. Its resources are aggregated to the +resource usage of a peer, and constrained by a service and protocol +scope. + +### User Transaction Scopes + +User transaction scopes can be created as a child of any extant +resource scope, and provide the prgrammer with a delimited scope for +easy resource accounting. Transactions may form a tree that is rooted +to some canonical scope in the scope DAG. + +For instance, a programmer may create a transaction scope within a +service that accounts for some control flow delimited resource +usage. Similarly, a programmer may create a transaction scope for some +interaction within a stream, e.g. a Request/Response interaction that +uses a buffer. + +## Limits + +Each resource scope has an associated limit object, which designates +limits for all basic resources. The limit is checked every time some +resource is reserved and provides the system with an opportunity to +constrain resource usage. + +There are separate limits for each class of scope, allowing us for +multiresolution and aggregate resource accounting. As such, we have +limits for the system and transient scopes, default and specific +limits for services, protocols, and peers, and limits for connections +and streams. + +## Implementation Notes + +- The package only exports a constructor for the resource manager and + basic types for defining limits. Internals are not exposed. +- Internally, there is a resources object that is embedded in every scope and + implements resource accounting. +- There is a single implementation of a generic resource scope, that + provides all necessary interface methods. +- There are concrete types for all canonical scopes, embedding a + pointer to a generic resource scope. +- Peer and Protocol scopes, which may be created in response to + network events, are periodically garbage collected.