Resource Limiting in the Flow Graph

Resource Limiting in the Flow Graph#

Note

To enable this feature, set the TBB_PREVIEW_FLOW_GRAPH_RESOURCE_LIMITING or TBB_PREVIEW_FLOW_GRAPH_FEATURES macro to 1.

Description#

The Resource Limiting feature enables Flow Graph nodes to safely coordinate access to shared external resources such as database connections, thread-unsafe libraries, etc.

The feature consists of two components:

  • flow::resource_limiter class - a provider that manages a set of resources.

  • flow::resource_limited_node class - a consumer node whose body is invoked only after the node acquires access to a resource from each associated resource_limiter.

A node that must hold resources from several limiters at once can make little progress if the limiters keep granting access to nodes that need only one resource. To avoid such starvation, a resource_limiter arbitrates between competing consumers on a best-effort priority basis, preferring the consumer whose request was made earlier, rather than granting access in an unspecified order. A request that cannot be satisfied retains its position, so a consumer waiting for several resources becomes preferred over later requests as it waits.

API#

Example#

In the example below, two nodes share an exclusive database connection through a resource_limiter managing a single handle:

#define TBB_PREVIEW_FLOW_GRAPH_RESOURCE_LIMITING 1
#include <oneapi/tbb/flow_graph.h>

int main() {
    using namespace tbb::flow;

    resource_limiter<DB_handle*> db_limiter{open_database()};

    graph g;

    using resource_limited_node_type = resource_limited_node<int, std::tuple<int>>;

    // Concurrency is unlimited, but db_limiter ensures exclusive DB access
    resource_limited_node_type db_reader(g, unlimited,
        std::tie(db_limiter),
        [](int id, auto& ports, DB_handle* db) {
            db->read();
            // other actions with the data read from db
            std::get<0>(ports).try_put(id);
        });

    function_node<int, int> processor(g, unlimited, processor_body{});

    resource_limited_node_type db_writer(g, unlimited,
        std::tie(db_limiter),
        [](int id, auto& ports, DB_handle* db) {
            // other actions with the database
            db->write();
            std::get<0>(ports).try_put(id);
        }
    );

    make_edge(output_port<0>(db_reader), processor);
    make_edge(processor, db_writer);
    // Other graph nodes and edges

    for (int id : input_ids) {
        db_reader.try_put(id);
    }

    g.wait_for_all();
}

Because db_limiter holds only one resource handle, the bodies of db_reader and db_writer are never invoked at the same time - even though both nodes allow unlimited concurrency.