TCM API Reference

TCM API Reference#

Protocol#

Clients request for and release permits using the Thread Composability Manager API. Each permit contains info about resources a client can use. Clients can have multiple requests at the same time, grouping them together using unique client IDs that are assigned by the Thread Composability Manager upon connecting to it.

It is expected that clients follow TCM recommendations on the resource usage and do not misbehave.

Connecting to and disconnecting from TCM#

Before asking for a permit every client should register itself with the Thread Composability Manager using the following function:

tcm_result_t tcmConnect(tcm_callback_t callback, tcm_client_id_t* client_id)

Parameter	Type	Description
`callback`	In	Permit renegotiation callback. See Callback Type for more info.
`client_id`	Out	Client ID assigned by the Thread Composability Manager for further relation.

Warning

Function returns unsuccessful status if TCM_ENABLE environment variable is not set to 1.

If the client does not expect to request or release resources anymore, it should close the connection by calling the function:

tcm_result_t tcmDisconnect(tcm_client_id_t client_id)

Parameter	Type	Description
`client_id`	In	Client ID that was previously assigned by `tcmConnect`.

Requesting a permit#

Clients request a permit for resources using:

tcm_result_t tcmRequestPermit(tcm_client_id_t client_id, tcm_permit_request_t request,
                              void* callback_arg, tcm_permit_handle_t* permit_handle,
                              tcm_permit_t* permit)

Parameter	Type	Description
`client_id`	In	Client ID obtained by `tcmConnect`.
`request`	In	Description of the resources being requested. See Permit Requests section for more info.
`callback_arg`	In	The argument to pass into the callback function (set previously using `tcmConnect`) in case of a subsequent permit renegotiation.
`permit_handle`	In/Out	Descriptor of resources permitted by the Thread Composability Manager for use by the client. Assign `nullptr` before passing to this function to request a new permit. Pass a descriptor of an existing permit to re-request it with different parameters.
`permit`	In/Out	The description of resources recommended to the client for use. Allocated/deallocated by a client, filled in by TCM.

The function return value is used to indicate possible execution errors, not the availability of resources. After a successful invocation, the caller should check the permit state and fields to see what resources are recommended for use.

Updating a permit request#

Updating of a permit request is done by using the tcmRequestPermit function.

To indicate that it is an update of an existing permit request rather than a request of a new one, client passes a permit_handle value that was returned by a previous call to tcmRequestPermit.

The parameters of a permit request that can be changed are:

Callback argument (callback_arg parameter of the tcmRequestPermit)
Minimum and maximum software threads (see Permit Requests section)
Permit properties (see Permit properties section)

Reading Permit Data#

To read current permit state and associated data, client calls tcmGetPermitData function.

tcm_result_t tcmGetPermitData(tcm_permit_handle_t permit_handle, tcm_permit_t* permit)

Parameter	Type	Description
`permit_handle`	In	Existing descriptor of resources permitted by the Thread Composability Manager for use by the client.
`permit`	In/Out	The description of the resources given to the client as a response to this request. Allocated/deallocated by the client, filled in by the Thread Composability Manager.

Warning

Due to possible concurrent requests from clients, resulting in redistribution of resources by the Thread Composability Manager, the data received in a permit argument might be already outdated by the time the thread returns from the tcmGetPermitData function. In some situations, the Thread Composability Manager can detect this is happening during the call to tcmGetPermitData, in which case a stale flag of the received permit is set to true (see section about Permit Properties). However, it is the responsibility of the client to synchronize multiple, possibly different, copies of a permit’s data.

Threads of a client#

A TCM client utilizes granted CPU resources by running one or more software threads.

When a thread starts participating in the permit, effectively consuming granted resources, it should register itself by invoking the tcmRegisterThread function.

tcm_result_t tcmRegisterThread(tcm_permit_handle_t permit_handle)

Parameter	Type	Description
`permit_handle`	In	Descriptor of the granted resources the invoking thread is going to consume.

When a thread stops consuming resources of a permit, it invokes the tcmUnregisterThread function.

tcm_result_t tcmUnregisterThread()

This API is meant to be called by every thread that is going to participate in a parallel region, for which the resource permit was obtained.

Idling, Activating and Deactivating a Permit#

When resources are not immediately needed, the client may mark them as idle by calling the tcmIdlePermit function. The idle state indicates that threads do not process payload but still can spend CPU cycles actively looking for work. This allows to re-activate the permit relatively quickly in case the resources become needed again.

tcm_result_t tcmIdlePermit(tcm_permit_handle_t permit_handle)

Parameter	Type	Description
`permit_handle`	In	Descriptor of the resources to mark as idle.

If the usage of resources is not anticipated soon, the client deactivates the permit by calling the tcmDeactivatePermit function.

tcm_result_t tcmDeactivatePermit(tcm_permit_handle_t permit_handle)

Parameter	Type	Description
`permit_handle`	In	Descriptor of the resources to deactivate.

TCM can also deactivate an idle permit and initiate a permit negotiation – particularly, if idle resources are needed to satisfy another request.

Once the resources are needed again, the client can re-activate the permit (either idle or inactive) by using the tcmActivatePermit function.

tcm_result_t tcmActivatePermit(tcm_permit_handle_t permit_handle)

Parameter	Type	Description
`permit_handle`	In/Out	Descriptor of the resources to re-activate.

Re-activating an idle permit is typically expected to succeed; however, the client might not (yet) be aware of TCM concurrently deactivating the permit. Re-activating an inactive permit is not guaranteed to succeed as its resources might be in use by another client. Therefore, the caller should check the permit state and fields to ensure resource usage is allowed.

Releasing a permit#

When the resources are not required anymore, the client releases the permit by calling tcmReleasePermit function.

tcm_result_t tcmReleasePermit(tcm_permit_handle_t permit_handle)

Parameter	Type	Description
`permit_handle`	In	Descriptor of the resources to release back to the Thread Composability Manager.

TCM Data Structures#

Dependency on HWLOC#

TCM uses HWLOC library to parse platform topology and obtain process concurrency, process CPU mask, NUMA node, and core type indices.

To make sure CPU masks, NUMA node and core type indices are interpreted by HWLOC library correctly, a TCM client can either link with a compatible version of HWLOC or write adapters for CPU masks.

Warning

Even compatible versions of HWLOC might have different results when parsing platform topology. Therefore, it is recommended to ensure that a single HWLOC library is used within the process.

CPU Mask Adapter#

Note

While NUMA node and core type indices are logical and thus may not correspond to physical indices provided by an operating system, CPU masks represented using tcm_cpu_mask_t are always filled with physical indices of an operating system, which can be used to bind software threads to hardware CPUs specified in the mask.

tcm_cpu_mask_t is a typedef-declaration of a pointer to hwloc_bitmap_s, which is defined in HWLOC as the following:

struct hwloc_bitmap_s {
    unsigned ulongs_count; /* how many ulong bitmasks are valid, >= 1 */
    unsigned ulongs_allocated; /* how many ulong bitmasks are allocated, >= ulongs_count */
    unsigned long *ulongs;
    int infinite; /* set to 1 if all bits beyond ulongs are set */
};

, where:

ulongs_count is the number of unsigned long elements used for bitmask representation.
ulongs_allocated is the size of allocated elements of an array.
ulongs is an array that holds the mask bits.
infinite is used as a flag to indicate whether bits not represented by ulongs array are set or not.

Note

hwloc_bitmap_s is one of the main data structures that HWLOC uses when it describes platform entities such as NUMA node, core type, or even CPUs that share certain levels of cache in terms of a CPU mask. It is unlikely that its layout changes in backward incompatible way.

Since hwloc_bitmap_s is filled with physical, operating system indices, the conversion between hwloc_bitmap_s and CPU masks used in operating system involves going over the mask bits in a loop and setting corresponding bits in a platform-specific mask representation.

TCM Function Result#

The tcm_result_t enum defines a set of possible values that the TCM API may return.

typedef enum _tcm_result_t {
  TCM_RESULT_SUCCESS,
  TCM_RESULT_ERROR_INVALID_ARGUMENT,
  TCM_RESULT_ERROR_UNKNOWN
} tcm_result_t;

Value	Description
`TCM_RESULT_SUCCESS`	Indicates successful execution of the function.
`TCM_RESULT_ERROR_INVALID_ARGUMENT`	Indicates that one or more function arguments are invalid.
`TCM_RESULT_ERROR_UNKNOWN`	Indicates erroneous situation during the function execution.

Permit State#

The tcm_permit_state_t structure describes various states of a permit that the Thread Composability Manager uses to indicate ownership of resources described by a permit.

enum tcm_permit_states_t {
  TCM_PERMIT_STATE_VOID,
  TCM_PERMIT_STATE_INACTIVE,
  TCM_PERMIT_STATE_PENDING,
  TCM_PERMIT_STATE_IDLE,
  TCM_PERMIT_STATE_ACTIVE
};

typedef uint8_t tcm_permit_state_t;

Value	Description
`TCM_PERMIT_STATE_VOID`	No permit. Neither client owns any resources associated with permit, nor does the Thread Composability Manager know about existence of a corresponding request.
`TCM_PERMIT_STATE_INACTIVE`	Client does not own and therefore should not be using resources related to this permit.
`TCM_PERMIT_STATE_PENDING`	Resources are given to another permit and cannot be re-assigned to this permit immediately, but will be considered as soon as they become available.
`TCM_PERMIT_STATE_IDLE`	Resources are not used for payload processing. However, they can be made so by activation of this or the other permit describing the same resources.
`TCM_PERMIT_STATE_ACTIVE`	Resources are owned by the client, and are used for payload processing.

Permit Properties#

The tcm_permit_flags_t describes the properties of a permit.

typedef struct _tcm_permit_flags_t {
  uint32_t stale : 1;
  uint32_t rigid_concurrency : 1;
  uint32_t request_as_inactive : 1;
} tcm_permit_flags_t;

Value	Description
`stale`	Indicates permit data is not up to date and should not be relied upon.
`rigid_concurrency`	Indicates permit requests whose concurrency cannot be changed once granted and in `TCM_PERMIT_STATE_ACTIVE` state. Useful for the client that cannot adjust the number of active threads during payload processing.
`request_as_inactive`	Indicates that TCM should not try satisfying the request, but rather return valid `permit_handle` that can be used in future API calls.

Callback Type#

The type of a function to pass into tcmConnect. The callback is called each time the permit has been changed due to TCM API invoked for different permits, even if the calls are made by the same client.

The purpose of invoking this callback function is to tell a client that the data of a permit has been changed. A client may call tcmGetPermitData inside a callback function in order to obtain the latest permit data.

typedef tcm_result_t (*tcm_callback_t)(tcm_permit_handle_t permit_handle, void* arg,
                                       tcm_callback_flags_t flags);

Value	Description
`permit_handle`	The unique permit handle, whose data has been changed.
`arg`	The callback argument that was previously passed to the `tcmRequestPermit` function.
`flags`	The reasons of callback invocation.

Callback Invocation Reasons#

The tcm_callbacks_flags_t describes the reasons client callbacks were invoked by the Thread Composability Manager.

typedef struct _tcm_callback_flags_t {
  bool new_concurrency : 1;
  bool new_state : 1;
} tcm_callback_flags_t;

Value	Description
`new_concurrency`	Indicates whether permit’s concurrency has been updated.
`new_state`	Indicates whether permit’s state has been updated.

Permits#

The tcm_permit_t structure represents the permit data that is filled in by the Thread Composability Manager. The client is responsible for allocating and deallocating memory for objects of this type, including the arrays of necessary size.

typedef struct _tcm_permit_t {
  uint32_t* concurrencies;
  tcm_cpu_mask_t* cpu_masks;
  uint32_t size;
  tcm_permit_state_t state;
  tcm_permit_flags_t flags;
} tcm_permit_t;

Field	Description
`concurrencies`	The array of permitted concurrencies.
`cpu_masks`	The array of permitted masks. The array items correspond to respective items of the `concurrencies` array.
`size`	The size of the arrays.
`state`	The state of the permit. See Permit State for details.
`flags`	The flags of the permit. See Permit Properties for details.

Note

cpu_masks is nullptr in the case that the subset of resources were not specified via tcm_cpu_constraints_t during the permit request. In this case, the array of concurrencies contains a single element and size equals to 1.

Permit Constraints#

Constraints describe the subset of CPU resources where the requested number of software threads will execute.

Note

The less constrained a resource request is, the more composable with other requests it is going to be. Therefore, it is better to avoid specifying constraints unless absolutely necessary. In cases where constraints are needed, specify them as loosely as possible so that TCM has more opportunities to balance resources between conflicting permit requests.

The subset of resources can be specified either by using a high-level or a low-level description. For a high-level description, the client sets values for numa_id, core_type_id, and threads_per_core struct fields. For a low-level description, the client specifies the mask. In case both low-level and high-level descriptions are specified, the TCM prefers the low-level description.

Objects of tcm_cpu_constraints_t type are required to be initialized using TCM_PERMIT_REQUEST_CONSTRAINTS_INITIALIZER:

tcm_cpu_constraints_t constraints = TCM_PERMIT_REQUEST_CONSTRAINTS_INITIALIZER;

The numa_id, core_type_id, and threads_per_core can be assigned a non-negative integer, in which case the meaning is the following:

Field	Semantics of Assigning a Non-Negative Integer
`numa_id`, `core_type_id`	Requesting resources from item with the index equal to specified value.
`threads_per_core`	The number of threads to use per core.

Besides natural numbers, these fields can be assigned the following special values:

Value	Description
`tcm_automatic`	The TCM decides on its own and may choose the value automatically based on internal heuristics and current load of the platform.
`tcm_any`	The TCM chooses one specific value based on internal heuristics and current load of the platform.

typedef struct hwloc_bitmap_s* tcm_cpu_mask_t;
typedef /*implementation-defined*/ tcm_numa_node_t;
typedef /*implementation-defined*/ tcm_core_type_t;

const /*implementation-defined*/ tcm_automatic =/*implementation-defined*/;
const /*implementation-defined*/ tcm_any =/*implementation-defined*/;

typedef struct _tcm_cpu_constraints_t {
  int32_t min_concurrency;
  int32_t max_concurrency;
  tcm_cpu_mask_t mask;
  tcm_numa_node_t numa_id;
  tcm_core_type_t core_type_id;
  int32_t threads_per_core;
} tcm_cpu_constraints_t;

Field	Description
`min_concurrency`	Minimum value of concurrency for the described hardware subset.
`max_concurrency`	Maximum value of concurrency for the described hardware subset.
`mask`	The low-level mask of the subset of CPU resources. Can be filled with physical indices of an operating system. See CPU Mask Adapter for details.
`numa_id`	High-level mask description. The logical index of the NUMA node to restrict the search for resources within.
`core_type_id`	High-level mask description. The logical index of the core type to restrict the search for resources within.
`threads_per_core`	High-level mask description. The number of threads per core to consider while searching for resources.

Note

To avoid issues with interpretation of logical indices used to enumerate NUMA nodes and core types, the specified values should correspond to the logical indices used by the HWLOC library with which the Thread Composability Manager is linked. See Dependency on HWLOC for more details.

Permit Requests#

The tcm_permit_request_t structure is the data structure that allows describing resources to be requested from the Thread Composability Manager.

typedef struct _tcm_permit_request_t {
  int32_t min_sw_threads;
  int32_t max_sw_threads;
  tcm_cpu_constraints_t* cpu_constraints;
  uint32_t constraints_size;
  tcm_permit_flags_t flags;
} tcm_permit_request_t;

Field	Description
`min_sw_threads`	The minimum number of software threads to satisfy. Permit requests whose minimum number of software threads cannot be satisfied right away get `TCM_PERMIT_STATE_PENDING` state.
`max_sw_threads`	The maximum number of software threads desired.
`cpu_constraints`	The array of hardware constraints, where the Thread Composability Manager should look for available resources. `nullptr` means no constraints are set. See Permit Constraints for details.
`constraints_size`	The size of the `cpu_constraints` array.
`flags`	The properties of the request. See Permit Properties for details.

Objects of tcm_permit_request_t type are required to be initialized using TCM_PERMIT_REQUEST_INITIALIZER:

tcm_permit_request_t request = TCM_PERMIT_REQUEST_INITIALIZER;

Note

The specified values for min_sw_threads and max_sw_threads in the tcm_permit_request_t should be compatible with the min_concurrency and max_concurrency values in the tcm_cpu_constraints_t array if the latter is specified. Otherwise, the behaviour is undefined.

The compatibility rule:

The sum of minimum concurrencies specified in the constraints array should be less or equal to the min_sw_threads specified in the request.
The value of min_sw_threads should be less or equal to max_sw_threads.
The value of max_sw_threads should be less or equal to the sum of maximum concurrencies specified in the constraints array.

Or using inequality notation, the compatibility rule can be written as the following:

\[\sum_{i=1}^N m_i \leq m \leq M \leq \sum_{i=1}^N M_i\]

where:

\(m_i\) is the min_concurrency values from the cpu_constraints array
\(M_i\) is the max_concurrency values from the cpu_constraints array
\(N\) is the value of constraints_size field
\(m\) is the value of min_sw_threads field
\(M\) is the value of max_sw_threads field

TCM API Reference

Contents

TCM API Reference#

Protocol#

Connecting to and disconnecting from TCM#

Requesting a permit#

Updating a permit request#

Reading Permit Data#

Threads of a client#

Idling, Activating and Deactivating a Permit#

Releasing a permit#

TCM Data Structures#

Dependency on HWLOC#

CPU Mask Adapter#

TCM Function Result#

Permit State#

Permit Properties#

Callback Type#

Callback Invocation Reasons#

Permits#

Permit Constraints#

Permit Requests#