3. Using PDC

3.1. Overview

This section provides a practical overview of how to use the PDC library to manage and transfer data in high-performance computing environments. It walks through the essential steps of initializing PDC, creating containers and objects, defining regions, and performing data transfers.

Basic Usage

Complete Examples

3.2. Initializing PDC

Prior to any interaction with PDC, the user needs to initialize it as shown below:

pdcid_t pdc_id = PDCinit("pdc");

At the end of the application a corresponding deinitialization function should be called:

PDCclose(pdc_id);

Note

Users should check that every PDC API call succeeds. In general, if a function returns a pdcid_t, 0 indicates an error. If a function returns a perr_t, a negative value indicates an error.

3.3. Container Lifecycle

Containers store objects and provide users a way to organize their data. Before creating a container, a container property must be constructed. The container property provides users a method for customizing a container’s behavior. For an exhaustive list of container properties, please see FIXME.

This is shown in the example below:

pdcid_t cont_prop_id = PDCprop_create(PDC_CONT_CREATE, pdc_id);

// Independent container creation
pdcid_t cont_id = PDCcont_create("cont", cont_prop_id)

// Collective container creation
pdcid_t cont_col_id = PDCcont_create_col("cont", cont_prop_id);

To open an existing container:

pdcid_t cont_id = PDCcont_open("cont");

The following functions should be used to free both the container and its associated property resources:

PDCprop_close(cont_prop_id);
PDCcont_close(cont_id);
PDCcont_close(cont_col_id);

3.4. Object Lifecycle

Objects represent user data and are the entities stored within containers in PDC. Before creating an object, an object property must be defined, which specifies metadata such as dimensionality, size, data type, and region partitioning. For an exhaustive list of object properties, please see FIXME. This allows for fine-grained control over how data is laid out and accessed.

Below is an example of setting up an object property and creating objects:

// Create object property
pdcid_t obj_prop_id = PDCprop_create(PDC_OBJ_CREATE, pdc_id);

// Set properties: type, dims, etc.
uint64_t dims[1] = {1024};
PDCprop_set_obj_dims(obj_prop_id, dims);
PDCprop_set_obj_type(obj_prop_id, PDC_FLOAT);

// Independent object creation
pdcid_t obj_id = PDCobj_create(cont_id, "obj", obj_prop_id);

// Collective object creation
pdcid_t obj_col_id = PDCobj_create_col(cont_id, "obj", obj_prop_id, my_rank, comm);

To open an existing object by name within a container:

pdcid_t obj_id = PDCobj_open(cont_id, "obj");

When the object and its property are no longer needed, they should be closed to free resources:

PDCprop_close(obj_prop_id);
PDCobj_close(obj_id);
PDCobj_close(obj_col_id);

3.5. Region Transfer Lifecycle

Regions define logical subranges within a PDC object and are used to specify what part of the object’s data will be transferred between memory and storage.

Transfers can be performed in three main modes:

  • Individually, with PDCregion_transfer_start()

  • Collectively, with PDCregion_transfer_start_mpi() across MPI processes

  • In batches, with PDCregion_transfer_start_all() and PDCregion_transfer_wait_all()

Basic Region Transfer

Create memory and object regions and initiate a transfer:

uint64_t offset[1] = {0};
uint64_t size[1] = {1024};

float *data_buf = malloc(sizeof(float) * size[0]);

pdcid_t mem_reg_id = PDCregion_create(1, offset, size);
pdcid_t obj_reg_id = PDCregion_create(1, offset, size);

pdcid_t xfer = PDCregion_transfer_create(data_buf, PDC_WRITE,
                                         obj_id, obj_reg_id, mem_reg_id);

PDCregion_transfer_start(xfer);
PDCregion_transfer_wait(xfer);

PDCregion_transfer_close(xfer);
PDCregion_close(mem_reg_id);
PDCregion_close(obj_reg_id);
free(data_buf);

Collective Transfer

If the transfer is intended to be performed collectively across MPI ranks, use:

PDCregion_transfer_start_mpi(xfer);

This function should be called by all processes participating in the transfer and is useful for coordinated I/O in distributed applications. The rest of the transfer workflow (e.g., PDCregion_transfer_wait()) remains unchanged.

Batch Region Transfer

For scenarios involving many objects or regions, PDC supports batch transfers to reduce overhead:

#define OBJ_NUM 10
#define BUF_LEN 256

int *data[OBJ_NUM];
pdcid_t transfer_requests[OBJ_NUM];
pdcid_t reg = PDCregion_create(1, offset, size);
pdcid_t reg_global = PDCregion_create(1, offset, size);

for (int i = 0; i < OBJ_NUM; ++i) {
    data[i] = malloc(sizeof(int) * BUF_LEN);
    for (int j = 0; j < BUF_LEN; ++j)
        data[i][j] = j;

    transfer_requests[i] = PDCregion_transfer_create(
        data[i], PDC_WRITE, obj[i], reg, reg_global);
}

// Start all transfers in one batch
PDCregion_transfer_start_all(transfer_requests, OBJ_NUM);

// Wait for all to complete
PDCregion_transfer_wait_all(transfer_requests, OBJ_NUM);

for (int i = 0; i < OBJ_NUM; ++i) {
    PDCregion_transfer_close(transfer_requests[i]);
    free(data[i]);
}

PDCregion_close(reg);
PDCregion_close(reg_global);

3.6. Complete Examples

2D Region Transfer Example

  1    #include <stdio.h>
  2    #include "pdc.h"
  3
  4    #define BUF_LEN 128
  5
  6    int
  7    main(int argc, char **argv)
  8    {
  9        pdcid_t  pdc, cont_prop, cont, obj_prop, memory_region, obj_region;
 10        pdcid_t  obj;
 11        char     cont_name[128], obj_name[128];
 12        pdcid_t  transfer_request;
 13        int      rank = 0, size = 1, i;
 14        int      ret_value = 0;
 15        uint64_t offset[3], offset_length[3];
 16        uint64_t dims[2];
 17        int     *data_write = (int *)malloc(sizeof(int) * BUF_LEN);
 18        int     *data_read  = (int *)malloc(sizeof(int) * BUF_LEN);
 19
 20    #ifdef ENABLE_MPI
 21        MPI_Init(&argc, &argv);
 22        MPI_Comm_rank(MPI_COMM_WORLD, &rank);
 23        MPI_Comm_size(MPI_COMM_WORLD, &size);
 24    #endif
 25
 26        // Initialize PDC runtime
 27        pdc = PDCinit("pdc");
 28
 29        // Configure and create container
 30        cont_prop = PDCprop_create(PDC_CONT_CREATE, pdc);
 31        sprintf(cont_name, "c%d", rank);
 32        cont = PDCcont_create(cont_name, cont_prop);
 33
 34        // Configure and create object
 35        obj_prop = PDCprop_create(PDC_OBJ_CREATE, pdc);
 36        PDCprop_set_obj_type(obj_prop, PDC_INT);
 37        dims[0] = BUF_LEN / 4;
 38        dims[1] = 4;
 39        PDCprop_set_obj_dims(obj_prop, 2, dims);
 40        sprintf(obj_name, "o1_%d", rank);
 41        obj = PDCobj_create(cont, obj_name, obj_prop);
 42
 43        // Configure regions
 44        offset[0]        = 0;
 45        offset[1]        = 0;
 46        offset_length[0] = BUF_LEN / 4;
 47        offset_length[1] = 4;
 48
 49        // Create local region and object region
 50        memory_region = PDCregion_create(1, offset, offset_length);
 51        obj_region    = PDCregion_create(2, offset, offset_length);
 52
 53        // Initialize memory buffer
 54        for (i = 0; i < BUF_LEN; ++i)
 55            data_write[i] = i;
 56
 57        // Create, start, wait, and close write data transfer
 58        transfer_request = PDCregion_transfer_create(data_write, PDC_WRITE, obj, memory_region, obj_region);
 59        PDCregion_transfer_start(transfer_request);
 60        PDCregion_transfer_wait(transfer_request);
 61        PDCregion_transfer_close(transfer_request);
 62
 63        // Create, start, wait, and close read data transfer
 64        transfer_request = PDCregion_transfer_create(data_read, PDC_READ, obj, memory_region, obj_region);
 65        PDCregion_transfer_start(transfer_request);
 66        PDCregion_transfer_wait(transfer_request);
 67        PDCregion_transfer_close(transfer_request);
 68
 69        // Validate data
 70        if (memcmp(data_read, data_write, sizeof(int) * BUF_LEN)) {
 71            printf("Data read was invalid\n");
 72            ret_value = 1;
 73        }
 74
 75        // Close regions
 76        PDCregion_close(memory_region);
 77        PDCregion_close(obj_region);
 78
 79        // Close object
 80        PDCobj_close(obj);
 81
 82        // Close container
 83        PDCcont_close(cont);
 84
 85        // Close object and container properties
 86        PDCprop_close(obj_prop);
 87        PDCprop_close(cont_prop);
 88
 89        // Close PDC runtime
 90        PDCclose(pdc);
 91
 92        // Free memory buffers
 93        free(data_write);
 94        free(data_read);
 95
 96    #ifdef ENABLE_MPI
 97        MPI_Finalize();
 98    #endif
 99
100        if (ret_value)
101            printf("Example had an error\n");
102        else
103            printf("Example ran successfully\n");
104
105        return ret_value;
106    }

2D Batch Region Transfer Example

  1    #include <stdio.h>
  2    #include "pdc.h"
  3
  4    #define BUF_LEN       400
  5    #define CHUNK_LEN     100
  6    #define NUM_TRANSFERS (BUF_LEN / CHUNK_LEN)
  7
  8    int
  9    main(int argc, char **argv)
 10    {
 11        pdcid_t  pdc, cont_prop, cont, obj_prop;
 12        pdcid_t  obj_id;
 13        pdcid_t  memory_regions[4], obj_regions[4], transfers[4];
 14        char     cont_name[128], obj_name[128];
 15        int      rank = 0, size = 1, i;
 16        int      ret_value = 0;
 17        uint64_t dims[2];
 18        uint64_t offsets[2];
 19        uint64_t region_size[2];
 20        int     *data_write;
 21        int     *data_read;
 22
 23        // Allocate buffers
 24        data_write = (int *)malloc(sizeof(int) * BUF_LEN);
 25        data_read  = (int *)calloc(BUF_LEN, sizeof(int));
 26
 27    #ifdef ENABLE_MPI
 28        MPI_Init(&argc, &argv);
 29        MPI_Comm_rank(MPI_COMM_WORLD, &rank);
 30        MPI_Comm_size(MPI_COMM_WORLD, &size);
 31    #endif
 32
 33        // Initialize PDC runtime
 34        pdc = PDCinit("pdc");
 35
 36        // Configure and create container
 37        cont_prop = PDCprop_create(PDC_CONT_CREATE, pdc);
 38        sprintf(cont_name, "c%d", rank);
 39        cont = PDCcont_create(cont_name, cont_prop);
 40
 41        // Configure and create object
 42        obj_prop = PDCprop_create(PDC_OBJ_CREATE, pdc);
 43        dims[0]  = 40; // total height
 44        dims[1]  = 10; // total width (total = 400 elements)
 45        PDCprop_set_obj_dims(obj_prop, 2, dims);
 46        PDCprop_set_obj_type(obj_prop, PDC_INT);
 47        sprintf(obj_name, "o1_%d", rank);
 48        obj_id = PDCobj_create(cont, obj_name, obj_prop);
 49
 50        // Define region size (10x10) and number of transfers
 51        region_size[0] = 10;
 52        region_size[1] = 10;
 53
 54        // Initialize memory buffer
 55        for (i = 0; i < BUF_LEN; i++)
 56            data_write[i] = i;
 57
 58        // Create memory and object regions and start write transfers
 59        for (i = 0; i < NUM_TRANSFERS; i++) {
 60            offsets[0] = i * 10; // offset along first dimension (object)
 61            offsets[1] = 0;      // offset along second dimension
 62
 63            // Minimal change: memory region always starts at {0,0}
 64            memory_regions[i] = PDCregion_create(2, (uint64_t[]){0, 0}, region_size);
 65            obj_regions[i]    = PDCregion_create(2, offsets, region_size);
 66
 67            // Create region transfer for writing correct slice of buffer
 68            transfers[i] = PDCregion_transfer_create(data_write + i * CHUNK_LEN, // offset in local memory
 69                                                    PDC_WRITE, obj_id, memory_regions[i], obj_regions[i]);
 70        }
 71
 72        // Start and wait for all writes
 73        PDCregion_transfer_start_all(transfers, NUM_TRANSFERS);
 74        PDCregion_transfer_wait_all(transfers, NUM_TRANSFERS);
 75
 76        // Close write transfers
 77        for (i = 0; i < NUM_TRANSFERS; i++)
 78            PDCregion_transfer_close(transfers[i]);
 79
 80        // Now read back into data_read in four slices
 81        for (i = 0; i < NUM_TRANSFERS; i++) {
 82            transfers[i] = PDCregion_transfer_create(data_read + i * CHUNK_LEN, // offset in local memory
 83                                                    PDC_READ, obj_id, memory_regions[i], obj_regions[i]);
 84        }
 85
 86        // Start and wait for all reads
 87        PDCregion_transfer_start_all(transfers, NUM_TRANSFERS);
 88        PDCregion_transfer_wait_all(transfers, NUM_TRANSFERS);
 89
 90        // Close read transfers and regions
 91        for (i = 0; i < NUM_TRANSFERS; i++) {
 92            PDCregion_transfer_close(transfers[i]);
 93            PDCregion_close(memory_regions[i]);
 94            PDCregion_close(obj_regions[i]);
 95        }
 96
 97        // Validate read-back
 98        if (memcmp(data_read, data_write, sizeof(int) * BUF_LEN) != 0) {
 99            printf("Data read was invalid\n");
100            ret_value = 1;
101        }
102
103        // Close object and container
104        PDCobj_close(obj_id);
105        PDCcont_close(cont);
106
107        // Close object and container properties
108        PDCprop_close(obj_prop);
109        PDCprop_close(cont_prop);
110
111        // Close PDC runtime
112        PDCclose(pdc);
113
114        // Free memory buffers
115        free(data_write);
116        free(data_read);
117
118    #ifdef ENABLE_MPI
119        MPI_Finalize();
120    #endif
121
122        if (ret_value)
123            printf("Example had an error\n");
124        else
125            printf("Example ran successfully\n");
126
127        return ret_value;
128    }

Get Put Object Example

 1    #include <stdio.h>
 2    #include <stdlib.h>
 3    #include <string.h>
 4    #include "pdc.h"
 5
 6    #define BUF_LEN 128 // size of data buffer
 7
 8    int
 9    main(int argc, char **argv)
10    {
11        pdcid_t pdc, cont_prop, cont;
12        pdcid_t obj1, obj2;
13        char    cont_name[128], obj_name1[128], obj_name2[128];
14        int    *data_write, *data_read;
15        int     rank      = 0, size, i;
16        int     ret_value = 0;
17
18    #ifdef ENABLE_MPI
19        MPI_Init(&argc, &argv);
20        MPI_Comm_rank(MPI_COMM_WORLD, &rank);
21        MPI_Comm_size(MPI_COMM_WORLD, &size);
22    #endif
23
24        // Allocate buffers
25        data_write = (int *)malloc(sizeof(int) * BUF_LEN);
26        data_read  = (int *)calloc(BUF_LEN, sizeof(int));
27
28        // Initialize PDC runtime
29        pdc = PDCinit("pdc");
30
31        // Initialize memory buffer
32        for (i = 0; i < BUF_LEN; i++)
33            data_write[i] = i;
34
35        // Configure and create container
36        cont_prop = PDCprop_create(PDC_CONT_CREATE, pdc);
37        sprintf(cont_name, "c%d", rank);
38        cont = PDCcont_create(cont_name, cont_prop);
39
40        // Initialize data and put first object
41        sprintf(obj_name1, "o1_%d", rank);
42        obj1 = PDCobj_put_data(obj_name1, data_write, BUF_LEN * sizeof(int), cont);
43
44        // Initialize data and put second object
45        sprintf(obj_name2, "o2_%d", rank);
46        obj2 = PDCobj_put_data(obj_name2, data_write, BUF_LEN * sizeof(int), cont);
47
48        // Get first object
49        PDCobj_get_data(obj1, data_read, BUF_LEN * sizeof(int));
50
51        // Validate first object
52        if (memcmp(data_write, data_read, BUF_LEN * sizeof(int)) != 0) {
53            printf("Data read was invalid for obj1\n");
54            ret_value = 1;
55        }
56
57        // Get second object
58        PDCobj_get_data(obj2, data_read, BUF_LEN * sizeof(int));
59
60        // Validate second object
61        if (memcmp(data_write, data_read, BUF_LEN * sizeof(int)) != 0) {
62            printf("Data read was invalid for obj2\n");
63            ret_value = 1;
64        }
65
66        // Close objects and container
67        PDCobj_close(obj1);
68        PDCobj_close(obj2);
69        PDCcont_close(cont);
70
71        // Close container property
72        PDCprop_close(cont_prop);
73
74        // Close PDC runtime
75        PDCclose(pdc);
76
77        // Free memory buffers
78        free(data_write);
79        free(data_read);
80
81    #ifdef ENABLE_MPI
82        MPI_Finalize();
83    #endif
84
85        if (ret_value)
86            printf("Example had an error\n");
87        else
88            printf("Example ran successfully\n");
89
90        return ret_value;
91    }

Add Get KV Tag

 1    #include <stdio.h>
 2    #include <stdlib.h>
 3    #include <string.h>
 4    #include "pdc.h"
 5
 6    int main() {
 7        pdcid_t pdc, cont_prop, cont, obj_prop1, obj_prop2, obj1, obj2;
 8        pdc_kvtag_t kvtag1, kvtag2, kvtag3;
 9        char *v1 = "value1";
10        int v2 = 2;
11        double v3 = 3.45;
12        pdc_var_type_t type1, type2, type3;
13        void *value1, *value2, *value3;
14        psize_t value_size;
15
16        // create a pdc
17        pdc = PDCinit("pdc");
18
19        // create container property and container
20        cont_prop = PDCprop_create(PDC_CONT_CREATE, pdc);
21        cont = PDCcont_create("c1", cont_prop);
22
23        // create object properties
24        obj_prop1 = PDCprop_create(PDC_OBJ_CREATE, pdc);
25        obj_prop2 = PDCprop_create(PDC_OBJ_CREATE, pdc);
26
27        // create objects
28        obj1 = PDCobj_create(cont, "o1", obj_prop1);
29        obj2 = PDCobj_create(cont, "o2", obj_prop2);
30
31        // define key-value tags
32        kvtag1.name = "key1string";
33        kvtag1.value = (void *)v1;
34        kvtag1.type = PDC_STRING;
35        kvtag1.size = strlen(v1) + 1;
36
37        kvtag2.name = "key2int";
38        kvtag2.value = (void *)&v2;
39        kvtag2.type = PDC_INT;
40        kvtag2.size = sizeof(int);
41
42        kvtag3.name = "key3double";
43        kvtag3.value = (void *)&v3;
44        kvtag3.type = PDC_DOUBLE;
45        kvtag3.size = sizeof(double);
46
47        // put tags for obj1
48        PDCobj_put_tag(obj1, kvtag1.name, kvtag1.value, kvtag1.type, kvtag1.size);
49        PDCobj_put_tag(obj1, kvtag2.name, kvtag2.value, kvtag2.type, kvtag2.size);
50
51        // put tag for obj2
52        PDCobj_put_tag(obj2, kvtag3.name, kvtag3.value, kvtag3.type, kvtag3.size);
53
54        // get tags
55        PDCobj_get_tag(obj1, kvtag1.name, (void *)&value1, (void *)&type1, (void *)&value_size);
56        PDCobj_get_tag(obj2, kvtag1.name, (void *)&value2, (void *)&type2, (void *)&value_size);
57        PDCobj_get_tag(obj2, kvtag3.name, (void *)&value3, (void *)&type3, (void *)&value_size);
58
59        // delete and put new tag for obj1
60        PDCobj_del_tag(obj1, kvtag1.name);
61        v1 = "New Value After Delete";
62        kvtag1.value = (void *)v1;
63        kvtag1.size = strlen(v1) + 1;
64        PDCobj_put_tag(obj1, kvtag1.name, kvtag1.value, kvtag1.type, kvtag1.size);
65        PDCobj_get_tag(obj1, kvtag1.name, (void *)&value1, (void *)&type1, (void *)&value_size);
66
67        // close objects, container, properties, and pdc
68        PDCobj_close(obj1);
69        PDCobj_close(obj2);
70        PDCcont_close(cont);
71        PDCprop_close(obj_prop1);
72        PDCprop_close(obj_prop2);
73        PDCprop_close(cont_prop);
74        PDCclose(pdc);
75
76        return 0;
77    }

Querying Object Data

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include "pdc.h"

int main(int argc, char **argv) {
    int rank = 0, size = 1;
    uint64_t size_MB;
    pdcid_t obj_id = -1;
    struct pdc_region_info region;
    uint64_t i, dims[1];
    pdc_selection_t sel;
    char *obj_name;
    int my_data_count;
    pdc_metadata_t *metadata;
    pdcid_t pdc, cont_prop, cont, obj_prop;
    int ndim = 1;
    int *mydata;

#ifdef ENABLE_MPI
    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
#endif

    if (argc < 3)
        return 1;

    obj_name = argv[1];
    size_MB  = atoi(argv[2]) * 1048576; // convert MB to bytes

    // create a PDC
    pdc = PDCinit("pdc");

    // create container property and container
    cont_prop = PDCprop_create(PDC_CONT_CREATE, pdc);
    cont = PDCcont_create("c1", cont_prop);

    // create object property
    obj_prop = PDCprop_create(PDC_OBJ_CREATE, pdc);

    my_data_count = size_MB / size;
    dims[0] = my_data_count;
    PDCprop_set_obj_dims(obj_prop, 1, dims);
    PDCprop_set_obj_user_id(obj_prop, getuid());
    PDCprop_set_obj_time_step(obj_prop, 0);
    PDCprop_set_obj_app_name(obj_prop, "DataServerTest");
    PDCprop_set_obj_tags(obj_prop, "tag0=1");
    PDCprop_set_obj_type(obj_prop, PDC_INT);

    // create object (only rank 0)
    if (rank == 0)
        obj_id = PDCobj_create(cont, obj_name, obj_prop);

#ifdef ENABLE_MPI
    MPI_Barrier(MPI_COMM_WORLD);
#endif

    region.ndim = ndim;
    region.offset = (uint64_t *)malloc(sizeof(uint64_t) * ndim);
    region.size   = (uint64_t *)malloc(sizeof(uint64_t) * ndim);
    region.offset[0] = rank * my_data_count;
    region.size[0]   = my_data_count;

    mydata = (int *)malloc(my_data_count);
    for (i = 0; i < my_data_count / sizeof(int); i++)
        mydata[i] = i + rank * 1000;

    PDC_Client_write(metadata, &region, mydata);

    // construct a simple query example
    int lo0 = 1000;
    pdc_query_t *q0 = PDCquery_create(obj_id, PDC_LT, PDC_INT, &lo0);
    PDCquery_sel_region(q0, &region);

    PDCquery_get_selection(q0, &sel);
    PDCselection_print(&sel);

    // free resources
    PDCquery_free_all(q0);
    PDCregion_free(&region);
    PDCselection_free(&sel);
    free(mydata);

    PDCcont_close(cont);
    PDCprop_close(cont_prop);
    PDCprop_close(obj_prop);
    PDCclose(pdc);

#ifdef ENABLE_MPI
    MPI_Finalize();
#endif

    return 0;
}