Client Propogate HG_Finalize error on PDCclose

[TheAssembler1]

This PR fixes this issue on the client tests. The server has similar issues. In the following functions bulk handles were created and never destroyed on the client side:

PDC_Client_transfer_request
PDC_Client_transfer_request_all
PDC_Client_transfer_request_metadata_query
PDC_Client_transfer_request_metadata_query2

These functions now take an additional hg_bulk_t* parameter which they set to the bulk handle created within the function. This is a pointer into the array of bulk handles for the corresponding region transfer.

To fix this an array of hg_bulk_t has been added to the PDC region transfer struct. The bulk handles created are added to the array. On a transfer close HG_Bulk_free is called on each bulk handle in the array.

• System: Perlmutter
• Nodes: 4
• Clients: 32 per node
• Servers: 1 per node
• Mercury: Single-threaded
• Cache: Enabled
• Commands:
  • VPICIO: ./vpicio_mts 8388608 5 20
  • BDCATS: ./vpicio ./bdcats

VPICIO_MTS Develop time (old):

Step	Time Type	Time Value (s)
0	Obj create time	3.79819e-02
0	Transfer create time	1.89206e-04
0	Transfer start time	8.81842e-01
0	Transfer wait time	5.68597e-02
0	Transfer close time	7.25157e-02
0	Obj close time	9.77335e-04
1	Obj create time	1.18654e-03
1	Transfer create time	2.67920e-05
1	Transfer start time	1.02360e+00
1	Transfer wait time	1.32388e-01
1	Transfer close time	1.20518e-01
1	Obj close time	7.96650e-05
2	Obj create time	7.67836e-03
2	Transfer create time	3.07790e-05
2	Transfer start time	9.94380e-01
2	Transfer wait time	1.03283e-01
2	Transfer close time	6.81297e-02
2	Obj close time	6.10180e-05
3	Obj create time	1.26300e-03
3	Transfer create time	2.60810e-05
3	Transfer start time	1.01179e+00
3	Transfer wait time	6.62225e-02
3	Transfer close time	8.85037e-02
3	Obj close time	5.97460e-05
4	Obj create time	1.30326e-03
4	Transfer create time	2.59000e-05
4	Transfer start time	1.04613e+00
4	Transfer wait time	5.82398e+00
4	Transfer close time	1.02357e-01
4	Obj close time	4.92950e-05

VPICIO_MTS New time:

Step	Time Type	Time Value (s)
0	Obj create time	3.66105e-02
0	Transfer create time	2.55625e-04
0	Transfer start time	8.81592e-01
0	Transfer wait time	7.21270e-02
0	Transfer close time	5.83703e-02
0	Obj close time	1.91511e-03
1	Obj create time	1.55751e-03
1	Transfer create time	1.11616e-04
1	Transfer start time	9.94855e-01
1	Transfer wait time	4.39608e-02
1	Transfer close time	1.08936e-01
1	Obj close time	8.30160e-04
2	Obj create time	1.20452e-03
2	Transfer create time	1.42807e-04
2	Transfer start time	1.02291e+00
2	Transfer wait time	1.04457e-01
2	Transfer close time	9.65955e-02
2	Obj close time	2.97206e-04
3	Obj create time	2.77333e-03
3	Transfer create time	5.50470e-05
3	Transfer start time	1.01106e+00
3	Transfer wait time	5.92011e-02
3	Transfer close time	9.21980e-02
3	Obj close time	3.01645e-04
4	Obj create time	1.82119e-03
4	Transfer create time	1.10264e-04
4	Transfer start time	1.07994e+00
4	Transfer wait time	5.38074e+00
4	Transfer close time	1.15311e-01
4	Obj close time	1.50832e-04

The following results were obtained using the time command.

VPICIO/BDCATS Develop time (old):

VPICIO

real    0m15.183s
user    0m0.054s
sys     0m0.122s

BDCATS

real    0m25.232s
user    0m0.065s
sys     0m0.122s

VPICIO/BDCATS New time:

VPICIO

real    0m15.354s
user    0m0.069s
sys     0m0.113s

BDCATS

real    0m25.279s
user    0m0.089s
sys     0m0.103s

Corresponding issue: #266

[jeanbez]

@houjun could you please approve this one as well?