Skip to content

Commit 8d5d097

Browse files
jeanbezhoujunhyunjuoohHyunju Ohgithub-actions[bot]
authored
Sync develop to stable (v.0.5) (#201)
* Update getting_started.rst (#184) * Removing gres option for ctest (#182) * Removing gres option for ctest * Removing gres option from scripts * Update check for core --------- Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * enable cache by default (#187) * Removing PDC macro (#189) * Removing gres option for ctest * Removing gres option from scripts * Update check for core * Remove PDC macro * Committing clang-format changes --------- Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * BDCATS fix (#193) * Fix issues with bdcats_batch * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Update mpi_test.sh (#197) * Update .gitlab-ci.yml (#195) * Updates for latest integration with Jacamar and Gitlab tokens in CI * VPICIO bugfix (#196) * Fix VPICIO bug * Add more checks and error out when no server is selected * Committing clang-format changes * Add VPICIO and BDCATS to MPI test --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Fix vpicio_mts (#199) --------- Co-authored-by: Houjun Tang <htang4@lbl.gov> Co-authored-by: Hyunju Oh <oh.693@osu.edu> Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>
1 parent a47af22 commit 8d5d097

31 files changed

+428
-238
lines changed

‎.gitlab-ci.yml‎

Lines changed: 83 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,12 @@ stages:
99

1010
perlmutter-no-cache-build:
1111
stage: build
12-
when: manual
12+
rules:
13+
- if: '$METRICS == null'
14+
when: manual
15+
id_tokens:
16+
SITE_ID_TOKEN:
17+
aud: https://software.nersc.gov/
1318
allow_failure: false
1419
tags:
1520
- perlmutter
@@ -32,7 +37,12 @@ perlmutter-no-cache-build:
3237

3338
perlmutter-cache-build:
3439
stage: build
35-
when: manual
40+
rules:
41+
- if: '$METRICS == null'
42+
when: manual
43+
id_tokens:
44+
SITE_ID_TOKEN:
45+
aud: https://software.nersc.gov/
3646
allow_failure: false
3747
tags:
3848
- perlmutter
@@ -60,6 +70,9 @@ perlmutter-cache-build:
6070

6171
perlmutter-no-cache-parallel-pdc:
6272
stage: test
73+
id_tokens:
74+
SITE_ID_TOKEN:
75+
aud: https://software.nersc.gov/
6376
rules:
6477
- if: '$METRICS == null'
6578
needs:
@@ -80,6 +93,9 @@ perlmutter-no-cache-parallel-pdc:
8093

8194
perlmutter-no-cache-parallel-obj:
8295
stage: test
96+
id_tokens:
97+
SITE_ID_TOKEN:
98+
aud: https://software.nersc.gov/
8399
rules:
84100
- if: '$METRICS == null'
85101
needs:
@@ -101,6 +117,9 @@ perlmutter-no-cache-parallel-obj:
101117

102118
perlmutter-no-cache-parallel-cont:
103119
stage: test
120+
id_tokens:
121+
SITE_ID_TOKEN:
122+
aud: https://software.nersc.gov/
104123
rules:
105124
- if: '$METRICS == null'
106125
needs:
@@ -122,6 +141,9 @@ perlmutter-no-cache-parallel-cont:
122141

123142
perlmutter-no-cache-parallel-prop:
124143
stage: test
144+
id_tokens:
145+
SITE_ID_TOKEN:
146+
aud: https://software.nersc.gov/
125147
rules:
126148
- if: '$METRICS == null'
127149
needs:
@@ -143,6 +165,9 @@ perlmutter-no-cache-parallel-prop:
143165

144166
perlmutter-no-cache-parallel-region:
145167
stage: test
168+
id_tokens:
169+
SITE_ID_TOKEN:
170+
aud: https://software.nersc.gov/
146171
rules:
147172
- if: '$METRICS == null'
148173
needs:
@@ -164,6 +189,9 @@ perlmutter-no-cache-parallel-region:
164189

165190
perlmutter-no-cache-parallel-region-all:
166191
stage: test
192+
id_tokens:
193+
SITE_ID_TOKEN:
194+
aud: https://software.nersc.gov/
167195
rules:
168196
- if: '$METRICS == null'
169197
needs:
@@ -190,6 +218,9 @@ perlmutter-no-cache-parallel-region-all:
190218

191219
perlmutter-cache-parallel-pdc:
192220
stage: test
221+
id_tokens:
222+
SITE_ID_TOKEN:
223+
aud: https://software.nersc.gov/
193224
rules:
194225
- if: '$METRICS == null'
195226
needs:
@@ -210,6 +241,9 @@ perlmutter-cache-parallel-pdc:
210241

211242
perlmutter-cache-parallel-obj:
212243
stage: test
244+
id_tokens:
245+
SITE_ID_TOKEN:
246+
aud: https://software.nersc.gov/
213247
rules:
214248
- if: '$METRICS == null'
215249
needs:
@@ -231,6 +265,9 @@ perlmutter-cache-parallel-obj:
231265

232266
perlmutter-cache-parallel-cont:
233267
stage: test
268+
id_tokens:
269+
SITE_ID_TOKEN:
270+
aud: https://software.nersc.gov/
234271
rules:
235272
- if: '$METRICS == null'
236273
needs:
@@ -252,6 +289,9 @@ perlmutter-cache-parallel-cont:
252289

253290
perlmutter-cache-parallel-prop:
254291
stage: test
292+
id_tokens:
293+
SITE_ID_TOKEN:
294+
aud: https://software.nersc.gov/
255295
rules:
256296
- if: '$METRICS == null'
257297
needs:
@@ -273,6 +313,9 @@ perlmutter-cache-parallel-prop:
273313

274314
perlmutter-cache-parallel-region:
275315
stage: test
316+
id_tokens:
317+
SITE_ID_TOKEN:
318+
aud: https://software.nersc.gov/
276319
rules:
277320
- if: '$METRICS == null'
278321
needs:
@@ -294,6 +337,9 @@ perlmutter-cache-parallel-region:
294337

295338
perlmutter-cache-parallel-region-all:
296339
stage: test
340+
id_tokens:
341+
SITE_ID_TOKEN:
342+
aud: https://software.nersc.gov/
297343
rules:
298344
- if: '$METRICS == null'
299345
needs:
@@ -313,26 +359,54 @@ perlmutter-cache-parallel-region-all:
313359
- ctest -L parallel_region_transfer_all
314360
- rm -rf ${PDC_TMPDIR} ${PDC_DATA_LOC}
315361

362+
perlmutter-metrics-build:
363+
stage: build
364+
id_tokens:
365+
SITE_ID_TOKEN:
366+
aud: https://software.nersc.gov/
367+
rules:
368+
- if: '$METRICS == "true"'
369+
allow_failure: false
370+
tags:
371+
- perlmutter
372+
variables:
373+
SCHEDULER_PARAMETERS: "-A m2621 --qos=debug --constraint=cpu --tasks-per-node=64 -N 1 -t 00:30:00"
374+
SUPERCOMPUTER: "perlmutter"
375+
MERCURY_DIR: "/global/cfs/cdirs/m2621/pdc-perlmutter/mercury/install"
376+
script:
377+
- module load libfabric/1.15.2.0
378+
- module list
379+
- mkdir -p ${PDC_BUILD_PATH}/perlmutter/metrics
380+
- cd ${PDC_BUILD_PATH}/perlmutter/metrics
381+
- cmake ../../.. -DBUILD_MPI_TESTING=ON -DBUILD_SHARED_LIBS=ON -DPDC_SERVER_CACHE=ON -DBUILD_TESTING=ON -DCMAKE_INSTALL_PREFIX=$PDC_DIR -DPDC_ENABLE_MPI=ON -DMERCURY_DIR=$MERCURY_DIR -DCMAKE_C_COMPILER=cc -DMPI_RUN_CMD="srun -A m2621 --qos=debug --constraint=cpu --tasks-per-node=64" -DCMAKE_INSTALL_PREFIX=${PDC_INSTALL_PATH}/perlmutter/metrics
382+
- make -j
383+
- make install
384+
artifacts:
385+
paths:
386+
- ${PDC_BUILD_PATH}/perlmutter/metrics
387+
- ${PDC_INSTALL_PATH}/perlmutter/metrics
388+
316389
perlmutter-metrics:
317390
stage: metrics
391+
id_tokens:
392+
SITE_ID_TOKEN:
393+
aud: https://software.nersc.gov/
318394
rules:
319395
- if: '$METRICS == "true"'
320396
needs:
321-
- perlmutter-cache-build
397+
- perlmutter-metrics-build
322398
tags:
323399
- perlmutter
324400
variables:
325-
PDC_N_NODES: 64
326-
PDC_N_CLIENTS: 127
327-
SCHEDULER_PARAMETERS: "-A m2621 --qos=regular --constraint=cpu --tasks-per-node=${PDC_N_CLIENTS} -N ${PDC_N_NODES} -t 00:30:00"
401+
SCHEDULER_PARAMETERS: "-A m2621 --qos=${PDC_QUEUE} --constraint=cpu --tasks-per-node=${PDC_N_CLIENTS} -N ${PDC_N_NODES} -t 00:30:00"
328402
SUPERCOMPUTER: "perlmutter"
329403
MERCURY_DIR: "/global/cfs/cdirs/m2621/pdc-perlmutter/mercury/install"
330404
PDC_TMPDIR: "${PDC_BUILD_PATH}/pdc-tmp-metrics"
331405
PDC_DATA_LOC: "${PDC_BUILD_PATH}/pdc-data-metrics"
332406
PDC_CLIENT_LOOKUP: "NONE"
333-
PDC_SERVER: "${PDC_BUILD_PATH}/perlmutter/cache/bin/pdc_server.exe"
334-
PDC_SERVER_CLOSE: "${PDC_BUILD_PATH}/perlmutter/cache/bin/close_server"
335-
PDC_CLIENT: "${PDC_BUILD_PATH}/perlmutter/cache/bin/vpicio_mts"
407+
PDC_SERVER: "${PDC_BUILD_PATH}/perlmutter/metrics/bin/pdc_server.exe"
408+
PDC_SERVER_CLOSE: "${PDC_BUILD_PATH}/perlmutter/metrics/bin/close_server"
409+
PDC_CLIENT: "${PDC_BUILD_PATH}/perlmutter/metrics/bin/vpicio_mts"
336410
PDC_JOB_OUTPUT: "pdc-metrics.log"
337411
script:
338412
- hostname

‎CMakeLists.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -380,7 +380,7 @@ endif()
380380
#-----------------------------------------------------------------------------
381381
# SERVER CACHE option
382382
#-----------------------------------------------------------------------------
383-
option(PDC_SERVER_CACHE "Enable Server Caching." OFF)
383+
option(PDC_SERVER_CACHE "Enable Server Caching." ON)
384384
if(PDC_SERVER_CACHE)
385385
set(PDC_SERVER_CACHE 1)
386386
set(PDC_SERVER_CACHE_MAX_GB "3" CACHE STRING "Max GB for server cache")

‎docs/source/getting_started.rst‎

Lines changed: 14 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -125,11 +125,8 @@ Install libfabric
125125

126126
.. attention::
127127

128-
If you're installing PDC on MacOS, you need to make sure you enable ``sockets``:
129-
130-
.. code-block: Bash
131-
132-
./configure CFLAG=-O2 --enable-sockets=yes --enable-tcp=yes --enable-udp=yes --enable-rxm=yes
128+
When installing on MacOS, make sure to enable ``sockets`` with the following configure command:
129+
``./configure CFLAG=-O2 --enable-sockets=yes --enable-tcp=yes --enable-udp=yes --enable-rxm=yes``
133130

134131

135132
Install Mercury
@@ -156,17 +153,13 @@ Install Mercury
156153
echo 'export PATH=$MERCURY_DIR/include:$MERCURY_DIR/lib:$PATH' >> $WORK_SPACE/pdc_env.sh
157154
158155
.. note::
156+
159157
``CC=mpicc`` may need to be changed to the corresponding compiler in your system, e.g. ``-DCMAKE_C_COMPILER=cc`` or ``-DCMAKE_C_COMPILER=gcc``.
160158
Make sure the ctest passes. PDC may not work without passing all the tests of Mercury.
161159

162160
.. attention::
163161

164-
If you're installing PDC on MacOS, for the tests to work you need to specify the protocol used by Mercury:
165-
166-
.. code-block: Bash
167-
168-
cmake -DCMAKE_INSTALL_PREFIX=$MERCURY_DIR -DCMAKE_C_COMPILER=mpicc -DBUILD_SHARED_LIBS=ON \
169-
-DBUILD_TESTING=ON -DNA_USE_OFI=ON -DNA_USE_SM=OFF -DNA_OFI_TESTING_PROTOCOL=sockets
162+
When installing on MacOS, specify the ``sockets`` protocol used by Mercury by replacing the cmake command from ``-DNA_OFI_TESTING_PROTOCOL=tcp`` to ``-DNA_OFI_TESTING_PROTOCOL=sockets``
170163

171164

172165
Install PDC
@@ -197,11 +190,7 @@ Install PDC
197190
If you are trying to compile PDC on MacOS, ``LibUUID`` needs to be installed on your MacOS first. Simple use ``brew install ossp-uuid`` to install it.
198191
If you are trying to compile PDC on Linux, you should also make sure ``LibUUID`` is installed on your system. If not, you can install it with ``sudo apt-get install uuid-dev`` on Ubuntu or ``yum install libuuid-devel`` on CentOS.
199192

200-
In MacOS you also need to export the following environment variable so PDC (i.e., Mercury) uses the ``socket`` protocol, the only one supported in MacOS:
201-
202-
.. code-block: Bash
203-
204-
export HG_TRANSPORT="sockets"
193+
In MacOS you also need to export the following environment variable so PDC (i.e., Mercury) uses the ``socket`` protocol, the only one supported in MacOS: ``export HG_TRANSPORT="sockets"``
205194

206195

207196
Test Your PDC Installation
@@ -219,6 +208,7 @@ You can also specify a timeout (e.g., 2 minutes) for the tests by specifying the
219208
ctest --timeout 120
220209
221210
.. note::
211+
222212
If you are using PDC on an HPC system, e.g. Perlmutter@NERSC, ``ctest`` should be run on a compute node, you can submit an interactive job on Perlmutter: ``salloc --nodes 1 --qos interactive --time 01:00:00 --constraint cpu --account=mxxxx``
223213

224214

@@ -232,23 +222,21 @@ But before adding any of your function, we need to enable the Julia support firs
232222
Prerequisite
233223
^^^^^^^^^^^^
234224

235-
Make sure you have Julia-lang installed. You can check with your system administrator to see if you already have Julia-lang installed. If not, you can either ask your system administrator to install it for you or you can install it yourself if permitted. On macOS, the best way to install Julia is via `Homebrew https://brew.sh`_. You may also refer to `Julia Download Page https://julialang.org/downloads/`_ for instructions on installing Julia.
236-
Once you installed Julia, you can set `JULIA_HOME` to be where Julia-lang is installed.
225+
Make sure you have Julia-lang installed. You can check with your system administrator to see if you already have Julia-lang installed. If not, you can either ask your system administrator to install it for you or you can install it yourself if permitted. On macOS, the best way to install Julia is via `Homebrew <https://brew.sh>`_. You may also refer to `Julia Download Page <https://julialang.org/downloads/>`_ for instructions on installing Julia.
226+
Once you installed Julia, you can set ``JULIA_HOME`` to be where Julia-lang is installed.
237227

238228
.. code-block:: Bash
239-
export JULIA_HOME=/path/to/julia/install/directory
240229
230+
export JULIA_HOME=/path/to/julia/install/directory
241231
242-
.. note:: Note on perlmutter:
243-
You can easily perform `module load julia` to load the Julia-lang environment. Then, you can do the following to set `$JULIA_HOME`:
232+
.. note::
244233

245-
.. code-block:: Bash
246-
export JULIA_HOME=$(dirname $(dirname $(which julia)))
234+
On Perlmutter You can ``module load julia`` to load the Julia-lang module and set ``$JULIA_HOME`` with: ``export JULIA_HOME=$(dirname $(dirname $(which julia)))``
247235

248236

249237
Enabling Julia Support for PDC Tests
250238
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
251-
Once the Prerequisite is satisfied, you can enable Julia support by adding `--DPDC_ENABLE_JULIA_SUPPORT=ON` to your cmake command and re-run it.
239+
Once the Prerequisite is satisfied, you can enable Julia support by adding ``--DPDC_ENABLE_JULIA_SUPPORT=ON`` to your cmake command and re-run it.
252240
Then you can compile your PDC project with Julia support.
253241

254242
Now, see Developer Notes to know how you can add your own Julia functions to enhance your test cases in PDC.
@@ -257,9 +245,10 @@ Now, see Developer Notes to know how you can add your own Julia functions to enh
257245
Build PDC in a Docker Container
258246
---------------------------------------------------
259247
Simply run the following command from the project root directory to build PDC in a Docker container:
248+
260249
.. code-block:: Bash
261-
.docker/run_dev_base.sh
262250
251+
.docker/run_dev_base.sh
263252
264253
Build PDC in Github Codespace
265254
---------------------------------------------------

‎examples/Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ CFLAGS = -O2 -Wall -Wextra
44
INCLUDES = -I. -I$(PDC_DIR)/include
55
LDFLAGS = -Wl,-rpath,$(PDC_DIR)/lib -L$(PDC_DIR)/lib
66

7-
LIBS = -lpdc
7+
LIBS = -lpdc -lpdc_commons
88

99
.c.o:
1010
$(CC) $(CFLAGS) $(INCLUDES) -c $<

0 commit comments

Comments
 (0)