From fchen14 at lsu.edu  Thu May  2 12:44:25 2024
From: fchen14 at lsu.edu (Feng Chen)
Date: Thu, 2 May 2024 17:44:25 +0000
Subject: [LA-SIGMA] LONI QB-4 Friendly User Mode Announcement
Message-ID: <SN4P221MB0827D8A9A659FA5843E749FEA9182@SN4P221MB0827.NAMP221.PROD.OUTLOOK.COM>

Dear LONI HPC users,

We are pleased to announce that the LONI QB-4 cluster can now be accessed in the friendly user mode. You can log in with your current LONI HPC username and password using qbd.loni.org as the hostname. Please bear in mind the QB-4 cluster is still in the deployment process, thus the "friendly user mode", which means that the hardware/software configuration and policy may change without advance notice.

Before you submit jobs on QB-4, please make sure that you review the user documentation here:

http://hpc.loni.org/docs/guides.php?system=Qb4

In summary:

  1.  QB-4 uses two-factor authentication for increased security. Please refer to the information here for how to set up and log in with two-factor authentication on QB-4: http://hpc.loni.org/docs/systems/twofa.php
  2.  The /work and /project filesystems for QB-4 are the same as they are for QB-3.  The /home filesystem is new.
  3.  If you already have a LONI HPC allocation, it should work on QB-4 as well. There are 64 cores on each QB-4 compute node, so please be mindful of the size and duration of your jobs when running on QB-4 otherwise you may find your compute allocation running out quickly.
  4.  QB-4 uses the same workload and resource manager, Slurm, as QB-3. Here's a quick guide:

     *   For GPU nodes, there are two partitions/queues on QB-4. The "gpu2" partition is for nodes with 2 A100 GPUs each (52 nodes in total), which "gpu4" is for nodes with 4 A100 GPUs each (10 nodes in total). When submitting jobs to these queues, one must use the "--gres=gpu:<# of GPUs>" flag to specify the number of GPUs per node. For instance, the command below launches an MPI GPU application with 2 nodes and 2 GPUs per node:

srun -N2 -n128 -p gpu2 --gres=gpu:2 -A <allocation name> ./my_gpu_app

How to submit GPU jobs:
http://hpc.loni.org/docs/slurm.php#gpu

     *   For MPI applications that are compiled with the default MPI on QB-4 (Intel MPI 2021.5.1 with Intel compiler 2021.5.0), please use the "srun" command to launch MPI applications. For instance, the command below launches the MPI application "my_mpmi_app" with 2 nodes, 2 MPI processes per node and 32 threads per MPI process:

srun -N2 -n4 -c32 ./my_mpi_app -p <partition name> -A <allocation name>

, where "-N<number of nodes>" indicates the number of nodes, "-n<number of processes>" the total number of MPI processes and "-c<number of threads>" the number of threads per MPI process.
     *   To submit a batch job, use the "sbatch" command.
     *   To request an interactive session, use the "salloc" command from the login nodes qbd1 or qbd2. For instance, the command below requests an interactive session with one full node for 4 hours in the checkpt partition/queue:

salloc -N1 -n64 -t 4:00:00 -p checkpt -A <allocation name>

Detailed information about Slurm can be found in the user guide.

Please send feedback and report issues to sys-help at loni.org<mailto:sys-help at loni.org>. Thank you!

Feng Chen, PhD
Manager of User Services
High Performance Computing
Louisiana State University
329 Frey Computing Services Center, Baton Rouge, LA  70803
office 225-578-2924
fchen14 at lsu.edu<mailto:fchen14 at lsu.edu>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.loni.org/pipermail/lasigma/attachments/20240502/78ed8842/attachment.html>