content_image/1009769.jpg
## Simulating different SMT modes 

The example cluster that is shown in Figure 7-4 contains two nodes (Power AC922 systems), each of which has two sockets. One nonuniform memory access (NUMA) node corresponds to one socket and has 20 physical POWER9 cores inside it. The systems are configured with simultaneous multithreading (SMT)-4, which means that a physical core can split a job between four logical CPUs (threads). 

<img src='content_image/1009769.jpg'>

IBM Parallel Environment MPI provides run options or environment variables that help to simulate runs with different SMT modes. The following cases describe two ways where a job uses only one logical CPU from each POWER9 core (SMT-1 mode) and fully uses all 176 logical CPUs by using the code in Example 7-15 on page 200 with IBM Parallel ESSL calls: 

GLYPH<SM590000> To configure the job to use only one logical CPU per POWER9 core, run the following command: 

MP_TASK_AFFINITY=core:1 MP_RESD=poe MP_PROCS=44 ./test_pdgemm 

MP_TASK_AFFINITY=core:1 directs the IBM Parallel Environment MPI that each MPI task can use only one POWER9 core. Overall, the full cluster has 88 POWER9 cores (each node contains 44 cores), so a maximum of 88 MPI ( MP_PROCS=88 ) tasks can be used for such run. 

Another possible solution to simulate SMT-1 mode is to take 20 MPI tasks with two POWER8 cores for each of them by running the following command: 

MP_TASK_AFFINITY=core:2 MP_RESD=poe MP_PROCS=44 ./test_pdgemm 

GLYPH<SM590000> The following command shows how to use the cluster and all 176 logical CPUs per node: 

MP_TASK_AFFINITY=cpu:16 MP_RESD=poe MP_PROCS=22 ./test_pdgemm 

MP_TASK_AFFINITY=cpu:16 means when you use the IBM Parallel Environment MPI that each MPI task can use only 16 logical CPUs. Each node in the cluster contains 176 logical CPUs, so 11 MPI tasks per node and 22 for the overall cluster can be used in this run. 

