16.2. Open MPI v5.0.x series
This file contains all the NEWS updates for the Open MPI v5.0.x series, in reverse chronological order.
16.2.1. Open MPI version 5.0.0rc7
- Date
13 May 2022
MPIR API has been removed
As was announced in summer 2017, Open MPI has removed support of MPIR-based tools beginning with the release of Open MPI v5.0.0.
The new PRRTE based runtime environment supports PMIx-tools API instead of the legacy MPIR API for debugging parallel jobs.
See for more information.
zlib is suggested for better user experience
PMIx will optionally use zlib to compress large data streams. This may result in faster startup times and smaller memory footprints (compared to not using compression). The Open MPI community recommends building zlib support with PMIx, regardless of whether you are using an externally-installed PMIx or the PMIx that is installed with Open MPI.
Caution
Open MPI no longer builds 3rd-party packages such as Libevent, HWLOC, PMIx, and PRRTE as MCA components and instead:
Relies on external libraries whenever possible, and
Builds the 3rd party libraries only if needed, and as independent libraries, rather than linked into the Open MPI core libraries.
Changes since rc6:
The PRRTE and OpenPMIx submodule pointers have been updated to bring in the following fixes:
Fixed a bug where
opal_show_help()output would not be aggregated and de-duplicated by default. This was a regression from the Open MPI v4.x series, and should now be fixed. Users can change the default by using the mca parameter
opal_base_help_aggregate.
Fixed a segmentation fault in the launcher when running with fault tolerance enabled.
Fixed issues when launching indirectly via
SLURM:
Fixed launch failures found in
rc6where the environment was not properly setup for launching.
Restored the use of
--cpu-bind=NONE.
Fixed a bug
--allow-run-as-rootwas not propagated to the backend
prtedaemons.
Changes were made to
mpirunto improve its detection of the backend launcher
prterun. This fixes most of the launch issues where
mpirunfailed to find
prterunin out-of-the-box RPM installs of Open MPI.
UCXone-sided MPI changes:
Added support for shared memory windows (
MPI_Win_allocate_shared()).
Various other updates and bug fixes.
Fixed a regression were the FORTRAN
OSHMEMwrapper compiler would fail to compile user applications.
OSHMEM: updated
shmem_calloc()to be standard-compliant regarding zero-byte inputs.
Fixed various memory leaks when running applications that use non-blocking collectives.
Fixed a segmentation fault in sparsely connected applications in
MPI_Finalize().
Issue a warning if
PMIxis unreachable and a
PMI1or
PMI2or
SLURMenvironment is detected before falling back to singleton mode launching. This will prevent confusion to end users running in these situtations, as
PMIsupport has been dropped from Open MPI v5.0.0.
MPI Sessions: Added support for
MPI_TAG_UB.
Fixed a build failure when compiling Open MPI with
PSM2(
configure --with-psm2=..).
Thanks to Sascha Hunold for the fix.
All other notable updates for v5.0.0:
Updated PMIx to the
v4.2branch - current hash:
8c39d8e.
Updated PRRTE to the
v2.1branch - current hash:
f75647a.
New Features:
ULFM Fault Tolerance support has been added. See the ULFM section
CUDAis now supported in the
ofiMTL.
mpirun option
--mca ompi_display_comm mpi_init/
mpi_finalizehas been added. This enables a communication protocol report: when
MPI_Initis invoked (using the
mpi_initvalue) and/or when
MPI_Finalizeis invoked (using the
mpi_finalizevalue).
The threading framework has been added to allow building OMPI with different threading libraries. It currently supports Argobots, Qthreads, and Pthreads. See the
--with-threadsoption in the
configurecommand. Thanks to Shintaro Iwasaki and Jan Ciesko for their contributions to this effort.
New Thread Local Storage API: Removes global visibility of TLS structures and allows for dynamic TLS handling.
Added load-linked, store-conditional atomics support for AArch64.
Added atomicity support to the
ompiocomponent.
Added support for MPI minimum alignment key to the one-sided
RDMAcomponent.
Add ability to detect patched memory to
memory_patcher. Thanks to Rich Welch for the contribution.
MPI-4.0 updates and additions:
Support for
MPI Sesisonshas been added.
Added partitioned communication using persistent sends and persistent receives.
Added persistent collectives to the
MPI_namespace (they were previously available via the
MPIX_prefix).
Added
MPI_Isendrecv()and its variants.
Added support for
MPI_Comm_idup_with_info().
Added support for
MPI_Info_get_string().
Added support for
initial_error_handlerand the
ERRORS_ABORTinfrastructure.
Added error handling for “unbound” errors to
MPI_COMM_SELF.
Made
MPI_Comm_get_info(),
MPI_File_get_info(), and
MPI_Win_get_info()compliant to the standard.
Droped unknown/ignored info keys on communicators, files, and windows.
Transport updates and improvements
One-sided Communication:
Many MPI one-sided and RDMA emulation fixes for the
tcpBTL.
This patch series fixs many issues when running with
--mca osc rdma --mca btl tcp, IE - TCP support for one sided MPI calls.
Many MPI one-sided fixes for the
ucxBTL.
Added support for
acc_single_intrinsicto the one-sided
ucxcomponent.
Removed the legacy
pt2ptone-sided component. Users should use the
rdmaone-sided component instead with the
tcpBTL and/or other BTLs to use MPI one sided-calls via TCP transport.
Updated the
tcpBTL to use graph solving for global interface matching between peers in order to improve
MPI_Init()wireup performance.
Changes to the BTL
OFIcomponent to better support the HPE SS11 network.
Shared Memory:
The legacy
sm(shared memory) BTL has been removed. The next-generation shared memory BTL
vaderreplaces it, and has been renamed to be
sm(
vaderwill still work as an alias).
Update the new
smBTL to not use Linux Cross Memory Attach (CMA) in user namespaces.
Fixed a crash when using the new
smBTL when compiled with Linux Cross Memory Attach (
XPMEM). Thanks to George Katevenis for reporting this issue.
Updated the
-mca pmloption to only accept one pml, not a list.
Deprecations and removals:
ORTE, the underlying OMPI launcher has been removed, and replaced with The PMIx Reference RunTime Environment (
PRTE).
PMI support has been removed from Open MPI; now only PMIx is supported. Thanks to Zach Osman for removing config/opal_check_pmi.m4.
Removed transports PML
yalla,
mxm, MTL
psm, and
ikritcomponents. These transports are no longer supported, and are replaced with
UCX.
Removed all vestiges of Checkpoint Restart (C/R) support.
32 bit atomics are now only supported via C11 compliant compilers.
Explicitly disable support for GNU gcc < v4.8.1 (note: the default gcc compiler that is included in RHEL 7 is v4.8.5).
Various atomics support removed: S390/s390x, Sparc v9, ARMv4 and ARMv5 with CMA support.
The MPI C++ bindings have been removed.
The mpirun options
--amand
--amcaoptions have been deprecated.
ompi/contrib: Removed
libompitrace. This library was incomplete and unmaintained. If needed, it is available in the v4/v4.1 series.
HWLOC updates:
Open MPI now requires HWLOC v1.11.0 or later.
The internal HWLOC shipped with OMPI has been updated to v2.7.1.
Enable –enable-plugins when appropriate.
Documentation updates and improvements:
Open MPI now uses readthedocs.io for all documentation.
Converted man pages to markdown. Thanks to Fangcong Yin for their contribution to this effort.
Various
README.mdand
HACKING.mdfixes - thanks to: Yixin Zhang, Samuel Cho, Robert Langfield, Alex Ross, Sophia Fang, mitchelltopaloglu, Evstrife, Hao Tong and Lachlan Bell for their contributions.
Various CUDA documentation fixes. Thanks to Simon Byrne for finding and fixing these typos.
Build updates and fixes:
Various changes and cleanup to fix, and better support the static building of Open MPI.
Change the default component build behavior to prefer building components as part of the core Open MPI library instead of individual DSOs. Currently, this means the Open SHMEM layer will only build if the UCX library is found.
autogen.plnow supports a
-joption to run multi-threaded. Users can also use the environment variable
AUTOMAKE_JOBS.
Updated
autogen.plto support macOS Big Sur. Thanks to @fxcoudert for reporting the issue.
Fixed bug where
autogen.plwould not ignore all excluded components when using the
--excludeoption.
Fixed a bug the
-roption of
buildrpm.shwhich would result in an rpm build failure. Thanks to John K. McIver III for reporting and fixing.
Removed the
C++compiler requirement to build Open MPI.
Updates to improve the handling of the compiler version string in the build system. This fixes a compiler error with clang and armclang.
Added OpenPMIx binaries to the build, including
pmix_info. Thanks to Mamzi Bayatpour for their contribution to this effort.
Open MPI now links to Libevent using
-levent_coreand
-levent_pthreadinstead of
-levent.
Added support for setting the wrapper C compiler. This adds a new option:
--with-wrapper-cc=to the
configurecommand.
Fixed compilation errors when running on IME file systems due to a missing header inclusion. Thanks to Sylvain Didelot for finding and fixing this issue.
Add support for GNU Autoconf v2.7.x.
Other updates and bug fixes:
Updated Open MPI to use
ROMIOv3.4.1.
Fixed Fortran-8-byte-INTEGER vs. C-4-byte-int issue in the
mpi_f08MPI Fortran bindings module. Thanks to @ahaichen for reporting the bug.
Add missing
MPI_Statusconversion subroutines:
MPI_Status_c2f08(),
MPI_Status_f082c(),
MPI_Status_f082f(),
MPI_Status_f2f08()and the
PMPI_*related subroutines.
Fixed Fortran keyword issue when compiling
oshmem_info. Thanks to Pak Lui for finding and fixing the bug.
Added check for Fortran
ISO_FORTRAN_ENV:REAL16. Thanks to Jeff Hammond for reporting this issue.
Fixed Fortran preprocessor issue with CPPFLAGS. Thanks to Jeff Hammond for reporting this issue.
MPI module: added the mpi_f08 TYPE(MPI_*) types for Fortran. Thanks to George Katevenis for the report and their contribution to the patch.
Fixed a typo in an error string when showing the stackframe. Thanks to Naribayashi Akira for finding and fixing the bug.
Fixed output error strings and some comments in the Open MPI code base. Thanks to Julien Emmanuel for finding and fixing these issues.
The
uctBTL transport now supports
UCXv1.9 and higher. There is no longer a maximum supported version.
Updated the UCT BTL defaults to allow Mellanox HCAs (
mlx4_0, and
mlx5_0) for compatibility with the one-sided
rdmacomponent.
Fixed a crash during CUDA initialization. Thanks to Yaz Saito for finding and fixing the bug.
Singleton
MPI_Comm_spawn()support has been fixed.
PowerPC atomics: Force usage of ppc assembly by default.
Various datatype bugfixes and performance improvements.
Various pack/unpack bugfixes and performance improvements.
Various OSHMEM bugfixes and performance improvements.
New algorithm for Allgather and Allgatherv has been added, based on the paper “Sparbit: a new logarithmic-cost and data locality-aware MPI Allgather algorithm”. Default algorithm selection rules are un-changed, to use these algorithms add:
--mca coll_tuned_allgather_algorithm sparbitand/or
--mca coll_tuned_allgatherv_algorithm sparbitto your
mpiruncommand. Thanks to: Wilton Jaciel Loch, and Guilherme Koslovski for their contribution.
Updated the usage of .gitmodules to use relative paths from absolute paths. This allows the submodule cloning to use the same protocol as OMPI cloning. Thanks to Felix Uhl for the contribution.