Z/Architecture
z/Architecture, initially and briefly called ESA Modal Extensions (ESAME), is IBM's 64-bit complex instruction set computer (CISC) instruction set architecture, implemented by its mainframe computers. IBM introduced its first z/Architecture-based system, the z900, in late 2000. Later z/Architecture systems include the IBM z800, z990, z890, System z9, System z10, zEnterprise 196, zEnterprise 114, zEC12, zBC12, z13, z14 and z15.

z/Architecture retains backward compatibility with previous 32-bit-data/31-bit-addressing architecture ESA/390 and its predecessors all the way back to the 32-bit-data/24-bit-addressing System/360. The IBM z13 is the last z Systems server to support running an operating system in ESA/390 architecture mode. However, all 24-bit and 31-bit problem-state application programs originally written to run on the ESA/390 architecture will be unaffected by this change.

Each z/OS address space, called a 64-bit address space, is 16 exabytes in size.

Code (or mixed) spaces
Most operating systems for the z/Architecture, including z/OS, generally restrict code execution to the first 2 GB (31 address bits, or 231 addressable bytes) of each virtual address space for reasons of efficiency and compatibility rather than because of architectural limits. The z/OS implementation of the Java programming language is an exception. The z/OS virtual memory implementation supports multiple 2 GB address spaces, permitting more than 2 GB of concurrently resident program code. The 64-bit version of Linux on IBM Z allows code to execute within 64-bit address ranges.

Data-only spaces
For programmers who need to store large amounts of data, the 64-bit address space usually suffices.

Dataspaces and hiperspaces
Applications that need more than a 16 exabyte data address space can employ extended addressability techniques, using additional address spaces or data-only spaces. The data-only spaces that are available for user programs are called:
 dataspaces (sometimes referred to as "data spaces") and 
 hiperspaces (High performance space).

These spaces are similar in that both are areas of virtual storage that a program can create, and can be up to 2 gigabytes. Unlike an address space, a dataspace or hiperspace contains only user data; it does not contain system control blocks or common areas. Program code cannot run in a dataspace or a hiperspace.

A dataspace differs from a hiperspace in that dataspaces are byte-addressable, whereas hiperspaces are page-addressable.

IBM mainframe expanded storage
Traditionally IBM Mainframe memory has been byte-addressable. This kind of memory is termed "Central Storage". IBM Mainframe processors through much of the 1980s and 1990s supported another kind of memory: Expanded Storage.

Expanded Storage is 4KB-page addressable. When an application wants to access data in Expanded Storage it must first be moved into Central Storage. Similarly, data movement from Central Storage to Expanded Storage is done in multiples of 4KB pages. Initially page movement was performed using relatively expensive instructions, by paging subsystem code.

The overhead of moving single and groups of pages between Central and Expanded Storage was reduced with the introduction 
of the MVPG (Move Page) instruction and the ADMF (Asynchronous Data Mover Facility) capability.

The MVPG instruction and ADMF are explicitly invoked—generally by middleware in z/OS or z/VM (and ACP?)—to access data in expanded storage. Some uses are namely:
 MVPG is used by VSAM Local Shared Resources (LSR) buffer pool management to access buffers in a hiperspace in Expanded Storage.
 Both MVPG and ADMF are used by DB2 to access hiperpools. Hiperpools are portions of a buffer pool located in a hiperspace.
 VM Minidisk Caching.

Until the mid-1990s Central and Expanded Storage were physically different areas of memory on the processor. Since the mid-1990s Central and Expanded Storage were merely assignment choices for the underlying processor memory.
These choices were made based on specific expected uses:
For example, Expanded Storage is required for the Hiperbatch function (which uses the MVPG instruction to access its hiperspaces).

In addition to the hiperspace and paging cases mentioned above there are other uses of expanded storage, including:
 Virtual I/O (VIO) to Expanded Storage which stored temporary data sets in simulated devices in Expanded Storage. (This function has been replaced by VIO in Central Storage.)
 VM Minidisk Caching.

z/OS removed the support for Expanded Storage. All memory in z/OS is now Central Storage. z/VM 6.4 fulfills Statement of Direction to drop support for all use of Expanded Storage.

MVPG and ADMF

MVPG
IBM described MVPG as "moves a single page and the central processor cannot execute any other instructions until the page move is completed."

The MVPG mainframe instruction (MoVe PaGe, opcode X'B254') has been compared to the MVCL (MoVe Character Long) instruction, both of which can move more than 256 bytes within main memory using a single instruction. These instructions do not comply with definitions for atomicity, although they can be used as a single instruction within documented timing and non-overlap restrictions.

The need to move more than 256 bytes within main memory had historically been addressed with software (MVC loops), MVCL, which was introduced with the 1970 announcement of the System/370, and MVPG, patented and announced by IBM in 1989, each have advantages.

ADMF
ADMF (Asynchronous Data Mover Facility), which was introduced in 1992, goes beyond the capabilities of the MVPG (Move Page) instruction, which is limited to a single page, and can move groups of pages between Central and Expanded Storage.

A macro instruction named IOSADMF, which has been described as an API that avoids "direct, low-level use of ADMF," can be used to read or write data to or from a hiperspace. Hiperspaces are created using DSPSERV CREATE.

To provide reentrancy, IOSADMF is used together with a "List form" and "Execute form."

z/Architecture operating systems
The z/VSE Version 4, z/TPF Version 1 and z/VM Version 5 operating systems, and presumably their successors, require z/Architecture.

z/Architecture supports running multiple concurrent operating systems and applications even if they use different address sizes. This allows software developers to choose the address size that is most advantageous for their applications and data structures.

Platform Solutions Inc. (PSI) previously marketed Itanium-based servers which were compatible with z/Architecture. IBM bought PSI in July 2008, and the PSI systems are no longer available. FLEX-ES, zPDT and the Hercules emulator also implement z/Architecture. Hitachi mainframes running newer releases of the VOS3 operating system implement ESA/390 plus Hitachi-unique CPU instructions, including a few 64-bit instructions. While Hitachi was likely inspired by z/Architecture, and formally collaborated with IBM on the z900-G2/z800 CPUs introduced in 2002, Hitachi's machines are not z/Architecture-compatible.

On July 7, 2009, IBM on occasion of announcing a new version of one of its operating systems implicitly stated that Architecture Level Set 4 (ALS 4) exists, and is implemented on the System z10 and subsequent machines. The ALS 4 is also specified in LOADxx as ARCHLVL 3, whereas the earlier z900, z800, z990, z890, System z9 specified ARCHLVL 2. Earlier announcements of System z10 simply specified that it implements z/Architecture with some additions: 50+ new machine instructions, 1 MB page frames, and hardware decimal floating point unit (HDFU).

Notes

References

Further reading
 Preshing on Programming - Atomic vs. Non-Atomic Operations
 Principles of Computer Design - Atomicity

IBM mainframe technology
Instruction set architectures
Computer-related introductions in 2000
mainframe expanded storage
64-bit computers