2.   Design

• Every component has an associated pelist. On mosaics, this is every tile on every component.
• The pelist is broken into non-overlapping lists: a comp_pelist which runs the model and an io_pelist, a set of PEs that only do I/O.
• When a domain decomposition is defined (call mpp_define_domains . . . ) on pelist, it actually creates two domain decompositions, domain and io_domain. (io_domain becomes part of the domain2D data structure. domain is defined on pelist (which, recall, is the union of comp_pelist and io_pelist) but the domain is actually spanned by comp_pelist. The domains defined on the io_pelist have null computational and data domains. (This is the tricky bit: will the data flow in all our models work if some PEs go through it with null domains?)

  Internally, we define a second decomposition of the same global domain, in io_domain. Schematically, the code running on 16 PEs looks like:

  (1)

  Within mpp_define_domains, this defines 2 decompositions:

  (2)

• The component executes on pelist: the PEs in io_pelist will skip all the computational parts, since they have null domains. (What will happen if there’s an mpp_sync?)
• When I/O occurs (call mpp_write(...,f,domain,...), domain is checked to see if an io_domain has been defined; if so we redistribute the data onto the io_domain. This will be a non-blocking call for the PEs on comp_pelist. which can move on to do more computation. The PEs on io_pelist do a blocked receive and then issue actual I/O calls. Schematically, the code flow looks like Figure 1.

  (3)

This, in essence, is an asynchronous I/O package. Some PEs do computation and do non-blocking sends of the data to I/O PEs. The I/O PEs flow through the computational code without executing any loops, but block at every call to mpp_write, to receive data and perform I/O.

All the code will be in mpp_io, mpp_domains and below. . . higher-level code, including fms_io and diag_manager, and model code, remains untouched (except for the domain setup).

balajiprinceton.edu