Load balancing

On a parallel run, work (particles and cells) is split across MPI ranks by patch. As the simulation evolves - particles move, AMR levels change shape - that split can become uneven. LoadBalancer controls how and when PHARE corrects for that. It is optional: without one, load balancing still runs with the defaults below.

class pyphare.pharein.LoadBalancer(mode: str = <factory>, tol: float = <factory>, on_init: bool = <factory>, auto: bool = <factory>, next_rebalance_backoff_multiplier: int = <factory>, next_rebalance: int = <factory>, max_next_rebalance: int = <factory>, every: int = <factory>, _register: bool = <factory>)[source]

LoadBalancer controls how simulation work (particles and cells) is redistributed across MPI ranks as the simulation runs, to keep ranks evenly loaded on an evolving, adaptively-refined domain. Optional - with refinement=”tagging” load balancing is always on, and if none is created the defaults below are used. It is not supported with refinement=”boxes”.

The proactive rebalancing of L0 described below only happens when L0 is hybrid. An MHD L0 (max_mhd_level > 0) is never proactively rebalanced.

Can be created any time after Simulation, at most once per simulation.

mode shapes how load is measured wherever patches get redistributed across ranks, at any level. tol does too, but is also reused as the threshold for a separate, proactive rebalancing of the coarsest level (L0) between regrids. auto, every and on_init control only that L0 proactive rebalancing - finer levels are redistributed across ranks only as part of their own AMR regrid, on their own cadence, never on this one.

Usage example:

import pyphare.pharein as ph

ph.LoadBalancer(mode="nppc", tol=0.05, every=200)

Parameters:

  • mode (str), {“nppc” (default), “homogeneous”}, how load is assessed per rank: “nppc” counts particles per rank, “homogeneous” counts cells per rank. Applies at every level, whenever that level’s patches are redistributed across ranks.

  • tol (float), default=0.05, acceptable relative imbalance between ranks. Used both hierarchy-wide (only used with mode=”nppc”) and as the threshold that triggers a proactive L0 rebalance between regrids.

  • on_init (bool), default=True, whether L0 is checked/rebalanced right after the simulation is initialized.

  • auto (bool), default=True unless every is given, whether L0’s proactive rebalancing cadence is chosen automatically (starting at next_rebalance steps, backing off by next_rebalance_backoff_multiplier up to max_next_rebalance) rather than at a fixed cadence. Mutually exclusive with every.

  • every (int), if set, proactively rebalance L0 every every steps instead of automatically. Mutually exclusive with auto.

import pyphare.pharein as ph

ph.LoadBalancer(mode="nppc", tol=0.05, every=200)

Use mode="nppc" (particles per rank) for a Hybrid simulation - it’s the quantity that actually drives compute cost there. mode="homogeneous" (cells per rank) is closer to what an MHD-only run needs, since there are no particles.

Two different scopes

Not all parameters act on the same part of the hierarchy:

  • mode shapes how load is measured everywhere - at any level, whenever that level’s patches are redistributed across ranks.

  • tol does too, but is also reused as the threshold for a second, separate mechanism: proactively rebalancing the coarsest level (L0) between regrids.

  • auto, every and on_init control only that second mechanism. They have no effect on finer levels - those are only ever redistributed across ranks as part of their own AMR regrid, on their own cadence.

In practice, this makes auto/every/on_init mostly relevant to Hybrid simulations: L0 covers the whole domain and typically holds most of the particles, so it is the level whose imbalance matters most between regrids.