Restarts

restart_options, passed to Simulation (see Simulation), controls two things: writing checkpoint files during a run, and resuming a later run from one of them.

Writing checkpoints

import numpy as np
import pyphare.pharein as ph

ph.Simulation(
    # ...
    restart_options={
        "dir": "restarts",
        "mode": "conserve",
        "timestamps": np.array([1.0, 2.0, 3.0]),
    },
)
  • mode is required: "conserve" keeps any existing checkpoint files; "overwrite" replaces them.

  • timestamps are simulation times, and must line up with the simulation’s time_step.

  • elapsed_timestamps writes a checkpoint every time that much wall-clock time has passed instead (in seconds, or as datetime.timedelta) - handy on a cluster with a wall-time limit, since it doesn’t require knowing in advance which simulation time will be reached.

  • keep_last=N automatically deletes older checkpoint directories, keeping only the N most recent - useful to bound disk usage on a long run that checkpoints often.

Tip

On supercomputers, jobs are typically killed after a fixed wall time (e.g. 24h), but you can rarely predict which simulation time the run will have reached by then. Prefer elapsed_timestamps over timestamps in that case, e.g. at 10h and 23h, so a checkpoint is guaranteed to be dumped with enough time left before the job is killed - rather than picking simulation timestamps that risk falling right after the job dies.

Resuming from a checkpoint

ph.Simulation(
    # ...
    restart_options={
        "dir": "restarts",
        "mode": "conserve",
        "restart_time": "auto",  # or an explicit time, e.g. 2.0
    },
)

restart_time="auto" picks the latest available checkpoint in dir. The restarted run must describe the same simulation as the one that produced the checkpoint; PHARE checks that cells, dl and max_nbr_levels match.