lightcone-cli on NERSC (Perlmutter)¶

A practical guide for running lightcone-cli on Perlmutter. The CLI itself behaves the same as on a laptop — the wrinkles are in the filesystem layout (DVS-mounted home, Lustre scratch), the container runtime (podman-hpc), and SLURM submission. This page covers all three.

0. Agentic CLI¶

lightcone-cli is the execution layer of the lightcone project — it harnesses an agent-based CLI to follow the astra standard while building and running an analysis. The choice of agent is open: anything that can drive a project shell works. This guide uses Claude Code as the running example — substitute your preferred agent CLI throughout if you use a different one.

Installing Claude Code:

curl -fsSL https://claude.ai/install.sh | bash   # installs to ~/.local/bin/claude

Make sure ~/.local/bin is on your PATH, then verify and authenticate:

claude --version
claude                                           # first run prompts for login (claude.ai or API key)

Other install routes (npm, native package managers) are documented in the Claude Code installation docs.

1. Python¶

Like the generic Install page, we recommend uv for managing Python on Perlmutter — it's faster than pip and gives you a Python independent of NERSC's module system. NERSC doesn't ship it, but it installs into your home dir with a single curl:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv python install 3.12

This drops both uv and an isolated Python 3.12 under ~/.local/. Make sure ~/.local/bin is on your PATH.

module load python      # NERSC Python (3.11+); brings conda and pip onto PATH

conda create -n your-env-name python=3.11 -y
conda activate your-env-name

conda deactivate
mv ~/.conda/envs/your-env-name $SCRATCH/conda-envs/
ln -s $SCRATCH/conda-envs/your-env-name ~/.conda/envs/your-env-name

2. Install lightcone-cli¶

The package on PyPI is lightcone-cli; the command it provides is lc. The recommended install uses uv tool, which isolates lc in its own venv under ~/.local/share/uv/tools/ and exposes a wrapper at ~/.local/bin/lc:

uv tool install lightcone-cli

astra-tools is a transitive dependency — pulled in automatically.

module load python
python -m pip install --user lightcone-cli   # --user lands in ~/.local/bin/

conda activate your-env-name
python -m pip install lightcone-cli

From source (contributors only)¶

If you want to track the latest commits or contribute back, clone the repo and install editably. Most users should stick with PyPI.

cd ~/.lightcone                                # or wherever you keep clones
git clone https://github.com/LightconeResearch/lightcone-cli.git
uv pip install -e ./lightcone-cli              # or: pip install -e ./lightcone-cli

To hack on astra-tools itself (PyPI name astra-tools, GitHub repo ASTRA):

git clone https://github.com/LightconeResearch/ASTRA.git
uv pip install -e ./ASTRA

For development tooling (pytest, ruff, mypy):

uv pip install -e "./lightcone-cli[dev]"

Sanity check¶

which lc            # should resolve under ~/.local/bin/ or your active env
lc --version
lc --help

3. Initialize a new project¶

Scaffold a project directory and drop into it with the agent:

lc init your-analysis      # scaffolds a fresh project tree
cd your-analysis
claude                     # launch your agent CLI (Claude Code shown here)

4. Start your research¶

Once your agent CLI is open (Claude Code in this guide's examples), drive everything from there. The lc-* skills are how you tell the agent what to build:

/lc-new Please sample a standard Gaussian distribution using numpy.

/lc-from-code I have code that samples a standard Gaussian distribution using numpy at @../gaussian_sampling. Please create an analysis based on it.

After that, just keep talking to the agent in plain English about what you want to build next.

5. Running on compute nodes¶

Login nodes are shared and rate-limited — fine for lc init, lc status, and small lc build calls, but anything heavyweight belongs on a compute node.

Pre-flight: pin the container runtime and build images¶

Perlmutter compute nodes ship podman-hpc. Pin it once globally:

# ~/.lightcone/config.yaml
container:
  runtime: podman-hpc

Then, on a login node, build and migrate your project's images:

cd /path/to/your-analysis
lc build

lc build runs podman-hpc build followed by podman-hpc migrate, which copies the image into each compute node's local container cache. See Running on a Cluster → Pre-flight for the underlying mechanics.

Interactive runs (agent-driven)¶

The agent calls lc run for you whenever a recipe needs to materialize — you never call it directly. What you do control is where the agent is running: it inherits the shell environment you launched it from. To put the agent's recipes onto a compute node, simply launch it from inside a SLURM allocation:

salloc -A <your_project> -q interactive -C gpu --nodes=1 -t 00:30:00
# salloc drops you onto a compute node; from there:
cd /path/to/your-analysis
claude                                            # or whichever agent CLI you use

Now everything the agent triggers (lc run, scripts, etc.) executes on the allocated node.

Unattended batch runs (no agent in the loop)¶

For production sweeps where the recipes are already nailed down, you can submit lc run directly as a batch job — no agent CLI involved. See Running on a Cluster → A typical SLURM workflow for the generic template; on Perlmutter, the only addition is the -A / -q directives:

#!/bin/bash
#SBATCH -A <your_project>
#SBATCH -q regular
#SBATCH -C gpu
#SBATCH -N 4
#SBATCH -t 04:00:00

cd $SCRATCH/your-analysis

# Make `lc` available — pick the line that matches your install:
export PATH=$HOME/.local/bin:$PATH                # uv tool install (default)
# source ~/.conda/envs/your-env-name/bin/activate # conda env

lc run -j 16

Storage gotcha: Snakemake state must live on $SCRATCH¶

lc redirects state automatically when it detects Perlmutter, so this usually just works. To pin explicitly at project creation:

lc init your-analysis --scratch '$SCRATCH'        # kept verbatim, expanded at run time

Or, after the fact, edit <project>/.lightcone/lightcone.yaml:

scratch_root: $SCRATCH

Further reading¶

• NERSC interactive jobs — salloc patterns and reservation queues
• Perlmutter system overview — node types and partitions
• NERSC Python guide — module, conda, and pip layering

6. Common troubleshooting¶

7. Updating¶

uv tool upgrade lightcone-cli

pip install -U lightcone-cli astra-tools

cd ~/.lightcone/lightcone-cli
git pull
uv pip install -e .                       # only needed if pyproject.toml changed

8. Uninstalling¶

uv tool uninstall lightcone-cli

pip uninstall lightcone-cli
rm -rf ~/.lightcone/lightcone-cli         # only for source installs