JupyterLab on HPC

Run JupyterLab on a compute node and access it from your local computer

Prerequisites

Either use module load or follow the instructions on the Software Setup page to configure your environment.

Start JupyterLab on a Compute Node

Download and submit the JupyterLab script:

  1. Download the script to your home directory:
curl -O https://wanggroup.org/scripts/jupyter_cumc_cloud.sh

  1. Modify the script as needed:

    • Adjust memory and wall time requirements
    • Add bash configurations before the jupyter-lab command (e.g., custom paths from your ~/.bashrc)

  2. Submit the job:

sbatch jupyter_cumc_cloud.sh -i <hpc_ip>

To request specific resources:

sbatch --constraint="cpu4mem32" --time=24:00:00 jupyter_cumc_cloud.sh -i <hpc_ip>

Access the Node via SSH Tunnel

After the job starts, check the job output file for connection info:

cat jupyter_<JOBID>.out

Replace <JOBID> with your actual job ID. You’ll see instruction like:

========================================
Jupyter Lab - Job 12345678
========================================
On your LOCAL machine, run:
  ssh -N -L 8234:compute-node:8234 username@hpc_ip

Then find the Jupyter URL with token in:
  cat jupyter_12345678.err | grep "lab?token"

Copy the full URL (including token) to your browser.
To stop: scancel 12345678
========================================

Windows MobaXterm info:
  Forwarded port: 8234
  Remote server:  compute-node
  Remote port:    8234
  SSH server:     hpc_ip
  SSH login:      username
  SSH port:       22

========================================

Session data: /home/username/jupyter-status/job_12345678

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!  If browser complains about secure connection,       !!!
!!!  use http:// instead of https://                     !!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Follow these instructions on your local machine to connect to JupterLab server.

Cost saving: Please remember to cancel your job if you are done with it.

Troubleshooting

Windows WSL: “jupyter-lab command not found”

If using Windows WSL with the default terminal, the error may occur because $HOME is not expanded correctly.

Change:

export PATH=$HOME/.pixi/bin:$PATH

To:

export PATH="/home/<username>/.pixi/bin:$PATH"

Replace <username> with your actual username.