Streamline Your Workflow: Key-Based Authentication for File Transfers

Transferring files to and from the HPC cluster is a frequent task for many researchers. However, entering your password and performing Duo two-factor authentication for every single file transfer (SCP, SFTP, or Rsync) can be tedious and interrupts automated workflows.

Key-based authentication is the solution. By setting up an SSH key pair, you can securely transfer files and log in without typing your password or waiting for Duo prompts every time. This is essential for:

• Automation: Running scheduled scripts that need to transfer data without human intervention.
• Convenience: Drag-and-drop file transfers in tools like MobaXterm, WinSCP, or FileZilla without constant re-authentication.
• Speed: reducing the time spent on the "handshake" process during logins.

How to Set Up Key Authentication

The setup involves generating a "key pair" on your computer and placing the public half of that key onto the HPC cluster. Once done, the cluster recognizes your computer automatically.

Option 1: Windows (using PowerShell)

1. Open PowerShell as an Administrator.
2. Ensure the OpenSSH client is active and generating a key:

   Get-Service -Name ssh-agent | Set-Service -StartupType Automatic
   Start-Service ssh-agent
   ssh-keygen -t ed25519

   (Press Enter to accept default file locations. You may set a passphrase for extra security, or leave it empty for automated scripts.)
3. Add the new key to your local agent:

   ssh-add $HOME\.ssh\id_ed25519

4. Copy the key to the HPC Cluster:
   Run the following command, replacing <username> with your ITSO account and the cluster address (e.g., hpc4.ust.hk):

   Get-Content $HOME\.ssh\id_ed25519.pub | ssh <username>@hpc4.ust.hk 'mkdir -p ~/.ssh && chmod 700 ~/.ssh && pubkey=$(cat) && if ! grep -qF -- "$pubkey" ~/.ssh/authorized_keys 2>/dev/null; then echo "$pubkey" >> ~/.ssh/authorized_keys; fi && chmod 600 ~/.ssh/authorized_keys'

Option 2: macOS and Linux

1. Open your Terminal.
2. Generate a new key pair:

   ssh-keygen -t ed25519

   (Press Enter to accept defaults.)
3. Copy the key to the HPC cluster using the built-in copy utility:

   ssh-copy-id -i ~/.ssh/id_ed25519.pub <username>@hpc4.ust.hk

Testing Your File Transfer

Once the setup above is complete, you can test a file transfer. You should not be asked for a password.

Example (SCP):

scp local_file.txt <username>@hpc4.ust.hk:~/remote_folder/

For more detailed technical documentation and troubleshooting, please refer to the official guide:
Login to HPC Cluster Without Using Password — HPC Handbook