Storage

• 1: Storage
• 2: Data Transfer

1 - Storage

Storage overview

DAIC provides access to multiple storage areas. Understanding their purposes and limitations is essential for effective work on the cluster.

Cluster home directory

Your cluster home (/trinity/home/<NetID>) has a very small quota of approximately 5 MB. This is intentionally limited and meant only for:

• Shell configuration files (.bashrc, .bash_profile)
• SSH keys and config (.ssh/)
• Small application configs

Check your cluster home quota:

quota -s
Disk quotas for user <NetID> (uid XXXXXX):
     Filesystem   space   quota   limit   grace   files   quota   limit   grace
control:/trinity/home
                   528K   4096K   5120K              21    4096    5120

TU Delft network storage

On first login, symlinks are created in your home directory pointing to TU Delft network storage:

Kerberos authentication

TU Delft network storage requires a valid Kerberos ticket. Without it, you will get “Permission denied” or “Stale file handle” errors when accessing linuxhome, windowshome, project, group, or bulk storage.

When logging in via SSH with a password, a Kerberos ticket is created automatically.

When logging in via SSH with a public key or through OpenOndemand, you must manually obtain a ticket:

kinit

Enter your NetID password when prompted.

Check your current ticket status:

klist

Example output with a valid ticket:

Ticket cache: KCM:656519
Default principal: <NetID>@TUDELFT.NET

Valid starting     Expires            Service principal
03/23/26 11:05:12  03/23/26 21:05:12  krbtgt/TUDELFT.NET@TUDELFT.NET
        renew until 03/30/26 12:05:03

• ~/linuxhome - Your Linux home on TU Delft storage
• ~/windowshome - Your Windows home on TU Delft storage

These are accessible from DAIC, your TU Delft workstation, and via webdata.

Project storage

Project storage is for research data accessible only to project members. Request project storage via the Self-Service Portal.

Access path: /tudelft.net/staff-umbrella/<project-name>

Group storage

Group storage is shared with your department or group. Not suitable for confidential data.

Access paths:

• /tudelft.net/staff-groups/<faculty>/<department>/<group>
• /tudelft.net/staff-bulk/<faculty>/<department>/<group>

Local temporary storage

Each compute node has local /tmp storage for temporary files during job execution.

• No quota, but shared with other users
• Files not accessed for 10 days are automatically deleted
• Not accessible from other nodes
• No backup

Use local storage for intermediate files that do not need to persist after job completion.

Checking disk usage

Check usage of a directory:

du -hs /tudelft.net/staff-umbrella/<project>
37G     /tudelft.net/staff-umbrella/<project>

Check available space:

df -h /tudelft.net/staff-umbrella/<project>
Filesystem      Size  Used Avail Use% Mounted on
...             1.0T   38G  987G   4% /tudelft.net/staff-umbrella/<project>

2 - Data Transfer

Overview

There are several ways to transfer data to and from DAIC:

Rclone

Rclone supports transfers to/from many cloud providers and remote systems. Run rclone on your local machine to transfer data to DAIC.

Install rclone locally:

See rclone install guide for your operating system.

Configure DAIC as an SFTP remote (one-time setup):

rclone config
# Choose: n (new remote)
# Name: daic
# Type: sftp
# Host: daic01.hpc.tudelft.nl
# User: <NetID>
# Use SSH key authentication

Copy from local to DAIC:

rclone copy /local/data/ daic:/tudelft.net/staff-umbrella/<project>/

Sync local directory to DAIC:

rclone sync /local/data/ daic:/tudelft.net/staff-umbrella/<project>/data/

See rclone documentation for more options and cloud backends.

Git clone

Clone repositories directly on DAIC:

git clone git@gitlab.tudelft.nl:your-group/your-repo.git

Rsync

Rsync is efficient for transferring large directories and synchronizing changes.

From local to DAIC:

rsync -avz --progress /local/path/ <NetID>@daic01.hpc.tudelft.nl:/tudelft.net/staff-umbrella/<project>/

From DAIC to local:

rsync -avz --progress <NetID>@daic01.hpc.tudelft.nl:/tudelft.net/staff-umbrella/<project>/ /local/path/

Common options:

• -a archive mode (preserves permissions, timestamps)
• -v verbose output
• -z compress during transfer
• --progress show transfer progress
• --dry-run test without transferring

SCP

For simple one-time file transfers:

Copy file to DAIC:

scp /local/file.tar.gz <NetID>@daic01.hpc.tudelft.nl:/tudelft.net/staff-umbrella/<project>/

Copy file from DAIC:

scp <NetID>@daic01.hpc.tudelft.nl:/tudelft.net/staff-umbrella/<project>/file.tar.gz /local/path/

Copy directory:

scp -r /local/directory/ <NetID>@daic01.hpc.tudelft.nl:/tudelft.net/staff-umbrella/<project>/

SFTP to staff-umbrella

You can transfer data directly to project storage without going through DAIC.

Using SFTP

Connect to sftp.tudelft.nl with your NetID credentials using the command line or clients like FileZilla, WinSCP, or Cyberduck.

Host: sftp.tudelft.nl
Username: <NetID>
Port: 22

Command line example:

sftp sftp.tudelft.nl
sftp> cd staff-umbrella/<project>
sftp> put localfile.txt
sftp> get remotefile.txt
sftp> put -r localfolder/
sftp> get -r remotefolder/
sftp> bye

Navigate to /staff-umbrella/<project>/ to access your project storage.