A High Performance Computing (HPC) cluster, is a collection of (large) computing resources, like Processors (CPUs), Graphics processors (GPUs), Memory and Storage, that are shared among a group of users. Using multiple computers as such makes it possible to perform lengthy and resource-intense computations beyond the capabilities of a single computer, and is especially handy for modern scientific computing applications where datasets are typically large in size, models are big in parameters’ size and complexity, and computations need specialized hardware (like GPUs and FPGAs).
What is DAIC?
The Delft AI Cluster (DAIC), formerly known as INSY-HPC or just plainly HPC, is a TU Delft High Performance Computing (HPC) cluster consisting of Linux compute nodes (ie servers) with a lot of processing power and memory for running large, long or GPU-enabled jobs.
From a CS only cluster in 2015, DAIC has grown in time to serve researchers across many TU Delft departments but maintained the needs of CS and AI in each expansion phase. Today, DAIC nodes are organized as partitions that correspond to the groups contributing these resources. (See Contributing departments and TU Delft clusters comparison).
DAIC partitions and access/usage best practices
1 - Contributors and funding
The Delft AI Cluster (DAIC) - formerly known as INSY-HPC or just plainly HPC- was initiated within the INSY department in 2015. Later, resources were joined with ST, collectively called CS@Delft, and with other departments across faculties in subsequent expansion cycles.
Joining DAIC?
If you are interested in joining DAIC as a contributor, please contact us via this TopDesk DAIC Contact Us form.
Contributing departments
The cluster is available (only) to users from participating departments, and access can be arranged through the department’s contact persons (see Access and accounts).
Table 1: Current partitions within DAIC and contributing TU Delft departments/faculties.
To help demonstrate the impact of DAIC, we ask that you both cite and acknowledge DAIC in your scientific publications. Please use the following formats:
@misc{DAIC,author={{Delft AI Cluster (DAIC)}},title={The Delft AI Cluster (DAIC), RRID:SCR_025091},year={2024},doi={10.4233/rrid:scr_025091},url={https://doc.daic.tudelft.nl/}}
TY - DATA
T1 - The Delft AI Cluster (DAIC), RRID:SCR_025091
UR - https://doi.org/10.4233/rrid:scr_025091
PB - TU Delft
PY - 2024
Acknowledgement text
Research reported in this work was partially or completely facilitated by computational resources and support of the Delft AI Cluster (DAIC) at TU Delft (RRID: SCR_025091), but remains the sole responsibility of the authors, not the DAIC team.
Scientific impact in numbers
Since 2015, DAIC has facilitated more than 2000 scientific outputs from the various DAIC-participating departments:
Article
Conference/Meeting contribution
Book/Book chapter/Book editing
Dissertation (TU Delft)
Abstract
Other
Editorial
Patent
Grand Total
Grand Total
1067
854
123
99
69
32
29
8
2281
These outputs span a wide range of application areas, with titles reflecting an emphasis on data analysis and machine learning:
Wordcloud of the most common words in titles of Scientific outputs produced via DAIC
Reference
The table and wordcloud provided here are based on retrospective retrieval of all DAIC users’ scientific outputs between 2015-2023 from TU Delft’s Pure database.
The data has been generated by the Strategic Development – Data Insights team.
Publications using DAIC
Note
he compilation of the following list is done retrospectively by the Data Insights team and/or is based on self-reporting by individual researchers. As a result, it may not be exhaustive nor complete. If your publication is missing, please let us know by posting it to the ScientificOutput MatterMost channel.
3 - TU Delft clusters comparison
Cluster comparison
TU Delft clusters
DAIC is one of several clusters accessible to TU Delft CS researchers (and their collaborators). The table below gives a comparison between these in terms of use case, eligible users, and other characteristics.
DAIC
DelftBlue
DAS
Primary use cases
Research, especially in AI
Research & Education
Distributed systems research, streaming applications, edge and fog computing, in-network processing, and complex security and trust policies, Machine learning research, ...
SURF, the collaborative organization for IT in Dutch education and research, has installed and is currently operating the Dutch National supercomputer, Snellius, which houses 144 40GB A100 GPUs as of Q3 2021 (36 gcn nodes x 4 A100 GPUs/node = 144 A100 GPUs total) with other specs detailed in the Snellius hardware and file systems wiki.
SURF also operates other clusters like Spider for processing large structured data sets, and ODISSEI Secure Supercomputer (OSSC) for large-scale analyses of highly-sensitive data. For an overview of SURF clusters, see the SURF wiki.
TU Delft researchers in TBM and CITG already have direct and easy access to the compute power and data services of SURF, while members of other faculties need to apply for access as detailed in SURF’s guide to Apply for access to compute services.
TU Delft cloud resources
For both education and research activities, TU Delft has established the Cloud4Research program. Cloud4Research aims to facilite the use of public cloud resources, primarily Amazon AWS. At the administrative level, Cloud4Research provides AWS accounts with an initial budget. Subsequent billing can be incurred via a project code, instead of a personal credit card. At the technical level, the ICT innovation teams provides intake meetings to facilitate getting started. Please refer to the Policies and FAQ pages for more details.
3D Geoinformation
Aerospace Structures and Materials
Cognitive Robotics
