Cyberinfrastructure Technology Integration

Clemson Computing and Information Technology (CCIT) provides research cyberinfrastructure resources and advanced research computing capabilities through its Cyberinfrastructure Technology Integration (CITI) group.

Clemson Research Cyberinfrastructure

Palmetto Cluster

Palmetto Cluster is a local high performance computing environment available to all Clemson students/faculty and staff as a dedicated research environment.

Overview

  • available to all Clemson students/faculty/staff for free
  • 2021 compute nodes, 23072 cores
  • heterogeneous configuration with various types of nodes (different CPU, memory, network, disk space)
  • 386 nodes are equipped with NVIDIA Tesla GPUs: 280 nodes with NVIDIA K20 GPUs (2 per node), 106 nodes with NVIDIA K40 GPUs (2 per node)
  • 4 nodes with Intel Phi co-processors (2 per node)
  • 6 large memory nodes (5 with 505GB, 1 with 2TB), 262 nodes with 128GB of memory
  • 100GB of personal space (backed up daily for 42 days)
  • Myrinet, 10Gbps Ethernet, Infiniband networks
  • global and local scratch spaces for temporary files (no quota per user)
  • maximum run time for a single task limited to 72 hours on Infiniband part and 168 hours on Myrinet part
  • ranked 4th among the public academic institutions in the US on Top500 list (155 on Top500) with performance of 814.4 TFlops (17,372 cores from Infiniband part of Palmetto)

More about Palmetto
New account on Palmetto
Reservation request

Support

HPC support is provided by Advanced Computing and Research Methods group

Condominium model

Palmetto cluster operates in a condominium model which allows faculty to invest in the cluster. Investments into Palmetto are based on purchases of compute nodes. By purchasing a compute node faculty get priority to use an equivalent hardware across whole Palmetto cluster. All not used compute cycles are made available for general Clemson users. Owners may preempt other users making the hardware they purchased immediately available. Purchased nodes are available to faculty for a period of 4 years, after that the priority to use them expires.

Being an owner allows users to

  • have immediate access to the amount they have purchased by preempting other users
  • have a dedicated group on Palmetto cluster
  • invite external collaborators (not associated with Clemson) to use their purchased resources
  • have extended maximum time for a single task up to 336 hours (14 days)

For more information about condominium model and purchasing Palmetto nodes, including Palmetto nodes on grants please contact Jeronica Williams jeronic@clemson.edu or Marcin Ziolkowski zziolko@clemson.edu.

Owners guide for Palmetto cluster

Temporary storage

Palmetto includes several file systems designed for storing temporary files

  1. Local disk on compute nodes
  2. Following global scratch systems
File system Directory Capacity Features
OrangeFS /scratch1 233 TB - distributed file system based on OrangeFS
- available to all compute nodes and the login node
- no quota per user
- files not accessed for 30 days deleted on first day of each month
- designed for parallel I/O
ZFS /scratch2 150 TB - single server sharing space to all compute nodes and the login node
- no quota per user
- files not accessed for 30 days deleted daily
- designed for general I/O patterns (small and/or single process I/O)
XFS /scratch3 129 TB - single server sharing space to all compute nodes and the login node
- no quota per user
- files not accessed for 30 days deleted daily
- designed for general I/O patterns (small and/or single process I/O)

Long term storage

Long term storage solutions are available to users seeking a dedicated high performance storage. This service is provided for fee to Clemson users. Palmetto users may purchase ZFS storage with either dedicated server (purchasing of 150TB) or shared server (purchase in 1TB increments).

Long term storage space includes snapshots of changes and mirror system for disaster recovery.

More information about long term storage


Cypress Cluster

Clemson cyberinfrastructure includes a dedicated Hadoop environment, called Cypress, that is integrated with Palmetto’s infrastructure. The Cypress Cluster uses the Hortonworks Data Platform distribution of Hadoop and Spark to support data intensive computing and analytics. Cypress is available to all students, faculty, and staff with Palmetto Cluster accounts.

Overview

  • available for free to all Clemson students, faculty, and staff
  • 3.64 PB (petabyte) global Hadoop Distributed File System (HDFS)
  • 40 worker nodes (responsible for computation and data storage)
    • 256 GB of RAM per node
    • 16 nodes each have 12 1-TB local disks
    • 24 nodes each have 24 6-TB local disks
  • one dedicated Cypress Cluster user node for job submission and data staging
  • Hortonworks Data Platform distribution of Hadoop, Spark, and other Hadoop ecosystem services

For more information about investing into Cypress Cluster please contact Jeronica Williams jeronic@clemson.edu or Linh Ngo linh@clemson.edu.

More about Cypress

Support

The Cypress Cluster and data related support are provided by the Data Science group:


Open Science Grid

Open Science Grid (OSG) is a freely accessible distributed computing resource for scientific calculations designed to handle huge number of “small” computational tasks - high throughput computing (HTC).

Clemson University has been working with OSG on providing seamless access to the OSG resources for Clemson researchers. OSG has been recently integrated into the Palmetto cluster for sending and receiving high throughput jobs using the OSG framework. Access to OSG is free of charge.

Access to OSG from Palmetto is available using Connect Client software. OSG uses separate accounting system and before trying it Clemson users need to request an OSG account.

Open Science Grid
OSG Connect
New OSG Account


XSEDE resources

The Extreme Science and Engineering Discovery Environment (XSEDE) is a collection of national advanced cyberinfrastructure resources. XSEDE provide access to both dedicated computing systems and experts is computationally oriented research areas. Computing resources include Stampede, Comet, SuperMIC, Jetstream, Wrangler, Bridges and other systems.

For more information about XSEDE resources contact one of the XSEDE Campus Champions at Clemson University:

XSEDE
List of XSEDE resources
XSEDE Allocations


Visualization Lab

The Visualization Lab (Barre Hall 2004) provides cyberinfrastructure for the visualization and virtual reality needs of Clemson students, faculty and staff, including:

  • Virtual reality head mounted displays (Oculus Rift, Microsoft HoloLens, Samsun Gear VR, etc.,)
  • Visualization workstations equipped with high-end Nvidia Graphics cards
  • Tiled displays and 3-D projector
  • Visualization cluster with 5 nodes and 40 Gbps connection to Palmetto Cluster

For details about available resources, please see here, or contact Wole Oyekoya ooyekoy@clemson.edu. For events, demos, and office hours, please see the visualization calendar.


Network

Clemson network infrastructure is connected with high speed network provided by Internet 2. The high speed (100Gbps) network provides external connectivity to Palmetto cluster and main campus (selected buildings).

Internet 2
Advanced Layer 2 Service


Research support

Cyberinfrastructure Technology Integration (CITI) group provides support to Clemson University researchers in broadly defined research computing. CITI provides workshops covering introduction to HPC systems, introduction to programming for researchers and area specific research computing. CITI staff provides assistance in utilizing local and external dedicated computing resources and assists in porting and optimizing workflows.

CITI group includes subgroups

Advanced Computing and Research Methods Group

Advanced Visualization

Data Science Group

Education and Workforce Development

Geographic Information System Group

Program Manager

More about CITI group
CITI Training and workshops


Programming support

Clemson faculty may use dedicated programmers time for research projects that need development of software. Programmers time is provided for fee.

For more information about buying programmers time please contact Jeronica Williams jeronic@clemson.edu.


Prices

All prices cover 4 year term.

Type Unit Description Price Comments
Storage 1TB - ZFS system available only to Palmetto cluster
- Snapshots included in user space
- Full mirror for system recovery
$150.00 Owners of existing SAMQFS spaces may expand existing storage for the same price as ZFS storage
Palmetto compute node 1 unit - 2 x Intel Xeon E5-2680v3 “Haswell” @2.5 GHz (for a total of 24 cores)
- 2 x NVIDIA Tesla K40c GPU accelerators
- 128 GB DDR4 RAM
- 2 x 1 TB local hard drives
- On-board 10 Gbps Ethernet NIC
- InfiniBand FDR 56 Gbps network card
$6250.00 All grant budgets should assume $8000 price as a projected price for future expansions of Palmetto