Compute Farms

Computing farms are pools of distributed computing resources designed to meet the needs of users beyond the capability of a single machine. Farms range in size and capability from a single machine on a user's desk to large integrated systems capable of functioning in parallel.

The School runs a number of farms and also has access to the University ECDF facility.  

The following flow chart should help guide you to the best distributed computing method for you[1]:

 





 Clustered computing flowchart

 

Interactive (NX)

In compute farming, interactive refers to software which accepts and responds to input from humans – for example, data or commands. Most interactive programs rely on a graphical interface. Such software is difficult to run unattended in batch mode (see the next section).

Software such as MATLAB, Star-CCM+ and even Firefox, are specially suited to running interactively.

In the School, you can achieve interactive computing either on your desktop machine or, if your application requires more processing power, through NX to the VLX pool.

Batch jobs (Condor)

Batch processing is execution of a series of programs ("jobs") on a computer without human interaction. Batch jobs are set up so they can be run to completion without human interaction, so all input data are provided through scripts or command-line parameters.

The advantage of using batch jobs is that you can run many jobs at once without having to be present to supervise or control them.

Software such as MATLAB scripts (compiled with mcc), small Abaqus simulations, Fortran/C simulations and other scripted jobs are specially suited to running as batch jobs.

In the School, you can run batch jobs through the Condor pool.

Multi-processor (CLX and ECDF)

For those users who are running software which can use multiple threads (using, for example, pthreads or OpenMP) or multiple co-operating processes with MPI (we have MPICH2), we recommend CLX or ECDF. These are the services to use for Abaqus, FDS, DL_POLY and CFX with large enough data. CLX is our in-house multi-processing cluster and ECDF. CLX is free at the point of use whilst ECDF's 'Eddie' cluster has some free at the point of use time and some paid time available. Eddie is a much bigger cluster than CLX.

 

[1] Windows

We don't currently offer any Windows based cluster or parallel compute service. This is for several reasons.

  • We don't have any computers that are reliably 'up' in Windows all the time.
  • There hasn't been enough demand (or money) to provision such a cluster. Most people working on parallelisable problems use Linux.
  • Licencing costs. Parallelising Linux is free - it costs money with Windows (for the multi-core version of the OS and then for the clustering components from MS).

[1] Solaris

The legacy Sun farm for Solaris applications is still available as the "tlc" command, see "man tlc"