Including Flux in Grant Proposals

Using Flux is the recommended approach to providing HPC resources to College of Engineering faculty. The College of Engineering discourages buying equipment to support computing activities that can be reasonably supported by Flux.

Flux is primarily a computing resource. Requirements and funding for storage, networking, and other cyberinfrastructure needs not addressed by Flux should be discussed with the Associate Dean for Research.

Flux Description

Computing
Flux is an HPC Linux-based cluster intended to support parallel and other applications that are not suitable for departmental or individual computers. Each Flux compute node comprises multiple CPU cores with at least 4 GB of RAM per core; Flux has over 8,000 cores. All compute nodes are interconnected with InfiniBand networking.

Computing jobs on Flux are managed through a combination of the Moab Scheduler, the Terascale Open-Source Resource and QUEue Manager (Torque) and the GOLD Allocation Manager from Adaptive Computing.

Flux Configuration has a detailed description of the Flux cluster.

Storage
The system also includes high speed scratch storage using the Lustre parallel network file system. The storage is also connected with InfiniBand. This file system allows researchers to store data on a short term basis to perform calculations; it is not for long term data storage or archival purposes.

Intra-networking
All Flux nodes are interconnected with quad-data rate InfiniBand, delivering up to 40 Gbps of bandwidth and less than 5μs latency.

Inter-networking
Flux is connected to the University of Michigan’s campus backbone to provide access to student and researcher desktops as well as other campus computing and storage systems. The campus backbone provides connectivity to the commodity Internet and the research networks Internet2 and MiLR.

Software
The Flux cluster includes a comprehensive software suite of commercial and open source research software, including major software compilers, and many of the common research specific applications such as Mathematica, Matlab, R and Stata.

Data Center Facilities
Flux is housed in a HP Performance Optimized Data Center (POD) 240a that is professionally run by Information and Technology Services.  The HP POD has batteries to provide power sufficient for a graceful shutdown of Flux.  The high cooling efficiency of the POD reduces the over-all cost for Flux allocations.

Support
Flux computing services are provided through a collaboration of University of Michigan units: The Office of Research Cyberinfrastructure (in the Office of the VP of Research and the Provost’s Office), the College of Engineering's central IT group, CAEN, Information and Technology Services, and computing groups in schools and colleges at the University.

The following steps will help you include Flux in a grant proposal

1. Determine the suitability of Flux for your research by considering whether a large computing resource is required. It is important that the proposed funds will provide computing cycles in a way that allows the team of researchers to allocate them as needed. The size of an allocation can be changed on a month by month basis to meet research needs and make the best possible use of the awarded funds. 
   • Faculty-owned or provided hardware cannot be accepted into Flux.
2. Determine if the constraints on access to Flux are suitable for your project. Access to Flux and the software library will be granted to all University of Michigan faculty, staff, graduate, and undergraduate students. Contractors and collaborators from other institutions may not use Flux because of licensing limitations with third party commercial software.
3. Determine an appropriate budget to include in the proposal; the cost per core month through June 2012 is an approved rate and may be charged as a direct cost to federal grants. Flux Costing contains the most current Flux rate for budgeting use in your proposal. Flux Sizing will help estimate Flux allocations for budget planning purposes. For questions or more information about estimating usage, contact flux-support@umich.edu.
4. Use the appropriate parts of the Flux Description above in your proposal.
   • In NSF proposals use the category “computer service” and the phrase “cluster compute allocation” with quantities expressed as core-months or core-years to describe Flux time.
5. Plan for the end of the award period or the exhaustion of the funds. At that time, the allocation on Flux expires and no more jobs associated with that Flux project can run.
6. Using the information from the previous steps:
   1. Complete the Flux PAF Supplement.
   2. Email the completed file to coe-flux-paf-review@umich.edu.
   3. Attach the approved Flux PAF Supplement to your PAF.