Search USGS⌕ Search
Home/Archive/USGS

USGS · 70118801

High throughput computing: a solution for scientific analysis

M. O'Donnell
View original source ↗Export metadata ↓

Abstract

Public land management agencies continually face resource management problems that are exacerbated by climate warming, land-use change, and other human activities. As the U.S. Geological Survey (USGS) Fort Collins Science Center (FORT) works with managers in U.S. Department of the Interior (DOI) agencies and other federal, state, and private entities, researchers are finding that the science needed to address these complex ecological questions across time and space produces substantial amounts of data. The additional data and the volume of computations needed to analyze it require expanded computing resources well beyond single- or even multiple-computer workstations. To meet this need for greater computational capacity, FORT investigated how to resolve the many computational shortfalls previously encountered when analyzing data for such projects. Our objectives included finding a solution that would: harness existing Computer Processing Units (CPUs) when they're idle to run multiple jobs concurrently, which reduces the overall processing time without requiring additional hardware; offer an effective, centralized job-management system; handle job failures due to hardware, software, or network interruptions (obviating the need to manually resubmit the job after each stoppage); be affordable; and most importantly, allow us to complete very large, complex analyses that otherwise would not even be possible. In short, we envisioned a job-management system that would take advantage of unused FORT CPUs within a local area network (LAN) to effectively distribute and run highly complex analytical processes. What we found was a solution that uses High Throughput Computing (HTC) and High Performance Computing (HPC) systems to do exactly that (Figure 1).

Explore related subjects

Report

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

M. O'Donnell. 2011. High throughput computing: a solution for scientific analysis. https://pubs.usgs.gov/publication/70118801

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related USGS reports

USGS2026

Factors affecting benthic macroinvertebrate health in the City of Roanoke, Virginia, 2020–2023

Major waterways in the City of Roanoke (City) have failed to meet Virginia’s aquatic life designated use since 1996. Segments of the upper Roanoke River lack healthy benthic macroinvertebrate communities which prompted a total maximum daily load (TMDL) study by the Virginia Department of Environmental Quality (VDEQ) to identify the most probable stressor(s) causing the impairment. Excess fine sediment was identified as the most probable stressor impairing benthic macroinvertebrates on portions of the Roanoke River in 2006, and a watershed implementation plan published in 2016 required communities within the impaired watershed to implement projects that would reduce the load of fine sediment entering the Roanoke River. Additional benthic macroinvertebrate sampling and stream habitat assessments along the Roanoke River and Tinker Creek (a tributary to the Roanoke River that flows through the City) revealed continued impaired conditions, and subsequent stressor identification analysis was completed in 2023. Samples collected downstream of the City on the Roanoke River and Tinker Creek generally showed more impaired conditions relative to samples collected at locations upstream of the City. Based on this evaluation, sediment and sediment-bound polychlorinated biphenyls (PCBs) were identified as probable stressors while specific conductance, total nitrogen, and sediment metals were possible stressors in Tinker Creek; however, only a sediment TMDL target was identified to address impaired benthic macroinvertebrate communities. In the Roanoke River upstream of the Niagara Dam, sediment and total phosphorus were identified as probable stressors, sediment polycyclic aromatic hydrocarbons and sediment PCB were considered possible stressors; however, the TMDL target was only for total phosphorus. The City partnered with the U.S. Geological Survey (USGS) in 2016 to continuously monitor water quality and streamflow conditions on a major tributary of Tinker Creek, Lick Run, and by 2020, four similar monitoring stations were installed on the Roanoke River and Tinker Creek near the locations of benthic macroinvertebrate sampling. Monitored parameters included streamflow and/or gage height (water level), water temperature, pH, dissolved oxygen, specific conductance, and turbidity. Turbidity is a measure of the relative clarity of the water and was previously used to model suspended-sediment concentrations at the monitoring stations. The City also contracted Kirk Environmental, LLP (KE) to collect benthic macroinvertebrate samples and stream habitat assessments near the locations of the water-quality monitoring stations. Identified benthic macroinvertebrates were used to calculate the Virginia Stream Condition Index (SCI), a multi-metric index composed of eight biological attributes that represent elements of the structure and function of the benthic macroinvertebrate community that measure diversity, composition, and tolerance to pollution. Study objective: In this report, benthic macroinvertebrate samples and stream habitat assessment scores collected at four locations on the Roanoke River and Tinker Creek by KE and the VDEQ between 2020 and 2023 were compared to measured water-quality and streamflow conditions prior to sampling to evaluate patterns between benthic macroinvertebrate health, water quality, and hydrology.

Virginia↗