Checking for non-preferred file/folder path names (may take a long time depending on the number of files/folders) ...

Run WRFHydro model on HPC resources using CyberGIS-Compute V2 (updated 2022-10)


Authors:
Owners: This resource does not have an owner who is an active HydroShare user. Contact CUAHSI (help@cuahsi.org) for information on this resource.
Type: Resource
Storage: The size of this resource is 441.0 KB
Created: Mar 01, 2022 at 3:44 a.m.
Last updated: Oct 27, 2022 at 7:32 p.m.
Citation: See how to cite this resource
Sharing Status: Public
Views: 1210
Downloads: 67
+1 Votes: Be the first one to 
 this.
Comments: No comments (yet)

Abstract

This notebook demonstrates how to prepare a WRFHydro model on CyberGIS-Jupyter for Water (CJW) for execution on a supported High-Performance Computing (HPC) resource via the CyberGIS-Compute service. First-time users are highly encouraged to go through the [NCAR WRFHydro Hands-on Training on CJW](https://www.hydroshare.org/resource/d2c6618090f34ee898e005969b99cf90/) to get familiar WRFHydro model basics including compilation of source code, preparation of forcing data and typical model configurations. This notebook will not cover those topics and assume users already have hands-on experience with local model runs.

CyberGIS-Compute is a CyberGIS-enabled web service sits between CJW and HPC resources. It acts as a middleman that takes user requests (eg. submission of a model) originated from CJW, carries out the actual job submission of model on the target HPC resource, monitors job status, and retrieves outputs when the model execution has completed. The functionality of CyberGIS-Compute is exposed as a series of REST APIs. A Python client, [CyberGIS-Compute SDK](https://github.com/cybergis/cybergis-compute-python-sdk), has been developed for use in the CJW environment that provides a simple GUI to guide users through the job submission process. Prior to job submission, model configuration and input data should be prepared and arranged in a certain way that meets specific requirements, which vary by models and their implementation in CyberGIS-Compute. We will walk through the requirements for WRFHydro below.

The general workflow for WRFHydro in CyberGIS-Compute works as follows:

1. User picks a Model_Version of WRFHydro to use;
2. User prepares configuration files and data for the model on CJW;
3. User submits configuration files and data to CyberGIS-Compute;
4. CyberGIS-Compute transfers configuration files and data to target HPC;
5. CyberGIS-Compute downloads the chosen Model_Version of WRFhydro codebase on HPC;
6. CyberGIS-Compute applies compile-time configuration files to the codebase, and compiles the source code on the fly;
7. CyberGIS-Compute applies run-time configuration files and data to the model;
8. CyberGIS-Compute submits the model job to HPC scheduler for model execution;
9. CyberGIS-Compute monitors job status;
10. CyberGIS-Compute transfers model outputs from HPC to CJW upon user request;
11. User performs post-processing work on CJW;

Some steps in this notebook require user interaction. "Run cell by cell" is recommended. "Run All" may not work as expected.

How to run the notebook:
1) Click on the OpenWith button in the upper-right corner;
2) Select "CyberGIS-Jupyter for Water";
3) Open the notebook and follow instructions;

Subject Keywords

Content

Related Resources

This resource belongs to the following collections:
Title Owners Sharing Status My Permission
CyberGIS-Jupyter for Water (CJW) 2022 Q1 Release Notes Zhiyu/Drew Li · Anand Padmanabhan · Shaowen Wang · Alexander Michels · David Tarboton  Public &  Shareable Open Access
CyberGIS-Jupyter for Water (CJW) 2022-Q2 Release Notes Zhiyu/Drew Li · Shaowen Wang · Anand Padmanabhan · Alexander Michels  Public &  Shareable Open Access

How to Cite

Li, Z., A. Nassar, S. Wang, A. Padmanabhan, CyberGIS Center, D. Tarboton (2022). Run WRFHydro model on HPC resources using CyberGIS-Compute V2 (updated 2022-10), HydroShare, http://www.hydroshare.org/resource/cc28d769943046fdac0f9b0c0fc2afc6

This resource is shared under the Creative Commons Attribution CC BY.

http://creativecommons.org/licenses/by/4.0/
CC-BY

Comments

There are currently no comments

New Comment

required