LWI Model Naming Conventions: Stream naming algorithm

This software implements the Louisiana Watershed Initiative (LWI) stream naming algorithm, which is part of the LWI model data naming conventions. These conventions define the naming of physical features (watersheds, streams) as well as HEC-HMS hydrologic and HEC-RAS hydraulic model elements to be incorporated in watershed models developed under the auspices of the LWI. These naming conventions will be published as an appendix of the LWI Guidance on Modeling Methodology.

Faithful application of these standards is critical to ensuring that LWI models are of high quality and are consistent with each other, to the greatest extent practicable. Consistency between models enables interoperability of models between geographies (e.g. coupled HUC8 watersheds) and across modeling frameworks (e.g. integrating HEC-RAS models with ADCIRC in the coastal zone).

License

Louisiana Watershed Initiative (LWI) Stream Naming Convention Algorithm. Copyright (C) 2021-present State of Louisiana, Division of Administration, Office of Community Development.

This software program was developed by researchers at the Louisiana Watershed Flood Center of the University of Louisiana at Lafayette with funding from the State of Louisiana, Division of Administration, Office of Community Development under the LWI program. The LWI is funded by Community Development Block Grant Mitigation funds from U.S. Department of Housing and Urban Development.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.

Setup

Pre-requisites

• Unix-like environment (WSL should work)
• ~25 GB of free disk space for installation (about 7 GB needed after installation)
• Python 3.7+
• GDAL 3+ (version 2+ may work as well)
• SQLite 3
• 7z
• wget
• git

Under Debian Linux (either in WSL or natively), you can install the dependencies as: sudo apt-get update sudo apt-get install python3 python3-venv sqlite3 gdal-bin p7zip-full wget git

Installation

git clone https://github.com/ulfloodcenter/lwi-model-naming-conventions.git cd lwi-model-naming-conventions python3 -m venv venv source venv/bin/activate python3 setup.py install

Download and prepare data

./bin/download-data.sh

This could take one to two hours or more to complete, depending on your internet connection speed and the speed of your computer.

Usage

Label streams (using NHDPlus V2 data)

mkdir -p output lwi-label-nhd-streams -f data/NHDFlowline_Network.spatialite -p data/NHD_PlusFlow.sqlite

By default, this will use the file input/LWI_watersheds.csv to control which HUC8 watersheds will have their streams labeled, and also control the two-letter watershed code to be used for each HUC8. To use another watershed definition file, use the -w option. Use the --help to see all options.

Also, by default, and in accordance with LWI Model Data Naming Conventions, stream level labels will be encoded as base32. To use hexadecimal instead, add the --hexadecimal command line option.

Output will be stored in a directory named output.

Combine output into one CSV file and add header

tail -q -n +2 *.csv > /tmp/LA_HUC8_stream_labels.csv cat /tmp/LA_HUC8_stream_labels.csv | sort > /tmp/LA_HUC8_stream_labels-sorted.csv head -n 1 AA_08080101.csv | cat - /tmp/LA_HUC8_stream_labels-sorted.csv > LA_HUC8_stream_labels.csv rm /tmp/LA_HUC8_stream_labels.csv /tmp/LA_HUC8_stream_labels-sorted.csv

Load CSV into your GIS and join to NHD Flowline layer

Use your favorite GIS to join the combined labeled streams in LA_HUC8_stream_labels.csv generated above to your own copy of the NHDFlowline vector layer. If you are using QGIS, you should be able to use data/NHDFlowline_Network.spatialite generated by download-data.sh as your NHDFlowline vector layer.

Note: Use the comid field in LA_HUC8_stream_labels.csv to join to the comid field in the NHDFlowline layer.

Example data

Example output that has been joined to NHD Flowlines for HUC8s in the state of Louisiana can be found here.

Decoding Crockford base32 values

You can easily decode and encode values from/to Crockford base32 using the base32_crockford Python library. This library is installed as part of lwi-model-naming-conventions. The easiest way to use base32_crockford is from within the virtual environment you created above: ``` $ python Python 3.9.7 (default, Sep 3 2021, 12:37:55) [Clang 12.0.5 (clang-1205.0.22.9)] on darwin Type "help", "copyright", "credits" or "license" for more information.

import base32_crockford as b32 b32.decode("61") 193 b32.decode("5e") 174 b32.encode("193") '61' b32.encode("174") '5E'

```

Appendix: setting up and using with NHDPlus HR data

NHDPlus HD currently (August 2021) seems to lack flow topology information in the NHDPlusFlow table needed to successfully run the LWI naming algorithm. However, the instructions below are included as a starting point for future use of NHDPlus HR.

Extract HUC4-based data for all of Louisiana

ogr2ogr -skipfailures -f "SQLite" -dsco "SPATIALITE=YES" NHDPlusHR-LA.sqlite NHDPLUS_H_1114_HU4_GDB/NHDPLUS_H_1114_HU4_GDB.gdb NHDFlowline NHDPlusFlow NHDPlusFlowlineVAA && \ ogr2ogr -skipfailures -append -f "SQLite" -dsco "SPATIALITE=YES" NHDPlusHR-LA.sqlite NHDPLUS_H_1204_HU4_GDB/NHDPLUS_H_1204_HU4_GDB.gdb NHDFlowline NHDPlusFlow NHDPlusFlowlineVAA && \ ogr2ogr -skipfailures -append -f "SQLite" -dsco "SPATIALITE=YES" NHDPlusHR-LA.sqlite NHDPLUS_H_1201_HU4_GDB/NHDPLUS_H_1201_HU4_GDB.gdb NHDFlowline NHDPlusFlow NHDPlusFlowlineVAA && \ ogr2ogr -skipfailures -append -f "SQLite" -dsco "SPATIALITE=YES" NHDPlusHR-LA.sqlite NHDPLUS_H_0809_HU4_GDB/NHDPLUS_H_0809_HU4_GDB.gdb NHDFlowline NHDPlusFlow NHDPlusFlowlineVAA && \ ogr2ogr -skipfailures -append -f "SQLite" -dsco "SPATIALITE=YES" NHDPlusHR-LA.sqlite NHDPLUS_H_0808_HU4_GDB/NHDPLUS_H_0808_HU4_GDB.gdb NHDFlowline NHDPlusFlow NHDPlusFlowlineVAA && \ ogr2ogr -skipfailures -append -f "SQLite" -dsco "SPATIALITE=YES" NHDPlusHR-LA.sqlite NHDPLUS_H_0807_HU4_GDB/NHDPLUS_H_0807_HU4_GDB.gdb NHDFlowline NHDPlusFlow NHDPlusFlowlineVAA && \ ogr2ogr -skipfailures -append -f "SQLite" -dsco "SPATIALITE=YES" NHDPlusHR-LA.sqlite NHDPLUS_H_0806_HU4_GDB/NHDPLUS_H_0806_HU4_GDB.gdb NHDFlowline NHDPlusFlow NHDPlusFlowlineVAA && \ ogr2ogr -skipfailures -append -f "SQLite" -dsco "SPATIALITE=YES" NHDPlusHR-LA.sqlite NHDPLUS_H_0805_HU4_GDB/NHDPLUS_H_0805_HU4_GDB.gdb NHDFlowline NHDPlusFlow NHDPlusFlowlineVAA && \ ogr2ogr -skipfailures -append -f "SQLite" -dsco "SPATIALITE=YES" NHDPlusHR-LA.sqlite NHDPLUS_H_0804_HU4_GDB/NHDPLUS_H_0804_HU4_GDB.gdb NHDFlowline NHDPlusFlow NHDPlusFlowlineVAA && \ ogr2ogr -skipfailures -append -f "SQLite" -dsco "SPATIALITE=YES" NHDPlusHR-LA.sqlite NHDPLUS_H_0318_HU4_GDB/NHDPLUS_H_0318_HU4_GDB.gdb NHDFlowline NHDPlusFlow NHDPlusFlowlineVAA

Add indexes for fast searching

sqlite3 NHDPlusHR-LA.sqlite \ "create index if not exists nhdflowline_nhdplusid_idx on nhdflowline (nhdplusid); \ create index if not exists nhdplusflowlinevaa_nhdplusid_idx on nhdplusflowlinevaa (nhdplusid); \ create index if not exists nhdplusflow_fromnhdpid_idx on nhdplusflow(fromnhdpid); \ create index if not exists nhdplusflow_tonhdpid_idx on nhdplusflow(tonhdpid);"

Usage

mkdir -p output lwi-label-nhd-streams -f /path/to/NHDPlusHR-LA.sqlite --nhdhr

WARNING: NHDPlus HD seems to lack flow topology information in the NHDPlusFlow table needed to successfully run the LWI naming algorithm.

Note: To encode stream level labels as base32 instead of hexadecimal, add the --base32 command line option.

Output will be stored in a directory named output.

Concatenate output into one file

tail -q -n +2 *.csv > /tmp/LA_HUC8_stream_labels.csv cat /tmp/LA_HUC8_stream_labels.csv | sort > /tmp/LA_HUC8_stream_labels-sorted.csv head -n 1 AA_08080101.csv | cat - /tmp/LA_HUC8_stream_labels-sorted.csv > LA_HUC8_stream_labels.csv rm /tmp/LA_HUC8_stream_labels.csv /tmp/LA_HUC8_stream_labels-sorted.csv