ABSTRACT:

How to Fix the Side Effect caused by New SSL Cert on HydroShare

Revisions:
March 18, 2021; Zhiyu/Drew Li; zhiyul@illinois.edu

Symptoms:
Jupyter Hub fails in OAuth handshaking with HydroShare
“HTTP 599: server certificate verification failed. CAfile: none CRLfile: none”
hs_restclient fails to authenticate
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.hydroshare.org', port=443): Max retries exceeded with url: /hsapi/userInfo/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)')))

Cause:
HydroShare deployed a new SSL cert on March 17, 202. It is based on off a new CA, which is NOT included in the latest “ca-certificates” package (CA Bundle) on Ubuntu 18.04 and 20.04 as of this writing (other Linux distribution may also be affected).

Remedy:
Manually add this new CA into the CA Bundle on all clients that might need to talk to HydroShare.

Download the new CA cert:
Go to HydroShare keybase and download: star_hydroshare_org_124173627DigiCertCA.crt
Go to https://www.digicert.com/kb/digicert-root-certificates.htm, search for “GeoTrust TLS DV RSA Mixed SHA256 2020 CA-1” and download PEM format.

For Hub Dockerfile:

USER root
# get latest ca-bundle
RUN apt-get update && apt-get install -y ca-certificates
# load hydroshare new ca to image
COPY ./star_hydroshare_org_124173627DigiCertCA.crt /usr/local/share/ca-certificates/star_hydroshare_org_124173627DigiCertCA.crt
# update ca-bundle
RUN update-ca-certificates

For different conda envs in Dockerfile:

#Append new HydroShare CA to cacert.pem in Base conda env
RUN cat ./star_hydroshare_org_124173627DigiCertCA.crt >> /opt/conda/lib/python<VERSION>/site-packages/certifi/cacert.pem
# Append new HydroShare CA to user-created conda env
RUN cat ./star_hydroshare_org_124173627DigiCertCA.crt >> /opt/conda/envs/<ENV_NAME>/lib/python<VERSION>/site-packages/certifi/cacert.pem

References:
https://incognitjoe.github.io/adding-certs-to-requests.html
https://www.techrepublic.com/article/how-to-install-ca-certificates-in-ubuntu-server/