How to make Python use CA certificates from Mac OS TrustStore?

If you put the additional certificates in a PEM bundle file you can use these two environment variables to overwrite the default cert stores used by Python openssl and requests.

SSL_CERT_FILE=/System/Library/OpenSSL/cert.pem
REQUESTS_CA_BUNDLE=/System/Library/OpenSSL/cert.pem

Please note that this file does not exist, you need to build it yourself.

This is also a problem in Python 3.6 with MacOS Sierrra. I know your use case is different. But I stumbled upon this thread while investigating this problem. So if anyone is also having this article is worth checking out:

http://www.cdotson.com/2017/01/sslerror-with-python-3-6-x-on-macos-sierra/

In a nutshell: Python 3.6 does not rely on MacOS' openSSL anymore. It comes with its own openSSL bundled and doesn't have access on MacOS' root certificates.

You have two options:

Run an install command shipped with Python 3.6

cd /Applications/Python\ 3.6/
./Install\ Certificates.command

or

Install the certifi package with

pip install certifi

我选择了第一个选项，它成功了。

For me /Applications/Python\ 3.6/./Install\ Certificates command fails on pip certifi install. I am on mac High Sierra and use python3 so pip somewhat fails and I have to use pip3 instead.

So here what I did:

1. Manually ran pip3 install --update certify in a shell
2. Remove the install certifi line from the command script
3. Reran the script and everything was fine.

Note that you will end up with a cert.pem symbolic link in: /Library/Frameworks/Python.framework/Versions/3.6/etc/openssl/

As an update and datapoint, I ran into this issue running Python 3.7.0 on macOS 10.13.4:

$ ipython
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.0.1 -- An enhanced Interactive Python. Type '?' for help.


In [1]: import bokeh.sampledata


In [2]: bokeh.sampledata.download()
Using data directory: /Users/me/.bokeh/data


...
SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)

Instructions for solving the problem are in /Applications/Python\ 3.7/ReadMe.rtf

Following the suggestion there and running /Applications/Python\ 3.7/Install\ Certificates.command solved the problem:

From the terminal:

$ /Applications/Python\ 3.7/Install\ Certificates.command

Re-starting IPython...

$ ipython
>>> import bokeh.sampledata


>>> bokeh.sampledata.download()
Using data directory: /Users/me/.bokeh/data
Downloading: CGM.csv (1589982 bytes)
1589982 [100.00%]
...

Mac brew install python env.

$ python3
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 16:52:21)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import certifi
>>> certifi.where()
'/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/certifi/cacert.pem'
>>>

Or from the command line:

$ python -m certifi

then need link cacert.pem as cert.pem

$ ln -s /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/certifi/cacert.pem cert.pem
$ pwd
/Library/Frameworks/Python.framework/Versions/3.7/etc/openssl


rehash

then work fine.

Run this to set the appropriate variables. This is a combination of the answers that have already been given here. Put it in your ~/.bash_profile to make it permanent.

CERT_PATH=$(python -m certifi)
export SSL_CERT_FILE=${CERT_PATH}
export REQUESTS_CA_BUNDLE=${CERT_PATH}

I see a lot of answers out there recommend to turn off certificate validation or to use certifi.where. While turning off SSL is obvious risk. certifi.where is also a risk, mainly if you intend to make this code a production code that will run in a customer env.

PEP describing why it is wrong. ssl.create_default_context is well integrate with linux and windows trustStore. the problem is, as in your case with mac. I solve this by loading the certificates using the integrated security commandline tool

def create_macos_ssl_context():
import subprocess
import ssl
import tempfile
ctx = ssl.create_default_context()
macos_ca_certs = subprocess.run(["security", "find-certificate", "-a", "-p",
"/System/Library/Keychains/SystemRootCertificates.keychain"],
stdout=subprocess.PIPE).stdout


with tempfile.NamedTemporaryFile('w+b') as tmp_file:
tmp_file.write(macos_ca_certs)
ctx.load_verify_locations(tmp_file.name)
print(ctx.get_ca_certs())

Note that this gives you the systemRoot certificates. if you need the user than simply change the value in the security command

Solution for MacOS or Linux with latest Python versions installed either as standalone or via port or brew

Download certificates from Certifi project at https://github.com/certifi/python-certifi/blob/master/certifi/cacert.pem. FYI Certifi is a 3rd party library that provides Mozilla’s curated collection of Root Certificates for validating the trustworthiness of SSL certificates while verifying the identity of TLS hosts.

Then add to your ~/.zshrc on the latest MacOS or ~/.bash_profile or similar:

export SSL_CERT_FILE=/pathtodownloadedfile/cacert.pem
export REQUESTS_CA_BUNDLE=/pathtodownloadedfile/cacert.pem

This was intended as an edit to an existing question, but since the queue was already full, posting as a separate answer.

Tested on MacOS 12.3.1 with python 3.10 installed with MacPorts.

If you prefer to trust root CA according to your OS, export them from System Roots keychain into a single file:

security export -t certs -f pemseq -k /System/Library/Keychains/SystemRootCertificates.keychain -o bundleCA.pem

If, additionally, you want to trust some internal self-signed CAs, export them, too. They are probably stored under the System keychain:

security export -t certs -f pemseq -k /Library/Keychains/System.keychain -o selfSignedCAbundle.pem

Merge the two files:

cat bundleCA.pem selfSignedCAbundle.pem >> allCAbundle.pem

Export as bash variable

export REQUESTS_CA_BUNDLE=/path/to/allCAbundle.pem

Consider adding the last code snippet to your .bash_profile

Note that REQUESTS_CA_BUNDLE works only for a single file, and does not for a directory.