Python 请求-如何使用系统 ca-securities (debian/ubuntu) ？

From https://stackoverflow.com/a/33717517/1695680

To make python requests use the system ca-certificates bundle, it needs to be told to use it over its own embedded bundle

export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

Requests embeds its bundles here, for reference:

/usr/local/lib/python2.7/site-packages/requests/cacert.pem
/usr/lib/python3/dist-packages/requests/cacert.pem

Or in newer versions use additional package to obtain certificates from: https://github.com/certifi/python-certifi

To verify from which file certificates are loaded, you can try:

Python 3.8.5 (default, Jul 28 2020, 12:59:40)
>>> import certifi
>>> certifi.where()
'/etc/ssl/certs/ca-certificates.crt'

I struggled with this for a week or so recently. I finally found that the way to verify a self-signed, or privately signed, certificate in Python. You need to create your own certificate bundle file. No need to update obscure certificate bundles every time you update a library, or add anything to the system certificate store.

Start by running the openssl command that you ran before, but add -showcerts. openssl s_client -connect mysite.local:443 -showcerts This will give you a long output, and at the top you'll see the entire certificate chain. Usually, this means three certs, the website's certificate, the intermediate certificate, and the root certificate in that order. We need to put just the root and intermediate certificates into a next file in the opposite order.

Copy the last cert, the root certificate, to a new text file. Grab just the stuff between, and including:

-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

Copy the middle cert (aka the intermediate certificate) to the new text file under the root cert. Again, grab the Begin and End Certificate lines and everything in between.

Save this text file to the directory where your Python script resides. My recommendation is to call it CertBundle.pem. (If you give it a different name, or put it somewhere else in your folder structure, make sure that the verify line reflects that.) Update your script to reference the new certificate bundle:

response = requests.post("https://www.example.com/", headers=headerContents, json=bodyContents, verify="CertBundle.pem")

And that's it. If you have only the root or only the intermediate certificate, then Python can't validate the entire certificate chain. But, if you include both of the certificates in the certificate bundle that you created, then Python can validate that the intermediate was signed by the root, and then when it accesses the website it can validate that the website's certificate was signed by the intermediate certificate.

edit: Fixed the file extension for the cert bundle. Also, fixed a couple of grammatical mistakes.

My two cents:

Thanks to this other answer, which had me check on actual requests code, I figured out that you don't have to use the env variable but can just set the "verify" param in your request:

requests.get("https://whatever", verify="/my/path/to/cacert.crt", ...)

It is also documented, although I could only find the documentation after having made the discovery (and the pypi project points to a dead link for doc) :D

requests uses certifi as default root certs package, which builts in a lot of good CAs but unable to modify.

Debian (and Ubuntu) maintainers changed certifi's behavior different from default:

def where():
return "/etc/ssl/certs/ca-certificates.crt"

So if you use apt-installed requests and certifi there is no problem.

But pip3 installed certifi inside virtual env use builtin CAs. So unable to use update-ca-certificates mechanism. Beside manually specifying root cert in app code (which may not be possible if request is called indirectly through 3rd party interfaces), it can also overriding with enviroument variable REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt to emulate the Debianized behavor.

If anyone ever has the problem. I tried a lot, but nothing really worked. a combination of installing discords cert and the Answer from Joren Boulanger here: https://stackoverflow.com/a/70199793/8889657 solved it for me. Dont have enough reputation to upvote or comment his answer.