用 Python 获取网页内容？

如果有帮助的话，我用的是 Python 3.1。

不管怎样，我正在尝试获取 这个网页的内容。我谷歌了一下，尝试了不同的方法，但都没有用。我想这应该是个简单的任务，但是... 我做不到。:/.

Urllib，urllib2的结果:

>>> import urllib2
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import urllib2
ImportError: No module named urllib2
>>> import urllib
>>> urllib.urlopen("http://www.python.org")
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
urllib.urlopen("http://www.python.org")
AttributeError: 'module' object has no attribute 'urlopen'
>>>

Python 3解决方案

谢谢你，杰森。

import urllib.request
page = urllib.request.urlopen('http://services.runescape.com/m=hiscore/ranking?table=0&category_type=0&time_filter=0&date=1519066080774&user=zezima')
print(page.read())
184482 次浏览
小开

You can use urlib2 and parse the HTML yourself.

Or try Beautiful Soup to do some of the parsing for you.

小开
最佳答案

Because you're using Python 3.1, you need to use the new Python 3.1 APIs.

Try:

urllib.request.urlopen('http://www.python.org/')

Alternately, it looks like you're working from Python 2 examples. Write it in Python 2, then use the 2to3 tool to convert it. On Windows, 2to3.py is in \python31\tools\scripts. Can someone else point out where to find 2to3.py on other platforms?

Edit

These days, I write Python 2 and 3 compatible code by using six.

from six.moves import urllib
urllib.request.urlopen('http://www.python.org')

Assuming you have six installed, that runs on both Python 2 and Python 3.

小开

Mechanize is a great package for "acting like a browser", if you want to handle cookie state, etc.

http://wwwsearch.sourceforge.net/mechanize/

小开

If you ask me. try this one

import urllib2
resp = urllib2.urlopen('http://hiscore.runescape.com/index_lite.ws?player=zezima')

and read the normal way ie

page = resp.read()

Good luck though

小开

If you're writing a project which installs packages from PyPI, then the best and most common library to do this is requests. It provides lots of convenient but powerful features. Use it like this:

import requests
response = requests.get('http://hiscore.runescape.com/index_lite.ws?player=zezima')
print (response.status_code)
print (response.content)

But if your project does not install its own dependencies, i.e. is limited to things built-in to the standard library, then you should consult one of the other answers.

小开

A solution with works with Python 2.X and Python 3.X:

try:
# For Python 3.0 and later
from urllib.request import urlopen
except ImportError:
# Fall back to Python 2's urllib2
from urllib2 import urlopen


url = 'http://hiscore.runescape.com/index_lite.ws?player=zezima'
response = urlopen(url)
data = str(response.read())
小开

Suppose you want to GET a webpage's content. The following code does it: 

# -*- coding: utf-8 -*-
# python


# example of getting a web page


from urllib import urlopen
print urlopen("http://xahlee.info/python/python_index.html").read()
小开

Also you can use faster_than_requests package. That's very fast and simple:

import faster_than_requests as r
content = r.get2str("http://test.com/")

Look at this comparison:

enter image description here


提交答案