将一个包含多行 JSON 的文件加载到 Pandas 中

我正在尝试将一个 JSON 文件读入 Python Pandas (0.14.0)数据帧中。下面是 JSON 文件的第一行:

{"votes": {"funny": 0, "useful": 0, "cool": 0}, "user_id": "P_Mk0ygOilLJo4_WEvabAA", "review_id": "OeT5kgUOe3vcN7H6ImVmZQ", "stars": 3, "date": "2005-08-26", "text": "This is a pretty typical cafe.  The sandwiches and wraps are good but a little overpriced and the food items are the same.  The chicken caesar salad wrap is my favorite here but everything else is pretty much par for the course.", "type": "review", "business_id": "Jp9svt7sRT4zwdbzQ8KQmw"}

我正在尝试做以下事情: df = pd.read_json(path)。

我得到了以下错误(完全回溯) :

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/d/anaconda/lib/python2.7/site-packages/pandas/io/json.py", line 198, in read_json
date_unit).parse()
File "/Users/d/anaconda/lib/python2.7/site-packages/pandas/io/json.py", line 266, in parse
self._parse_no_numpy()
File "/Users/d/anaconda/lib/python2.7/site-packages/pandas/io/json.py", line 483, in _parse_no_numpy
loads(json, precise_float=self.precise_float), dtype=None)
ValueError: Trailing data

什么是 Trailing data错误? 我如何读入一个数据帧？

根据一些建议，下面是.json 文件的几行代码:

{"votes": {"funny": 0, "useful": 0, "cool": 0}, "user_id": "P_Mk0ygOilLJo4_WEvabAA", "review_id": "OeT5kgUOe3vcN7H6ImVmZQ", "stars": 3, "date": "2005-08-26", "text": "This is a pretty typical cafe.  The sandwiches and wraps are good but a little overpriced and the food items are the same.  The chicken caesar salad wrap is my favorite here but everything else is pretty much par for the course.", "type": "review", "business_id": "Jp9svt7sRT4zwdbzQ8KQmw"}
{"votes": {"funny": 0, "useful": 0, "cool": 0}, "user_id": "TNJRTBrl0yjtpAACr1Bthg", "review_id": "qq3zF2dDUh3EjMDuKBqhEA", "stars": 3, "date": "2005-11-23", "text": "I agree with other reviewers - this is a pretty typical financial district cafe.  However, they have fantastic pies.  I ordered three pies for an office event (apple, pumpkin cheesecake, and pecan) - all were delicious, particularly the cheesecake.  The sucker weighed in about 4 pounds - no joke.\n\nNo surprises on the cafe side - great pies and cakes from the catering business.", "type": "review", "business_id": "Jp9svt7sRT4zwdbzQ8KQmw"}
{"votes": {"funny": 0, "useful": 0, "cool": 0}, "user_id": "H_mngeK3DmjlOu595zZMsA", "review_id": "i3eQTINJXe3WUmyIpvhE9w", "stars": 3, "date": "2005-11-23", "text": "Decent enough food, but very overpriced. Just a large soup is almost $5. Their specials are $6.50, and with an overpriced soda or juice, it's approaching $10. A bit much for a cafe lunch!", "type": "review", "business_id": "Jp9svt7sRT4zwdbzQ8KQmw"}

我使用的这个. JSON 文件按照规范在每一行中包含一个 JSON 对象。

我尝试了 Jsonlint.com的网站建议，它给出了以下错误:

Parse error on line 14:
...t7sRT4zwdbzQ8KQmw"}{    "votes": {
----------------------^
Expecting 'EOF', '}', ',', ']'
142994 次浏览
小开

你必须一行一行地读。例如，您可以在 Reddit上使用 氪星人提供的以下代码:

import pandas as pd


# read the entire file into a python array
with open('your.json', 'rb') as f:
data = f.readlines()


# remove the trailing "\n" from each line
data = map(lambda x: x.rstrip(), data)


# each element of 'data' is an individual JSON object.
# i want to convert it into an *array* of JSON objects
# which, in and of itself, is one large JSON object
# basically... add square brackets to the beginning
# and end, and have all the individual business JSON objects
# separated by a comma
data_json_str = "[" + ','.join(data) + "]"


# now, load it into pandas
data_df = pd.read_json(data_json_str)
小开
最佳答案

从0.19.0版本的熊猫，你可以使用 lines参数，像这样:

import pandas as pd


data = pd.read_json('/path/to/file.json', lines=True)
小开

下面的代码帮助我将 JSON内容加载到 dataframe中:

import json
import pandas as pd


with open('Appointment.json', encoding="utf8") as f:
data = f.readlines()
data = [json.loads(line) for line in data] #convert string to dict format
df = pd.read_json(data) # Load into dataframe
小开

我也有过类似的问题。

原来 pd.read_json(myfile.json)会自动在父文件夹中搜索，但是如果您不在与文件相同的文件夹中，它会返回这个“尾随数据”错误。

我弄明白了，因为当我试图用 open('myfile.json', 'r')做这件事的时候，我得到了一个 FileNotFound错误，所以我检查了路径。

我没有把 myfile.json 放到笔记本的同一个文件夹中。

改成 pd.read_json('../myfile.json')成功了。

小开

我也遇到过同样的问题。当您的数据以行写入时会发生这种情况，这些行之间用’n’这样的结束行分隔; 您需要首先以行的形式读取它们，然后将每一行转换为 python 内置类型。 我是这样解决的:

with open("/path/to/file") as f:
content = f.readlines()


data = [eval(c) for c in content]
data = pd.DataFrame(data)

祝你好运！


提交答案