逐行读取 R 格式的文本文件

我想用一个 for 循环和文件的长度逐行读取一个 R 格式的文本文件。问题是它只打印字符(0)。这是密码:

fileName="up_down.txt"
con=file(fileName,open="r")
line=readLines(con)
long=length(line)
for (i in 1:long){
linn=readLines(con,1)
print(linn)
}
close(con)
159819 次浏览
小开

只要在你的文件中使用 readLines:

R> res <- readLines(system.file("DESCRIPTION", package="MASS"))
R> length(res)
[1] 27
R> res
[1] "Package: MASS"
[2] "Priority: recommended"
[3] "Version: 7.3-18"
[4] "Date: 2012-05-28"
[5] "Revision: $Rev: 3167 $"
[6] "Depends: R (>= 2.14.0), grDevices, graphics, stats, utils"
[7] "Suggests: lattice, nlme, nnet, survival"
[8] "Authors@R: c(person(\"Brian\", \"Ripley\", role = c(\"aut\", \"cre\", \"cph\"),"
[9] "        email = \"ripley@stats.ox.ac.uk\"), person(\"Kurt\", \"Hornik\", role"
[10] "        = \"trl\", comment = \"partial port ca 1998\"), person(\"Albrecht\","
[11] "        \"Gebhardt\", role = \"trl\", comment = \"partial port ca 1998\"),"
[12] "        person(\"David\", \"Firth\", role = \"ctb\"))"
[13] "Description: Functions and datasets to support Venables and Ripley,"
[14] "        'Modern Applied Statistics with S' (4th edition, 2002)."
[15] "Title: Support Functions and Datasets for Venables and Ripley's MASS"
[16] "License: GPL-2 | GPL-3"
[17] "URL: http://www.stats.ox.ac.uk/pub/MASS4/"
[18] "LazyData: yes"
[19] "Packaged: 2012-05-28 08:47:38 UTC; ripley"
[20] "Author: Brian Ripley [aut, cre, cph], Kurt Hornik [trl] (partial port"
[21] "        ca 1998), Albrecht Gebhardt [trl] (partial port ca 1998), David"
[22] "        Firth [ctb]"
[23] "Maintainer: Brian Ripley <ripley@stats.ox.ac.uk>"
[24] "Repository: CRAN"
[25] "Date/Publication: 2012-05-28 08:53:03"
[26] "Built: R 2.15.1; x86_64-pc-mingw32; 2012-06-22 14:16:09 UTC; windows"
[27] "Archs: i386, x64"
R>

有一个 整个手册致力于此。

小开
最佳答案

下面是使用 for循环的解决方案。重要的是，它将对 readLines的一次调用从 for 循环中取出，这样就不会一次又一次地不正确地调用它。这就是:

fileName <- "up_down.txt"
conn <- file(fileName,open="r")
linn <-readLines(conn)
for (i in 1:length(linn)){
print(linn[i])
}
close(conn)
小开

你应该小心与 readLines(...)和大文件。在记忆中阅读所有的行可能是有风险的。下面是一个如何读取文件并一次只处理一行的示例:

processFile = function(filepath) {
con = file(filepath, "r")
while ( TRUE ) {
line = readLines(con, n = 1)
if ( length(line) == 0 ) {
break
}
print(line)
}


close(con)
}

也要了解在内存中读取一行的风险。没有换行符的大文件也可以填满你的内存。

小开

我写了一个代码来逐行读取文件，以满足我的需求，不同的行有不同的数据类型如下文章: 逐行读取文件和 确定行数记录。我认为，对于大型文件来说，这应该是一个更好的解决方案。我的 R 版本(3.3.2)。

con = file("pathtotargetfile", "r")
readsizeof<-2    # read size for one step to caculate number of lines in file
nooflines<-0     # number of lines
while((linesread<-length(readLines(con,readsizeof)))>0)    # calculate number of lines. Also a better solution for big file
nooflines<-nooflines+linesread


con = file("pathtotargetfile", "r")    # open file again to variable con, since the cursor have went to the end of the file after caculating number of lines
typelist = list(0,'c',0,'c',0,0,'c',0)    # a list to specific the lines data type, which means the first line has same type with 0 (e.g. numeric)and second line has same type with 'c' (e.g. character). This meet my demand.
for(i in 1:nooflines) {
tmp <- scan(file=con, nlines=1, what=typelist[[i]], quiet=TRUE)
print(is.vector(tmp))
print(tmp)
}
close(con)
小开

我建议你查看一下 chunked和 disk.frame，它们都有函数可以一块一块地读取 CSV。

特别是，disk.frame::csv_to_disk.frame可能是你所追求的功能？

小开 
fileName = "up_down.txt"


### code to get the line count of the file
length_connection = pipe(paste("cat ", fileName, " | wc -l", sep = "")) # "cat fileName | wc -l" because that returns just the line count, and NOT the name of the file with it
long = as.numeric(trimws(readLines(con = length_connection, n = 1)))
close(length_connection) # make sure to close the connection
###


for (i in 1:long){


### code to extract a single line at row i from the file
linn_connection_cmd = paste("head -n", format(x = i, scientific = FALSE, big.mark = ""), fileName, "| tail -n 1", sep = " ") # extracts one line from fileName at the desired line number (i)
linn_connection = pipe(linn_connection_cmd)
linn = readLines(con = linn_connection, n = 1)
close(linn_connection) # make sure to close the conection
###
    

# the line is now loaded into R and anything can be done with it
print(linn)
}
close(con)

通过使用 R 的 pipe()命令，并使用 shell 命令来提取我们需要的内容，完整的文件将永远不会加载到 R 中，而是逐行读取。

paste("head -n", format(x = i, scientific = FALSE, big.mark = ""), fileName, "| tail -n 1", sep = " ")

正是这个命令完成了所有的工作; 它从所需的文件中提取一行。

编辑 : R 的默认行为是当 少于为100,000时返回正常数值，但当 大于或等于为100,000(1e + 05)时开始返回科学记数法的 i。因此，在我们的管道命令中使用 format(x = i, scientific = FALSE, big.mark = "")来确保 pipe()命令总是收到一个普通形式的数字，这是该命令所能理解的全部内容。如果 pipe()命令被赋予任何类似1e + 05的数字，它将无法理解它，并将导致以下错误:

head: 1e+05: invalid number of lines

提交答案