读取一个文件夹中的所有文件，并对每个数据帧应用一个函数

我正在做一个相对简单的分析，我已经把一个函数的所有文件在一个特定的文件夹。我想知道是否有人有任何建议，以帮助我自动化的过程中的一些不同的文件夹。

  1. 首先，我想知道是否有一种方法可以直接将特定文件夹中的所有文件读入 R。我相信下面的命令会列出所有文件:

files <- (Sys.glob("*.csv"))

我在 使用 R 列出具有指定扩展名的所有文件上找到的

然后下面的代码将所有这些文件读入 R。

listOfFiles <- lapply(files, function(x) read.table(x, header = FALSE))

来自 在 R 中操作多个文件

但是这些文件看起来像是一个连续的列表，而不是单独的文件... 我怎样才能改变脚本来打开一个特定文件夹中的所有 csv 文件作为单独的数据框架？

  1. 其次，假设我可以分别读取所有文件，那么如何一次性完成所有这些数据框架上的函数。例如，我已经创建了四个小型数据框架，这样我就可以说明我想要的东西:

     Df.1 <- data.frame(A = c(5,4,7,6,8,4),B = (c(1,5,2,4,9,1)))
Df.2 <- data.frame(A = c(1:6),B = (c(2,3,4,5,1,1)))
Df.3 <- data.frame(A = c(4,6,8,0,1,11),B = (c(7,6,5,9,1,15)))
Df.4 <- data.frame(A = c(4,2,6,8,1,0),B = (c(3,1,9,11,2,16)))

我还创建了一个示例函数:

Summary<-function(dfile){
SumA<-sum(dfile$A)
MinA<-min(dfile$A)
MeanA<-mean(dfile$A)
MedianA<-median(dfile$A)
MaxA<-max(dfile$A)


sumB<-sum(dfile$B)
MinB<-min(dfile$B)
MeanB<-mean(dfile$B)
MedianB<-median(dfile$B)
MaxB<-max(dfile$B)


Sum<-c(sumA,sumB)
Min<-c(MinA,MinB)
Mean<-c(MeanA,MeanB)
Median<-c(MedianA,MedianB)
Max<-c(MaxA,MaxB)
rm(sumA,sumB,MinA,MinB,MeanA,MeanB,MedianA,MedianB,MaxA,MaxB)


Label<-c("A","B")
dfile_summary<-data.frame(Label,Sum,Min,Mean,Median,Max)
return(dfile_summary)}

我通常会使用下面的命令对每个单独的数据框应用函数。

摘要 <-摘要(dfile)

是否有一种方法可以代替将该函数应用于所有数据框架，并使用汇总表中的数据框架标题(即 Df1.Summary)。

非常感谢,

凯蒂

165191 次浏览
小开

On the contrary, I do think working with list makes it easy to automate such things.

Here is one solution (I stored your four dataframes in folder temp/).

filenames <- list.files("temp", pattern="*.csv", full.names=TRUE)
ldf <- lapply(filenames, read.csv)
res <- lapply(ldf, summary)
names(res) <- substr(filenames, 6, 30)

It is important to store the full path for your files (as I did with full.names), otherwise you have to paste the working directory, e.g. 

filenames <- list.files("temp", pattern="*.csv")
paste("temp", filenames, sep="/")

will work too. Note that I used substr to extract file names while discarding full path.

You can access your summary tables as follows:

> res$`df4.csv`
A              B
Min.   :0.00   Min.   : 1.00
1st Qu.:1.25   1st Qu.: 2.25
Median :3.00   Median : 6.00
Mean   :3.50   Mean   : 7.00
3rd Qu.:5.50   3rd Qu.:10.50
Max.   :8.00   Max.   :16.00

If you really want to get individual summary tables, you can extract them afterwards. E.g., 

for (i in 1:length(res))
assign(paste(paste("df", i, sep=""), "summary", sep="."), res[[i]])
小开

usually i don't use for loop in R, but here is my solution using for loops and two packages : plyr and dostats

plyr is on cran and you can download dostats on https://github.com/halpo/dostats (may be using install_github from Hadley devtools package)

Assuming that i have your first two data.frame (Df.1 and Df.2) in csv files, you can do something like this.

require(plyr)
require(dostats)


files <- list.files(pattern = ".csv")




for (i in seq_along(files)) {


assign(paste("Df", i, sep = "."), read.csv(files[i]))


assign(paste(paste("Df", i, sep = ""), "summary", sep = "."),
ldply(get(paste("Df", i, sep = ".")), dostats, sum, min, mean, median, max))


}

Here is the output

R> Df1.summary
.id sum min   mean median max
1   A  34   4 5.6667    5.5   8
2   B  22   1 3.6667    3.0   9
R> Df2.summary
.id sum min   mean median max
1   A  21   1 3.5000    3.5   6
2   B  16   1 2.6667    2.5   5
小开

Here is a tidyverse option that might not the most elegant, but offers some flexibility in terms of what is included in the summary:

library(tidyverse)
dir_path <- '~/path/to/data/directory/'
file_pattern <- 'Df\\.[0-9]\\.csv' # regex pattern to match the file name format


read_dir <- function(dir_path, file_name){
read_csv(paste0(dir_path, file_name)) %>%
mutate(file_name = file_name) %>%                # add the file name as a column
gather(variable, value, A:B) %>%                 # convert the data from wide to long
group_by(file_name, variable) %>%
summarize(sum = sum(value, na.rm = TRUE),
min = min(value, na.rm = TRUE),
mean = mean(value, na.rm = TRUE),
median = median(value, na.rm = TRUE),
max = max(value, na.rm = TRUE))
}


df_summary <-
list.files(dir_path, pattern = file_pattern) %>%
map_df(~ read_dir(dir_path, .))


df_summary
# A tibble: 8 x 7
# Groups:   file_name [?]
file_name variable   sum   min  mean median   max
<chr>     <chr>    <int> <dbl> <dbl>  <dbl> <dbl>
1 Df.1.csv  A           34     4  5.67    5.5     8
2 Df.1.csv  B           22     1  3.67    3       9
3 Df.2.csv  A           21     1  3.5     3.5     6
4 Df.2.csv  B           16     1  2.67    2.5     5
5 Df.3.csv  A           30     0  5       5      11
6 Df.3.csv  B           43     1  7.17    6.5    15
7 Df.4.csv  A           21     0  3.5     3       8
8 Df.4.csv  B           42     1  7       6      16

提交答案