如何使用Java从互联网下载和保存文件?

下载一个文件需要你阅读它。无论哪种方式，您都必须以某种方式查看该文件。而不是逐行，你可以从流中逐字节读取:

BufferedInputStream in = new BufferedInputStream(new URL("http://www.website.com/information.asp").openStream())
byte data[] = new byte[1024];
int count;
while((count = in.read(data, 0, 1024)) != -1)
{
out.write(data, 0, count);
}

试试Java NIO:

URL website = new URL("http://www.website.com/information.asp");
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream("information.html");
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);

使用transferFrom()是潜在的比从源通道读取并写入此通道的简单循环更有效。许多操作系统可以直接将字节从源通道传输到文件系统缓存中，而不需要实际复制它们。

检查更多关于它在这里。

请注意: transferFrom中的第三个参数是要传输的最大字节数。Integer.MAX_VALUE将传输最多2^31字节，Long.MAX_VALUE将允许最多2^63字节(比现有的任何文件都大)。

public void saveUrl(final String filename, final String urlString)
throws MalformedURLException, IOException {
BufferedInputStream in = null;
FileOutputStream fout = null;
try {
in = new BufferedInputStream(new URL(urlString).openStream());
fout = new FileOutputStream(filename);


final byte data[] = new byte[1024];
int count;
while ((count = in.read(data, 0, 1024)) != -1) {
fout.write(data, 0, count);
}
} finally {
if (in != null) {
in.close();
}
if (fout != null) {
fout.close();
}
}
}

您将需要处理异常，可能是该方法的外部异常。

就我个人而言，我发现Apache HttpClient的比我所需要做的一切都有能力。在这里是一个很好的使用HttpClient的教程

使用Apache 下议院IO。它只是一行代码:

FileUtils.copyURLToFile(URL, File)

import java.io.*;
import java.net.*;


public class filedown {
public static void download(String address, String localFileName) {
OutputStream out = null;
URLConnection conn = null;
InputStream in = null;


try {
URL url = new URL(address);
out = new BufferedOutputStream(new FileOutputStream(localFileName));
conn = url.openConnection();
in = conn.getInputStream();
byte[] buffer = new byte[1024];


int numRead;
long numWritten = 0;


while ((numRead = in.read(buffer)) != -1) {
out.write(buffer, 0, numRead);
numWritten += numRead;
}


System.out.println(localFileName + "\t" + numWritten);
}
catch (Exception exception) {
exception.printStackTrace();
}
finally {
try {
if (in != null) {
in.close();
}
if (out != null) {
out.close();
}
}
catch (IOException ioe) {
}
}
}


public static void download(String address) {
int lastSlashIndex = address.lastIndexOf('/');
if (lastSlashIndex >= 0 &&
lastSlashIndex < address.length() - 1) {
download(address, (new URL(address)).getFile());
}
else {
System.err.println("Could not figure out local file name for "+address);
}
}


public static void main(String[] args) {
for (int i = 0; i < args.length; i++) {
download(args[i]);
}
}
}

更简单的非阻塞I / O用法:

URL website = new URL("http://www.website.com/information.asp");
try (InputStream in = website.openStream()) {
Files.copy(in, target, StandardCopyOption.REPLACE_EXISTING);
}

简单使用有一个问题:

org.apache.commons.io.FileUtils.copyURLToFile(URL, File)

如果你需要下载和保存非常大的文件，或者在一般情况下，如果你需要自动重试以防连接断开。

在这种情况下，我建议使用Apache HttpClient以及org.apache.commons.io.FileUtils。例如:

GetMethod method = new GetMethod(resource_url);
try {
int statusCode = client.executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
logger.error("Get method failed: " + method.getStatusLine());
}
org.apache.commons.io.FileUtils.copyInputStreamToFile(
method.getResponseBodyAsStream(), new File(resource_file));
} catch (HttpException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
method.releaseConnection();
}

这个答案几乎和选中的答案完全一样，但是有两个增强:它是一个方法，它关闭了FileOutputStream对象:

    public static void downloadFileFromURL(String urlString, File destination) {
try {
URL website = new URL(urlString);
ReadableByteChannel rbc;
rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream(destination);
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
fos.close();
rbc.close();
} catch (IOException e) {
e.printStackTrace();
}
}

当使用Java 7+时，使用以下方法从Internet下载文件并将其保存到某个目录:

private static Path download(String sourceURL, String targetDirectory) throws IOException
{
URL url = new URL(sourceURL);
String fileName = sourceURL.substring(sourceURL.lastIndexOf('/') + 1, sourceURL.length());
Path targetPath = new File(targetDirectory + File.separator + fileName).toPath();
Files.copy(url.openStream(), targetPath, StandardCopyOption.REPLACE_EXISTING);


return targetPath;
}

文档是在这里。

这是另一个基于Brian Risk的回答的Java 7变体，使用来试试语句:

public static void downloadFileFromURL(String urlString, File destination) throws Throwable {


URL website = new URL(urlString);
try(
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream(destination);
) {


fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
}
}

可以使用Apache的HttpComponents而不是下议院IO来下载文件。这段代码允许您根据URL在Java中下载文件，并将其保存到特定的目的地。

public static boolean saveFile(URL fileURL, String fileSavePath) {


boolean isSucceed = true;


CloseableHttpClient httpClient = HttpClients.createDefault();


HttpGet httpGet = new HttpGet(fileURL.toString());
httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0");
httpGet.addHeader("Referer", "https://www.google.com");


try {
CloseableHttpResponse httpResponse = httpClient.execute(httpGet);
HttpEntity fileEntity = httpResponse.getEntity();


if (fileEntity != null) {
FileUtils.copyInputStreamToFile(fileEntity.getContent(), new File(fileSavePath));
}


} catch (IOException e) {
isSucceed = false;
}


httpGet.releaseConnection();


return isSucceed;
}

与单行代码相比:

FileUtils.copyURLToFile(fileURL, new File(fileSavePath),
URLS_FETCH_TIMEOUT, URLS_FETCH_TIMEOUT);

这段代码将让你对进程有更多的控制，不仅允许你指定超时，还允许你指定User-Agent和Referer值，这对许多网站来说是至关重要的。

在underscore-java库中有一个方法U.fetch(url)。

文件pom.xml:

<dependency>
<groupId>com.github.javadev</groupId>
<artifactId>underscore</artifactId>
<version>1.74</version>
</dependency>

代码示例:

import com.github.underscore.U;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;


public class Download {
public static void main(String[] args) throws IOException {
Files.write(Paths.get("data.bin"),
U.fetch("https://stackoverflow.com/questions"
+ "/921262/how-to-download-and-save-a-file-from-internet-using-java").blob());
}
}

总结(并以某种方式润色和更新)之前的答案。以下三种方法实际上是等效的。(我添加了明确的超时，因为我认为这是必须的。没有人希望下载在连接丢失时永远冻结。)

public static void saveUrl1(final Path file, final URL url,
int secsConnectTimeout, int secsReadTimeout))
throws MalformedURLException, IOException {


// Files.createDirectories(file.getParent()); // Optional, make sure parent directory exists
try (BufferedInputStream in = new BufferedInputStream(
streamFromUrl(url, secsConnectTimeout,secsReadTimeout));
OutputStream fout = Files.newOutputStream(file)) {


final byte data[] = new byte[8192];
int count;
while((count = in.read(data)) > 0)
fout.write(data, 0, count);
}
}


public static void saveUrl2(final Path file, final URL url,
int secsConnectTimeout, int secsReadTimeout))
throws MalformedURLException, IOException {


// Files.createDirectories(file.getParent()); // Optional, make sure parent directory exists
try (ReadableByteChannel rbc = Channels.newChannel(
streamFromUrl(url, secsConnectTimeout, secsReadTimeout)
);
FileChannel channel = FileChannel.open(file,
StandardOpenOption.CREATE,
StandardOpenOption.TRUNCATE_EXISTING,
StandardOpenOption.WRITE)
) {


channel.transferFrom(rbc, 0, Long.MAX_VALUE);
}
}


public static void saveUrl3(final Path file, final URL url,
int secsConnectTimeout, int secsReadTimeout))
throws MalformedURLException, IOException {


// Files.createDirectories(file.getParent()); // Optional, make sure parent directory exists
try (InputStream in = streamFromUrl(url, secsConnectTimeout,secsReadTimeout) ) {
Files.copy(in, file, StandardCopyOption.REPLACE_EXISTING);
}
}


public static InputStream streamFromUrl(URL url,int secsConnectTimeout,int secsReadTimeout) throws IOException {
URLConnection conn = url.openConnection();
if(secsConnectTimeout>0)
conn.setConnectTimeout(secsConnectTimeout*1000);
if(secsReadTimeout>0)
conn.setReadTimeout(secsReadTimeout*1000);
return conn.getInputStream();
}

我没有发现明显的差异，在我看来都是对的。它们既安全又高效。(速度的差异似乎无关紧要——我从本地服务器写入180 MB到SSD磁盘的时间大约在1.2到1.5秒之间波动)。它们不需要外部库。所有这些都可以使用任意大小和(根据我的经验)HTTP重定向。

此外，如果没有找到资源(通常是404错误)，所有抛出FileNotFoundException，如果DNS解析失败则抛出java.net.UnknownHostException;其他IOException对应传输过程中的错误。

这里有许多优雅而有效的答案。但是简洁会让我们失去一些有用的信息。特别是，一个通常不希望将连接错误视为异常，另一个可能想要不同地处理某种与网络相关的错误——例如，决定是否应该重试下载。

下面是一个方法，它不会为网络错误抛出异常(仅用于真正异常的问题，如url格式错误或写入文件的问题)

/**
* Downloads from a (http/https) URL and saves to a file.
* Does not consider a connection error an Exception. Instead it returns:
*
*    0=ok
*    1=connection interrupted, timeout (but something was read)
*    2=not found (FileNotFoundException) (404)
*    3=server error (500...)
*    4=could not connect: connection timeout (no internet?) java.net.SocketTimeoutException
*    5=could not connect: (server down?) java.net.ConnectException
*    6=could not resolve host (bad host, or no internet - no dns)
*
* @param file File to write. Parent directory will be created if necessary
* @param url  http/https url to connect
* @param secsConnectTimeout Seconds to wait for connection establishment
* @param secsReadTimeout Read timeout in seconds - trasmission will abort if it freezes more than this
* @return See above
* @throws IOException Only if URL is malformed or if could not create the file
*/
public static int saveUrl(final Path file, final URL url,
int secsConnectTimeout, int secsReadTimeout) throws IOException {
Files.createDirectories(file.getParent()); // make sure parent dir exists , this can throw exception
URLConnection conn = url.openConnection(); // can throw exception if bad url
if( secsConnectTimeout > 0 ) conn.setConnectTimeout(secsConnectTimeout * 1000);
if( secsReadTimeout > 0 ) conn.setReadTimeout(secsReadTimeout * 1000);
int ret = 0;
boolean somethingRead = false;
try (InputStream is = conn.getInputStream()) {
try (BufferedInputStream in = new BufferedInputStream(is); OutputStream fout = Files
.newOutputStream(file)) {
final byte data[] = new byte[8192];
int count;
while((count = in.read(data)) > 0) {
somethingRead = true;
fout.write(data, 0, count);
}
}
} catch(java.io.IOException e) {
int httpcode = 999;
try {
httpcode = ((HttpURLConnection) conn).getResponseCode();
} catch(Exception ee) {}
if( somethingRead && e instanceof java.net.SocketTimeoutException ) ret = 1;
else if( e instanceof FileNotFoundException && httpcode >= 400 && httpcode < 500 ) ret = 2;
else if( httpcode >= 400 && httpcode < 600 ) ret = 3;
else if( e instanceof java.net.SocketTimeoutException ) ret = 4;
else if( e instanceof java.net.ConnectException ) ret = 5;
else if( e instanceof java.net.UnknownHostException ) ret = 6;
else throw e;
}
return ret;
}

public class DownloadManager {


static String urls = "[WEBSITE NAME]";


public static void main(String[] args) throws IOException{
URL url = verify(urls);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
InputStream in = null;
String filename = url.getFile();
filename = filename.substring(filename.lastIndexOf('/') + 1);
FileOutputStream out = new FileOutputStream("C:\\Java2_programiranje/Network/DownloadTest1/Project/Output" + File.separator + filename);
in = connection.getInputStream();
int read = -1;
byte[] buffer = new byte[4096];
while((read = in.read(buffer)) != -1){
out.write(buffer, 0, read);
System.out.println("[SYSTEM/INFO]: Downloading file...");
}
in.close();
out.close();
System.out.println("[SYSTEM/INFO]: File Downloaded!");
}
private static URL verify(String url){
if(!url.toLowerCase().startsWith("http://")) {
return null;
}
URL verifyUrl = null;


try{
verifyUrl = new URL(url);
}catch(Exception e){
e.printStackTrace();
}
return verifyUrl;
}
}

你可以使用Java netloader在一行中完成:

new NetFile(new File("my/zips/1.zip"), "https://example.com/example.zip", -1).load(); // Returns true if succeed, otherwise false.

如果你使用代理，你可以在Java程序中设置代理，如下所示:

Properties systemSettings = System.getProperties();
systemSettings.put("proxySet", "true");
systemSettings.put("https.proxyHost", "HTTPS proxy of your org");
systemSettings.put("https.proxyPort", "8080");

如果您没有使用代理，请不要在代码中包含上述代码行。完整的工作代码下载文件时，你是一个代理。

public static void main(String[] args) throws IOException {
String url = "https://raw.githubusercontent.com/bpjoshi/fxservice/master/src/test/java/com/bpjoshi/fxservice/api/TradeControllerTest.java";
OutputStream outStream = null;
URLConnection connection = null;
InputStream is = null;
File targetFile = null;
URL server = null;


// Setting up proxies
Properties systemSettings = System.getProperties();
systemSettings.put("proxySet", "true");
systemSettings.put("https.proxyHost", "HTTPS proxy of my organisation");
systemSettings.put("https.proxyPort", "8080");
// The same way we could also set proxy for HTTP
System.setProperty("java.net.useSystemProxies", "true");
// Code to fetch file
try {
server = new URL(url);
connection = server.openConnection();
is = connection.getInputStream();
byte[] buffer = new byte[is.available()];
is.read(buffer);


targetFile = new File("src/main/resources/targetFile.java");
outStream = new FileOutputStream(targetFile);
outStream.write(buffer);
} catch (MalformedURLException e) {
System.out.println("THE URL IS NOT CORRECT ");
e.printStackTrace();
} catch (IOException e) {
System.out.println("I/O exception");
e.printStackTrace();
}
finally{
if(outStream != null)
outStream.close();
}
}

下面是一个简洁的、可读的、仅使用jdk的解决方案，其中包含适当的封闭资源:

static long download(String url, String fileName) throws IOException {
try (InputStream in = URI.create(url).toURL().openStream()) {
return Files.copy(in, Paths.get(fileName));
}
}

两行代码，没有依赖关系。

下面是一个带有输出、错误检查和命令行参数检查的完整的文件下载示例程序:

package so.downloader;


import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.nio.file.Files;
import java.nio.file.Paths;


public class Application {
public static void main(String[] args) throws IOException {
if (2 != args.length) {
System.out.println("USAGE: java -jar so-downloader.jar <source-URL> <target-filename>");
System.exit(1);
}


String sourceUrl = args[0];
String targetFilename = args[1];


long bytesDownloaded = download(sourceUrl, targetFilename);


System.out.println(String.format("Downloaded %d bytes from %s to %s.", bytesDownloaded, sourceUrl, targetFilename));
}


static long download(String url, String fileName) throws IOException {
try (InputStream in = URI.create(url).toURL().openStream()) {
return Files.copy(in, Paths.get(fileName));
}
}
}

如so-downloader库 README中所述:

运行文件下载程序:

java -jar so-downloader.jar <source-URL> <target-filename>

例如:

java -jar so-downloader.jar https://github.com/JanStureNielsen/so-downloader/archive/main.zip so-downloader-source.zip

下面是用Java代码从网上下载电影的示例代码:

URL url = new
URL("http://103.66.178.220/ftp/HDD2/Hindi%20Movies/2018/Hichki%202018.mkv");
BufferedInputStream bufferedInputStream = new  BufferedInputStream(url.openStream());
FileOutputStream stream = new FileOutputStream("/home/sachin/Desktop/test.mkv");


int count = 0;
byte[] b1 = new byte[100];


while((count = bufferedInputStream.read(b1)) != -1) {
System.out.println("b1:" + b1 + ">>" + count + ">> KB downloaded:" + new File("/home/sachin/Desktop/test.mkv").length()/1024);
stream.write(b1, 0, count);
}

第一种方法采用新通道

ReadableByteChannel aq = Channels.newChannel(new url("https//asd/abc.txt").openStream());
FileOutputStream fileOS = new FileOutputStream("C:Users/local/abc.txt")
FileChannel writech = fileOS.getChannel();

第二种方法使用FileUtils

FileUtils.copyURLToFile(new url("https//asd/abc.txt", new local file on system("C":/Users/system/abc.txt"));

第三种使用方法

InputStream xy = new ("https//asd/abc.txt").openStream();

这就是我们如何通过使用基本的Java代码和其他第三方库来下载文件。这些只是作为快速参考。请用谷歌以上关键词获取详细信息及其他选项。

这可以读取互联网上的文件，并将其写入文件。

import java.net.URL;
import java.io.FileOutputStream;
import java.io.File;


public class Download {
public static void main(String[] args) throws Exception {
URL url = new URL("https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png");  // Input URL
FileOutputStream out = new FileOutputStream(new File("out.png"));  // Output file
out.write(url.openStream().readAllBytes());
out.close();
}
}

在java.net.http.HttpClient上使用授权的解决方案:

HttpClient client = HttpClient.newHttpClient();


HttpRequest request = HttpRequest.newBuilder()
.GET()
.header("Accept", "application/json")
// .header("Authorization", "Basic ci5raG9kemhhZXY6NDdiYdfjlmNUM=") if you need
.uri(URI.create("https://jira.google.ru/secure/attachment/234096/screenshot-1.png"))
.build();


HttpResponse<InputStream> response = client.send(request, HttpResponse.BodyHandlers.ofInputStream());


try (InputStream in = response.body()) {
Files.copy(in, Paths.get(target + "screenshot-1.png"), StandardCopyOption.REPLACE_EXISTING);
}