从 HTTPClient 响应中解压缩 GZip 流

我试图从 WCF 服务(WCF 服务到 WCF 服务)连接到一个 api，它返回 GZip 编码的 JSON。我使用 HTTPClient连接到 API，并且能够以字符串的形式返回 JSON 对象。然而，我需要能够将这些返回的数据存储在数据库中，因此我认为最好的方法是返回并将 JSON 对象存储在一个数组或字节中，或者沿着这些方向存储。

具体来说，我遇到的问题是 GZip 编码的解压缩，我已经尝试了很多不同的例子，但仍然不能得到它。

下面的代码是我如何建立我的连接和获得响应，这是从 API 返回字符串的代码。

public string getData(string foo)
{
string url = "";
HttpClient client = new HttpClient();
HttpResponseMessage response;
string responseJsonContent;
try
{
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
response = client.GetAsync(url + foo).Result;
responseJsonContent = response.Content.ReadAsStringAsync().Result;
return responseJsonContent;
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
return "";
}
}

我一直遵循一些不同的例子，如这些 StackExchange API，MSDN，和一对夫妇在堆栈溢出，但我还没有能够得到这些为我工作。

最好的方法是什么，我走对路了吗？

Thanks guys.

66997 次浏览
小开

好吧，我最终解决了我的问题。如果有更好的方法请让我知道: -)

        public DataSet getData(string strFoo)
{
string url = "foo";


HttpClient client = new HttpClient();
HttpResponseMessage response;
DataSet dsTable = new DataSet();
try
{
//Gets the headers that should be sent with each request
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
//Returned JSON
response = client.GetAsync(url).Result;
//converts JSON to string
string responseJSONContent = response.Content.ReadAsStringAsync().Result;
//deserializes string to list
var jsonList = DeSerializeJsonString(responseJSONContent);
//converts list to dataset. Bad name I know.
dsTable = Foo_ConnectAPI.ExtentsionHelpers.ToDataSet<RootObject>(jsonList);
//Returns the dataset
return dsTable;
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
return null;
}
}


//deserializes the string to a list. Utilizes JSON.net. RootObject is a class that contains the get and set for the JSON elements


public List<RootObject> DeSerializeJsonString(string jsonString)
{
//Initialized the List
List<RootObject> list = new List<RootObject>();
//json.net deserializes string
list = (List<RootObject>)JsonConvert.DeserializeObject<List<RootObject>>(jsonString);


return list;
}

RootObject 包含将获取 JSON 值的 get 集。

public class RootObject
{
//These string will be set to the elements within the JSON. Each one is directly mapped to the JSON elements.
//This only takes into account a JSON that doesn't contain nested arrays
public string EntityID { get; set; }


public string Address1 { get; set; }


public string Address2 { get; set; }


public string Address3 { get; set; }


}

创建上述类的最简单方法是使用 Json2charp，它将相应地对其进行格式化，并提供正确的数据类型。

The following is from another answer on Stackoverflow 同样，它没有考虑到嵌套的 JSON。

    internal static class ExtentsionHelpers
{
public static DataSet ToDataSet<T>(this List<RootObject> list)
{
try
{
Type elementType = typeof(RootObject);
DataSet ds = new DataSet();
DataTable t = new DataTable();
ds.Tables.Add(t);


try
{
//add a column to table for each public property on T
foreach (var propInfo in elementType.GetProperties())
{
try
{
Type ColType = Nullable.GetUnderlyingType(propInfo.PropertyType) ?? propInfo.PropertyType;


t.Columns.Add(propInfo.Name, ColType);


}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
}


}
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
}


try
{
//go through each property on T and add each value to the table
foreach (RootObject item in list)
{
DataRow row = t.NewRow();


foreach (var propInfo in elementType.GetProperties())
{
row[propInfo.Name] = propInfo.GetValue(item, null) ?? DBNull.Value;
}


t.Rows.Add(row);
}
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
}


insert.insertCategories(t);
return ds.
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);


return null;
}
}
};

最后，为了将上面的数据集插入到具有映射到 JSON 的列的表中，我使用了 SQL 批量复制和以下类

public class insert
{
public static string insertCategories(DataTable table)
{
SqlConnection objConnection = new SqlConnection();
//As specified in the App.config/web.config file
objConnection.ConnectionString = System.Configuration.ConfigurationManager.ConnectionStrings["foo"].ToString();


try
{
objConnection.Open();
var bulkCopy = new SqlBulkCopy(objConnection.ConnectionString);


bulkCopy.DestinationTableName = "dbo.foo";
bulkCopy.BulkCopyTimeout = 600;
bulkCopy.WriteToServer(table);


return "";
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
return "";
}
finally
{
objConnection.Close();
}
}
};

因此，上面的工作是将 JSON 从 webAPI 插入到数据库中。这是我的工作。但我绝不期望它是完美的。如果你有任何改进，那么请相应地更新它。

小开
最佳答案

只需像这样实例化 HttpClient:

HttpClientHandler handler = new HttpClientHandler()
{
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};


using (var client = new HttpClient(handler)) //see update below
{
// your code
}

2020年6月19日更新: 不建议在“ using”块中使用 httpclient，因为它可能导致端口耗尽。

private static HttpClient client = null;
    

ContructorMethod()
{
if(client == null)
{
HttpClientHandler handler = new HttpClientHandler()
{
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};
client = new HttpClient(handler);
}
// your code
}

如果使用.Net Core 2.1 + ，可以考虑使用 IHttpClientFactory并在启动代码中像这样注入。

 var timeout = Policy.TimeoutAsync<HttpResponseMessage>(
TimeSpan.FromSeconds(60));


services.AddHttpClient<XApiClient>().ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
}).AddPolicyHandler(request => timeout);
小开

我从下面的链接使用代码来解压缩 GZip 流。然后使用解压缩后的字节数组获取所需的 JSON 对象。希望对某人有所帮助。

var readTask = result.Content.ReadAsByteArrayAsync().Result;
var decompressedData = Decompress(readTask);
string jsonString = System.Text.Encoding.UTF8.GetString(decompressedData, 0, decompressedData.Length);
ResponseObjectClass responseObject = Newtonsoft.Json.JsonConvert.DeserializeObject<ResponseObjectClass>(jsonString);

Https://www.dotnetperls.com/decompress

static byte[] Decompress(byte[] gzip)
{
using (GZipStream stream = new GZipStream(new MemoryStream(gzip), CompressionMode.Decompress))
{
const int size = 4096;
byte[] buffer = new byte[size];
using (MemoryStream memory = new MemoryStream())
{
int count = 0;
do
{
count = stream.Read(buffer, 0, size);
if (count > 0)
{
memory.Write(buffer, 0, count);
}
}
while (count > 0);
return memory.ToArray();
}
}
}
小开

如果您的服务器使用另一种压缩方案，比如内容类型为: br 的“ Brotli”，那么您的代码迟早可能会中断

我用 clientHandler 制作了一个视频来处理 httpClient 中的解压缩:

compression algos handled by httpClientHandler


提交答案