jQuery read AJAX stream incrementally?

小开

最佳答案

You're going to want to use straight up javascript for this. The reason is that you're going to want to continuously poll and not wait for the callbacks to fire. You don't need jQuery for this, it's pretty simple. They have some nice source code for this on the Ajax Patterns website.

Essentially, you'll just want to keep track of your last position in the response and periodically poll for more text past that location. The difference in your case is that you can subscribe to the complete event and stop your polling.

小开

Since you say your server is stream friendly (asynchronous) and was looking for a jquery solution, have you checked out the jQuery Stream Plugin?

It is really easy to use and allows you to not really worry about much of anything. It has pretty good documentation as well.

小开

Use XMLHttpRequest.js

https://github.com/ilinsky/xmlhttprequest

http://code.google.com/p/xmlhttprequest

Delivers unobtrusive standard-compliant (W3C) cross-browser implementation of the XMLHttpRequest 1.0 object
Fixes ALL browsers quirks observed in their native XMLHttpRequest object implementations
Enables transparent logging of XMLHttpRequest object activity

To use long polling with PHP:

output.php:

<?php
header('Content-type: application/octet-stream');


// Turn off output buffering
ini_set('output_buffering', 'off');
// Turn off PHP output compression
ini_set('zlib.output_compression', false);
// Implicitly flush the buffer(s)
ini_set('implicit_flush', true);
ob_implicit_flush(true);
// Clear, and turn off output buffering
while (ob_get_level() > 0) {
// Get the curent level
$level = ob_get_level();
// End the buffering
ob_end_clean();
// If the current level has not changed, abort
if (ob_get_level() == $level) break;
}
// Disable apache output buffering/compression
if (function_exists('apache_setenv')) {
apache_setenv('no-gzip', '1');
apache_setenv('dont-vary', '1');
}


// Count to 20, outputting each second
for ($i = 0;$i < 20; $i++) {
echo $i.str_repeat(' ', 2048).PHP_EOL;
flush();
sleep(1);
}

run.php:

<script src="http://code.jquery.com/jquery-1.6.4.js"></script>
<script src="https://raw.github.com/ilinsky/xmlhttprequest/master/XMLHttpRequest.js"></script>


<script>
$(function() {
var xhr = new XMLHttpRequest();
xhr.open('GET', '/longpoll/', true);
xhr.send(null);
var timer;
timer = window.setInterval(function() {
if (xhr.readyState == XMLHttpRequest.DONE) {
window.clearTimeout(timer);
$('body').append('done <br />');
}
$('body').append('state: ' + xhr.readyState + '<br />');
console.log(xhr.responseText);
$('body').append('data: ' + xhr.responseText + '<br />');
}, 1000);
});
</script>

This should output:

state: 3
data: 0
state: 3
data: 0 1
state: 3
data: 0 1 2
state: 3
data: 0 1 2 3
state: 3
data: 0 1 2 3 4
...
...
...
state: 3
data: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
state: 3
data: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
state: 3
data: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
done
state: 4
data: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

For IE you need to look into XDomainRequest

http://blogs.msdn.com/b/ieinternals/archive/2010/04/06/comet-streaming-in-internet-explorer-with-xmlhttprequest-and-xdomainrequest.aspx

http://msdn.microsoft.com/en-us/library/cc288060(VS.85).aspx

小开

This is quite straightforward when outputting text or HTML. Below is an example.

(You'll run into issues if trying to output JSON however, which I'll tackle further down.)

PHP FILE

header('Content-type: text/html; charset=utf-8');
function output($val)
{
echo $val;
flush();
ob_flush();
usleep(500000);
}
output('Begin... (counting to 10)');
for( $i = 0 ; $i < 10 ; $i++ )
{
output($i+1);
}
output('End...');

HTML FILE

<!DOCTYPE>
<html>
<head>
<title>Flushed ajax test</title>
<meta charset="UTF-8" />
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js"></script>
</head>
<body>
<script type="text/javascript">
var last_response_len = false;
$.ajax('./flushed-ajax.php', {
xhrFields: {
onprogress: function(e)
{
var this_response, response = e.currentTarget.response;
if(last_response_len === false)
{
this_response = response;
last_response_len = response.length;
}
else
{
this_response = response.substring(last_response_len);
last_response_len = response.length;
}
console.log(this_response);
}
}
})
.done(function(data)
{
console.log('Complete response = ' + data);
})
.fail(function(data)
{
console.log('Error: ', data);
});
console.log('Request Sent');
</script>
</body>
</html>

What if I need to do this with JSON?

It's not actually possible to load a single JSON object incrementally (before it's fully loaded) because until you have the complete object, the syntax will always be invalid.

But if your response has multiple JSON objects, one after another, then it's possible to load one at a time, as they come down the pipe.

So I tweaked my code above by...

Changing PHP FILE line 4 from echo $val; to echo '{"name":"'.$val.'"};'. This outputs a series of JSON objects.
Changing HTML FILE line 24 from console.log(this_response); to
```
this_response = JSON.parse(this_response);
console.log(this_response.name);
```
Note that this rudimentary code assumes that each "chunk" coming to the browser is a valid JSON object. This will not always be the case because you cannot predict how packets will arrive - you may need to split the string based on semi-colons (or come up with another separator character).

Don't use `application/json`

Do NOT For change your headers to application/json - I did this and it had me Googling for 3 days. When the response type is application/json, the browser waits until the response is complete, as in fully complete. The full response is then parsed to check if it is infact JSON. However our FULL response is {...};{...};{...}; which is NOT valid JSON. The jqXHR.done method assumes there was an error, because the complete response cannot be parsed as JSON.

As mentioned in the comments, you can disable this check on the client side by using:

$.ajax(..., {dataType: "text"})

Hope some people find this useful.

小开

Here is a straightforward way to achieve this using JQuery (as requested by the OP):

First, extend the ajax object to support onreadystatechange by running the below code from https://gist.github.com/chrishow/3023092 (appended at the bottom of this response). Then just call ajax using an onreadystatechange function that will check xhr.responseText for new text.

If you wanted to get even fancier, you could clear the responseText data each time you read it, such as described here).

For example, see https://jsfiddle.net/g1jmwcmw/1/, which will download the response from https://code.jquery.com/jquery-1.5.js and output it in chunks inside your console window, using the code below (which you can just copy into an html page and then open in your browser):

<!-- jquery >= 1.5. maybe earlier too but not sure -->
<script src=https://code.jquery.com/jquery-1.5.min.js></script>
<script>
/* One-time setup (run once before other code)
*   adds onreadystatechange to $.ajax options
*   from https://gist.github.com/chrishow/3023092)
*   success etc will still fire if provided
*/
$.ajaxPrefilter(function( options, originalOptions, jqXHR ) {
if ( options.onreadystatechange ) {
var xhrFactory = options.xhr;
options.xhr = function() {
var xhr = xhrFactory.apply( this, arguments );
function handler() {
options.onreadystatechange( xhr, jqXHR );
}
if ( xhr.addEventListener ) {
xhr.addEventListener( "readystatechange", handler, false );
} else {
setTimeout( function() {
var internal = xhr.onreadystatechange;
if ( internal ) {
xhr.onreadystatechange = function() {
handler();
internal.apply( this, arguments );
};
}
}, 0 );
}
return xhr;
};
}
});


// ----- myReadyStateChange(): this will do my incremental processing -----
var last_start = 0; // using global var for over-simplified example
function myReadyStateChange(xhr /*, jqxhr */) {
if(xhr.readyState >= 3 && xhr.responseText.length > last_start) {
var chunk = xhr.responseText.slice(last_start);
alert('Got chunk: ' + chunk);
console.log('Got chunk: ', chunk);
last_start += chunk.length;
}
}


// ----- call my url and process response incrementally -----
last_start = 0;
$.ajax({
url: "https://code.jquery.com/jquery-1.5.js", // whatever your target url is goes here
onreadystatechange: myReadyStateChange
});


</script>

小开

I had to supply a grid with a large JSON payload that kept running into the maximum allowed size limit. I was using MVC and jquery, and so I adapted the solution of AlexMorley-Finch above.

The server code was from "Streaming data using Web API". Also https://github.com/DblV/StreamingWebApi.

public class StreamingController : ApiController
{


[HttpGet]
[ActionName("GetGridDataStream")]
public HttpResponseMessage GetGridDataStream(string id)
{
var response = Request.CreateResponse();
DynamicData newData = new DynamicData();
var res = newData.GetDataRows(id);
response.Content = new PushStreamContent((stream, content, context) =>
{
foreach (var record in res)
{
var serializer = new JsonSerializer();
using (var writer = new StreamWriter(stream))
{
serializer.Serialize(writer, record);
stream.Flush();
}


// Thread.Sleep(100);
}


stream.Close();
});


return response;
}
}

This created a stream of {json object}{json object}{json object} that needed delimiting commas and surrounding [ ] to be parsed as json successfully.

The client code was supplied the missing characters thus:

 var jsonData = {};


$.ajax("api/Streaming/GetGridDataStream/" + viewName, {
xhrFields: {
onprogress: function (e) {
// console.log(this_response);
}
}
}, { dataType: "text" }) //<== this is important for JSON data
.done(function (data) {


data = "[" + data.replace(/\}\{/gi, "},{") + "]";


jsonData["DataList"] = JSON.parse(data);
//more code follows to create grid
})
.fail(function (data) {
console.log('Error: ', data);
});

I hope this helps someone using .Net MVC and jQuery.

jQuery read AJAX stream incrementally?

PHP FILE

HTML FILE

What if I need to do this with JSON?

Don't use application/json

Don't use `application/json`