Is this a bug in json::value::parse?

Jun 3, 2015 at 5:13 AM
Edited Jun 3, 2015 at 2:42 PM
I have stumbled upon what looks like a bug in json::value::parse...
I was trying to parse a JSON object, which was returned by HTTP GET:
        concurrency::streams::istream bodyStream = response.body();
        concurrency::streams::container_buffer<std::string> inStringBuffer;
        auto readTask = bodyStream.read(inStringBuffer, 100) // 100 is too much!
        .then([](size_t bytesRead)
        {
            std::string responseBody = inStringBuffer.collection();
            // responseBody.resize(bytesRead); // <--- uncomment this to prevent parse error
            const utility::string_t responseUStr = to_string_t(responseBody);
            web::json::value json = web::json::value::parse(responseUStr);
parse failed because responseBody contained a bunch of trailing '\0' characters.
If I trim it down to bytesRead, everything works fine and parse succeeds.
But shouldn't it disregards trailing '\0'?

This problem can be reproduced in non_default_locale test (one of only two places in JSON tests suite, where json::value::parse is invoked!) by replacing
    utility::string_t str(U("[true,false,-1.55,5,null,{\"abc\":5555}]"));
with
    std::string s("[true,false,-1.55,5,null,{\"abc\":5555}]");
    s.resize(s.length()+2, '\0'); // comment this line and the test succeeds
    utility::string_t str = utility::conversions::to_string_t(s);
Is this a bug in web::json::value::parse or in utility::conversions::to_string_t?

Also, how should one specify the maximum number of characters to read in concurrency::streams::istream::read to avoid the trailing '\0' in the result?

Thanks,
Boris
Coordinator
Jun 4, 2015 at 10:39 PM
Hi Boris,

Ok there are a couple of things here so let me see if I can get them all :)

I don't consider this a bug in our JSON parser, you are passing in a string of the form "[true, false]00". While technically the zeros are after the JSON array value is closed I'd consider this still invalid. Perhaps we could ignore all characters after the top level JSON array or object is closed, but I think this is of little value.

Regarding JSON parsing tests, when I search for json::value::parse we have a lot more than just two tests covering, there are tests in json_parsing.cpp, to_and_as_operator_tests.cpp, negative_parsing_tests.cpp, iterator_tests.cpp, etc...

Ok so regarding what you are trying to accomplish, it sounds like you want to get a json::value from the response body of an HTTP GET request. The easiest way is to use the http_response::extract_json() API. All of the code you have there would become something like this:
web::json::value json = response.extract_json();
If you don't want to use extract_json() there are several other options, like reading directly from the underlying response stream. This could be done using an API like read_to_end(). Another option is to set the exact buffer you want to store the response in with the http_request::set_response_stream(...) API. Using a container_buffer backed by a string you could directly have the HTTP response body written into your string.

Steve