Basic verification for responses

[sztomi]

This PR adds a new crate called integration_tests which has a build script that generates tests from all response variants. The tests have two forms:

For variants that don't hold data:

  #[test]
  fn validate_terminatethreads_response() {
    let resp = Response {
      request_seq: 1,
      success: true,
      message: None,
      body: Some(ResponseBody::TerminateThreads),
    };
    let value = resp_to_value(&resp);
    assert!(SCHEMA.is_valid(&value));
  }

For variants that hold data:

  #[test]
  fn validate_threads_response() {
    let resp = Response {
      request_seq: 1,
      success: true,
      message: None,
      body: Some(ResponseBody::Threads(ThreadsResponse::default())),
    };
    let value = resp_to_value(&resp);
    assert!(SCHEMA.is_valid(&value));
  }

So this relies on the Default impl for each type. Obviously, this will mean that any Options will be None so many fields are not actually tested. But this is a good starting point. syn can be used to discover the struct fields and set them to generate more kinds of tests.

DRAFT: currently I cannot make the tests fail by renaming a field completely so I'm doing something wrong.

[sztomi]

@pileghoff I can't add you as a reviewer apparently, but would you mind taking a look? (it might be that GH only allows adding maintainers which I would still be happy to add you as).

[pileghoff]

Other than two small things, it looks good 👍🏽

[pileghoff]

Also, feel free to add me as a maintainer 😀

[sztomi]

Finally, some progress (and lots of broken responses, unsurprisingly):

[sztomi]

@pileghoff the generated tests are now passing, please give this another go if you don't mind. Most of the time the required fix was not serializing None/null. While I think that shouldn't necessarily break the schema validation, I'm fine with doing this because the generated JSON will be smaller.

I will likely cut a release after this is merged.

For a bit more sophisticated testing I will generate structs with random data (that's not null). I don't know if there's already some crate out there that does that but in the worst case I can write a macro for it.

For testing the requests deserialization, a similar approach can be followed, but by generating JSON first and deserializing that.

[pileghoff]

I hope to look at it over the weekend 👍🏽 I was actually about to ask about making a new release with all the new fixes. I think after this is merged is a great time.

Regarding generating test data, would it be worth creating an example application that could talk to some of the most popular editors, so we could record the interaction and get some data that way? Then we could ensure that we were not only compliant with the spec, but that we also (sort of) works with those editors, in case they have some quirks or additions?

[sztomi]

Regarding generating test data, would it be worth creating an example application that could talk to some of the most popular editors, so we could record the interaction and get some data that way?

Good idea and probably more useful than a synthetic test for the requests. They might do things slightly differently and dap-rs should work with all of them.

[pileghoff]

The tests are passing, which is the most important thing 🥇 I have a few cosmetic suggestion, but its great to see how many errors you were able to catch here, so we dont have to fix them individually as users encounters them.

[pileghoff]

I think it looks good 👍🏽