OpenAPI/Swagger – Tweaking your JSON output

August 3, 2020 | By Brian | No Comments | Filed in: work.
Image by cwright1 from Pixabay

This week I got my first chance to use the OpenAPI spec, you might have heard it called Swagger. It’s a common way to define APIs. It’s great because it can autogenerate both client and server code, and even documentation on how to use the API that you create.

There is plenty to read about OpenAPI/Swagger, but I want to focus on one particular part that I struggled a bit with and talk through how I reconciled it.

Our application, like most these days, is expected to consume and produce JSON. This is no problem for OpenAPI/Swagger because it allows you to define Media Types as part of the definition. So you can easily return JSON, XML, or anything else.

I needed to return a list of Groups. For this writing, I will simplify the Group such that it only has a “name” and a “description”. So, if you look at the example on the Swagger site you can see how to define an array. It looks like this:

type: array
items:
  type: object
  properties:
    id:
      type: integer

So, for our list of Groups, we’ll tweak this to:

type: array
items:
  type: object
  properties:
    name:
      type: string
    description:
      type: string

This would produce JSON output that looks like the following :

[{
  "name": "MyName",
  "description": "Test description"
}, {
  "name": "Other Group",
  "description": "Test 123"
}]

Not bad!

But then I realized, I don’t want my JSON output to be a top-level Array. I always prefer to return Objects. This makes it easier to modify down the line if I need to add additional fields, like additional Objects/Arrays, etc. An example would be some metadata about the response, such as how long the request took, or how many total Groups there are.

I had trouble finding any rules that say “Don’t return an Array”. The closest thing i could find was this REST site that shows Objects being returned. Since this is what I’ve grown accustomed to, and because this is how Mirage JS returns its data, I figured I was on a good path.

If you have any links or documentation that talks about Best Practices around JSON Arrays, please share in the comments below!

So, what I really want my data to look like is:

{
  "groups": [{
    "name": "MyName",
    "description": "Test description"
  }, {
    "name": "Other Group",
    "description": "Test 123"
  }]
}

To make this happen, we need to tweak our YAML a bit. We’ll introduce an Object that wraps the Array:

type: object
properties:
  groups:
    type: array
    items:
      type: object
      properties:
        name:
          type: string
        description:
          type: string

And there you have it! The JSON that is returned is an Object with a “groups” Array ! Once I change my client code to use response.groups instead of response, everything works as expected.

I hope this has been helpful. I’ve written my own JSON responses in various languages, not really thinking too much about it. I now see the benefit of OpenAPI/Swagger, as it provides a robust, language agnostic, way to describe and define your APIs. This is especially important in a large enterprise. I expect to be doing a lot more with it in the future. Thanks for stopping by!

Tags: , , , ,

Introducing jaycue – jq in your JS

March 11, 2020 | By Brian | No Comments | Filed in: work.
jaycue logo
Shout out to my son for making the logo! Nice work Joseph!

Previously, I wrote about how useful lodash is when working with JSON in JavaScript. Then we took a look at how to deal with JSON from the command line using the power of jq.

It should come as no surprise where I’m headed here: I want the power of jq filters in my JavaScript code!

Off we go

Instead of doing what I normally do: googling to see if it existed, then getting depressed that it wasn’t unique, I decided to just do it. I thought it would be a fun way to do a little test-driven development, which I kinda miss doing.

I figured that if I could find a way to write a test and run it against my code, grab the output, run it against jq proper, then compare the results, it’d prove that my code was doing what it should.

I’ve executed shell commands from Node.js before, so it seemed doable.

> const exec = require('child_process').execSync;
undefined
> let command = `echo '{"foo": 123}' | jq ".foo"`
undefined
> exec(command).toString();
'123\n'

Cool. That was easy enough. So, in an attempt to make it more generic for a utility method that I can use in the tests, I should be able to do something like:

> let command = `echo '${json}' | jq "${filter}"`

and pass any JSON string and any filter, then collect up the result, and compare!

My goal is to make a function that would allow me to perform an expectation like:

expect(myjq(jsonObject, filter))
    .toEqual(jq(jsonObject, filter));

Check it out here: https://github.com/olore/jaycue/blob/master/tests/test-helper.js#L22

Now that we can test, let’s code!

Once that was in place, it was a matter of choosing which types of filters I wanted to support first. From the documentation, there are A LOT. I decided to go after the ones I would use most often, and expand from there. In particular, there was a single use case I wanted to solve. But to get there, I had to start with the basics.

The “basic” filters could be serviced by lodash.get. For instance, both “versionString” and “.versionString” will work with

lodash.get("versionString");

I would just need to chop off the leading period. Something like “name.firstName” would also work with lodash.get.

From there, I started down the list of Basic Filters. Adding in array indexing and eventually “select” filtering, which was the last piece of the puzzle for the use case I had in mind. 

Here it is:

{
  "applicants": [
    {
      "identities": [
        {
          "type": null,
          "number": null
        },
        {
          "type": "SSN",
          "number": "987651234"
        }
      ]
    }
  ]
}

We need to get the “number” whose type is “SSN”, if it exists.
We had code that looked like this:

const ssn = get(data, 'submit.applicants[0].identities', [])
  .reduce((accum, identity) => {
    if (identity.type === 'SSN' && identity.number) {
      accum = identity.number;
    }
    return accum;
  }, '');

Whereas, a jq command like this would work:

cat test.json | jq '.applicants[0].identities[] | select(.type=="SSN") .number'

Now that we had select functionality, this get/reduce JavaScrit code from above could be replaced with:

const ssn = jq(data, '.applicants[0].identities[] | select(.type=="SSN") .number');

And just like that, we’re successful!

Wrapping up

Please go install jaycue and let me know what you think:

npm install jaycue

A big shout-out to my 13yo son for making the jaycue logo! It was a true, family effort. Nice work Joseph!

I hope you have found this useful. I’d love to hear about what features of jq you think should be added next. And as always, I’d love to have you contribute to the jaycue project!

Tags: , , ,