NestJS materials

Write an awesome doc for NestJS tips, tricks, notes, things which are not in the doc (or are but not really obvious) and experimented to figure them out and use them.

View on GitHub

API Designing phase

Hero scenarios

Iteration on API

Contracts

When designing your RESTful API it is very wise to establish a well structured standard.

Headers & queries

Review process

  1. Does the API matches these general expectations:
    • Are the URLs properly formed?
    • Do they have idempotency?
    • Are they atomic?
    • Are the JSONs correctly formed?
    • Do they follow same casing (camelCase)?
    • Are you returning the right status code?
    • Is pagination implemented for lists?
    • Do you have setup your service for “long running operations”?
  2. Is it consistent with the rest of the APIs?
  3. Is it sustainable?
    • No breaking changes (keep it to a minimum).
  4. Check for performance issues.

Idempotency

[!TIP]

If you’re developing a client library for your API, try to define a timeout so that your frontend app (or whoever is using your client library) won’t hang indefinitely.

What’s idempotent

An endpoint is idempotent when retrying a request has the same intended affect, even if the original request succeeded. But the response might differ.

Method Description Status code
PUT Create/replace the whole resource. 200-OK/201-Created.
PATCH Modify the resource with Patch-merge. 200-OK.
GET Read the resource. 200-OK.
DELETE Remove the resource. 200-OK or 204-No content.

[!CAUTION]

For the DELETE if it did find the resource and deleted it we’re gonna return 200 HTTP status code.

But if it could not find the resource we should return 204 HTTP status code so that our client won’t retry. Remember that the whole idea behind idempotency was to allow our client retry the request if it did fail or did not receive a proper response from the server. But 404 add confusion to our idempotent RESTful API.

So the bottom line for DELETE is to avoid using 404 HTTP status code.

REST

CRUD

A complete example for airports resource.

  1. We have our database table/collection:

    Field Data type Set by Mutability
    airportCode string client create
    createdAt string service read
    runways number client update
    takeoffsPerHour number client update

    Mutability:

    • # “create” here means that our data is modifiable only at creation time, i.e. airportCode.
    • # “read” fields are the ones managed and controlled soly by server. So if you send it in a PUT/PATCH you’ll see an error.
      • E.g. createdAt.
      • Note: this is not the same as whitelisting. Here we wanna explicitly prohibit users from touching it. Can be ignored and just whitelisted since most of the APIs that we develop ain’t public facing APIs.
    • “update” signifies that you can change that field over the lifetime of a record/document. E.g. runways and takeoffsPerHour.

  2. Requests:

    Create

    PATCH /airports/LAX

{ "runways": 4 }

    Here if there were no records/documents with that same airportCode we’ll create one, otherwise we’ll modify the one with that airportCode. So our table now looks like this:

    airportCode createdAt runways takeoffsPerHour
    HND 2000-12-22 21:05:15.723 10 3000
    LAX 1994-12-22 21:05:15.723 4  

    Update

    PATCH /airports/LAX
content-type: application/merge-patch+json

{ "takeoffsPerHour": 2000 }

    So now with this HTTP request our resource’s state will be:

    airportCode createdAt runways takeoffsPerHour
    HND 2000-12-22 21:05:15.723 10 3000
    LAX 1994-12-22 21:05:15.723 4 2000

    Read

    GET /airports/LAX

    # The response body will contains LAX’s info:

    {
  "airportCode": "LAX",
  "createdAt": "1994-12-22 21:05:15.723",
  "runways": 4,
  "takeoffsPerHour": 2000
}

    List

    GET /airports/

    # The response body could look like this:

    {
  "data": [
    {
      "airportCode": "HND",
      "createdAt": "2000-12-22 21:05:15.723",
      "runways": 10,
      "takeoffsPerHour": 3000
    }
    {
      "airportCode": "LAX",
      "createdAt": "1994-12-22 21:05:15.723",
      "runways": 4,
      "takeoffsPerHour": 2000
    }
  ],
  "nextLink": "https://example.com/api/airports?cursor=f23as4GS126po5LK4Fdf"
}

    Note: While we are paginating our resources for a specific client, another client might add or remove something. That’s why it is crucial that our client is programmed robustly enough so it can handle skipped, or duplicated data.

    [!IMPORTANT]

    Learn more about pagination here.

    Delete

    DELETE /airports/LAX

    This will try to find a record/document in the database and remove it. Here we’re not returning any response and the response status code will be 200 or 204. Our database after this request:

    airportCode createdAt runways takeoffsPerHour
    HND 2000-12-22 21:05:15.723 10 3000

JSON resource schema

Processing of a PUT/PATCH request

  1. The preliminary validations:

    HTTP method Request contains HTTP status code
    PUT/PATCH Unknown/invalid JSON field names or values. 400-Bad request.
    PUT/PATCH Read fields. 400-Bad request.

  2. Now we have two scenarios:

    • The resource does not exist on the server:

      HTTP method Request HTTP status code
      PUT/PATCH Misses some mandatory fields for create/replace. 400-Bad request.

      And now that we’ve made sure the request is valid we can create it and return a 201-Created status code.

    • The resource exists on the server:

      HTTP method Request HTTP status code
      PUT/PATCH Contains create fields. 400-Bad request.
      PUT Misses some mandatory fields. 400-Bad request.

      If we passed all those steps then we can go ahead and modify/replace the record/document.

And if you’re wondering should you use 400 or 422 please read this Stackoverflow Q&A.

Processing an HTTP request in general

Scenario HTTP status code
Something unexpected happened. 500-Internal server error

Note that from here on out these check are done in the order that they are listed here, those who are defined first will be checked first.

Order Check HTTP status code
1 Server is booting/too busy/shutting down. 503-Service unavailable
2 HTTP version. 505-HTTP version not supported
3 Authorization. 401-Unauthorized
4 Client making too many requests per second. 429-Too many requests
5 URL too long. 414-URI too long
6 HTTP method is not supported. 501-Not implemented
7 Resource is not accessible to the client (permission). 403-Forbidden
8 Resource does not support this HTTP method at the moment. 405-Method not allowed
9 accept header is application/json. 406-Not acceptable

[!CAUTION]

I am not sure if we wanna perform all of these checks since I guess you’d agree with me that most of them ain’t necessarily a must for most RESTful APIs we design.

[!TIP]

How #6 is different from #8? I can shed light on this by explaining the #8 a bit more. Let’s say our server receives a PUT request from client1 and now is in the middle of processing that request, then clientB sends another request to delete that same resource. So in this case server can return 405 HTTP status code so that client can retry it after the previous request process is completed.

Note: In NodeJS we need to have some sort of mechanism to do it, I mean some sort of locking the resource.

Rest of remaining validations

HTTP method Check HTTP status code
GET If if-none-match header exists. 200 + requested resource only if it doesn't have an etag matching the given ones.
Or 304 if they match. Learn more about ETag here
PUT, POST, PATCH If content-length is missing. 411-Length required.
PUT, POST, PATCH If content-length is too big. 413-Request entity too large.
PUT, POST If content-type is application/json. 415-Unsupported media type.
PATCH If content-type is application/merge-patch+json. 415-Unsupported media type.
PUT, POST, PATCH If request body is a valid JSON. Although I had never done this part manually. I mean NestJS will throw a 400 error automatically if instead of JSON it receives something else. 400-Bad request.
PUT, PATCH If the passed ID is not creatable. TBF I am not 100% onboard with this idea. I would have throw a 400 error. My logic is that we're giving the control of ID creation to the client and as such if they're providing invalid ID it is their fault from where I stand. 404-Not found.
PATCH, DELETE If resource is not in updatable/deletable state. 409-Conflict.

Response body schema

Up until now we’ve kinda briefly touched this topic for the response body:

  1. When fetching a list of resources.
  2. Fetching a single resource.
  3. For update you could return the final state of the resource. It’ll be the same structure as for when client requests a single resource.
  4. For a delete operation, you could return nothing in the response body, or you can opt to return the state of the resource that was deleted.

  5. Finally for error we have two options:

    1. Let the framework take care of it for you. That’s what I do. I just let NestJS throw errors as it pleases.
    2. Customize it and follow one of these standards:

    Use error codes in your response beside the human readable error message.

[!CAUTION]

Notes:

  • Your error code should not change from version to version since it introduces breaking changes.
  • For the “RFC 7807” I did not find error code in the response.
  • You can repeat the error code in a custom response header so to lessen the work that your client had to do. Parsing response body is not necessary anymore in that case.

Resources

Etag

An ETag (Entity Tag) is a unique identifier assigned by the server to a specific version of a resource. When a client first requests a resource, the server can respond with the resource and an ETag header. This ETag value represents the current state of the resource (ETag changes whenever the resource is updated).

It can be:

Note: Adding it to the response body can prove beneficial since if you fetch a list of the resource, client needs to know ETag value for each resource.

Use cases:

The client stores the ETag value and sends it back to the server in subsequent requests using the If-None-Match header. The server then compares the ETag value sent by the client with the current ETag of the requested resource.

If they match, it means the resource has not changed, and the server can respond with a 304 Not Modified status, indicating that the client can use the cached version.

HTTP POST method or performing an action

POST HTTP method

Throttling client request

Scenario HTTP status code
Server is overloaded (quota > requests per unit of time)? 503-Service unavailable.
Tenant exceeds its quota? 429-Too many requests.

[!TIP]

Add a request id to each request and store the logs associated with that request ID so that when a customer opens a ticket you can trace back the requests and support them.

Include the request id to the response header, enabling users to share it with us.

[!TIP]

Send and log the telemetry. It identifies the request was made from:

  • Which SDK version? in which language? language version
  • Which OS, what was its version?
  • Etc.

You can have your own custom x-telemetry header or use user-agent header. Note that since this header os gonna stay the same for the same SDK, no matter how many times they are calling our API we might wanna log it only once. But if you wanna add additional data that maybe are specific to a certain situation, you can add them to a different custom request header.

Its structure can be something like this: AppName/AppVersion restApiNamesdk-programmingLanguage/sdkVersion (frameworkVersion; OSVersionAndType).

E.g. you-say-mobile/12.0.212 you-say-sdk-js/1.2.3 (Node 22.8.0; Manjaro; Linux x86_64; rv:34.0)

By storing these data we can answer question like:

  • How many customers calling into me are using JavaScript?
  • Which applications are using our API?
  • We had a bug in SDK version 1.2.3, we fixed it in version 1.3.3. How many users are still using SDK version 1.2.3?
  • And more useful info.

Distributed tracing between microservices

Versioning API

Considerations before introducing a new version

  1. New version should not break incoming requests.
  2. Your client should be able to adopt the new version without any code change. Unless they wanna use some of the newly added features.

[!NOTE]

API versioning is a misnomer. We’re justing adding new APIs and telling our users that the new version is preferred but NOT mandatory.

Another things is that our new version should run alongside the old ones.

Resource state should be backward & forward compatible. E.g. changing a string field max length is a bad idea since the new API version can pick up on that and will conform with that requirement but the older versions cannot do the same. So our app might crash or client’s app.

[!CAUTION]

If you have ever changed the API contracts (request body, query string, response body, etc) change the API version too. This prevents any potential confusion.

Glossary

What is OpenAPI?
A specification.
A YAML/JSON file.
It used to be called Swagger.
Enables you to describe how your service's contract look like.

Footnotes

  1. A specific use case or user journey that represents the primary or most common way your API will be used. It’s a scenario that captures the core functionality or the most important workflow that the API is meant to support. ↩

  2. A round-trippable value is convertible from string to a specific data type in a programming language and back to string from that data type again. A date string which follows ISO 8601 is round-trippable since you can convert it to a Date type in a programming language and convert it back to ISO 8601 string again. ↩