Difficulties in API design: Resource State

Posted by Riaas Mokiem on November 2, 2020

In this series of articles, I’d like to address some of the difficulties that occur during the design of REST APIs. This article is about difficulties with designing a resource that behaves differently under different circumstances. We call these circumstances its state. A REST API should present the resource state in a fundamentally different way than what most developers would already be familiar with. Because of this, developers will often present the resource state in a way that doesn’t adhere to the REST constraints. This in turn means we don’t get the benefits intended by those constraints.

What is the resource state?

For a REST API, the state of a resource determines what you can do with it. So for every state the resource can perform a unique set of actions. These actions may transition the resource to another state, allowing the resource to perform another set of actions.

Finding the resource states

Typically, you would create a state diagram for this. You can probably find many tutorials on how to exactly to do this. For example, below you can see a simplified state diagram for Calls (also known as incidents).

It clearly shows the actions that cause the resource to transition to another state. For example, the call becomes “in progress” if you respond to it. You can also complete and close it. These states each have a unique set of available actions.

  • Call: [Respond, Complete, Close]
  • Call in progress: [Complete, Close]
  • Completed Call: [Close]
  • Closed Call: [] (None)

You can now indicate the state of the resource by only exposing an action when you’re allowed to perform it. This is essentially how you use Hypermedia As The Engine of Application State (HATEOAS).

The emphasis here is not on identifying exactly which state the resource is in, though that is still possible. Instead, the emphasis is on what you can do with the resource, since that is what a client is most likely interested in.

The pitfall

So far we have a clear definition of what the resource state is, how to find it and how to represent it in the API. So what’s so difficult about this?

The problem is that people don’t always think about their resources in the way described above. They might think in terms of how to store the resources or what they want to show to their users in a GUI. Often, they will represent the resource state by just adding a field called “status” to the resource. The value of this field could then be “New” or “In Progress” or some other name for the resource state. With this approach, the actions that can be taken are usually not included with the resource so each client must implement this based on the resource documentation.

The consequences

While the solution above does allow you to identify the resource state quite clearly, it also causes some problems. One of the problems, as this Devoxx presentation shows, is that the resource presents the status in English. If you want to support different languages, you now need to map this English status text to those other languages in your client or your API needs to maintain all possible translations for the status field for all clients of the API or apply some other workaround.

You could still deal with this by not using text in the “status” field. You could just use numbers, “status” : 1 or “status”: 2, that imply a specific state that you describe in your resource documentation. This effectively forces the client to always map this status to some translated text defined by the client.

This works to a certain extent and many APIs do work like this. However, this still makes the client responsible for understanding what actions are possible in each state. So each client needs to implement this separately.

Using the example for Call, as described in the state diagram above, we may want to remove the option to “complete” a call. Instead, we decide to always “close” it directly. If this happens, the client will likely cause some errors when it incorrectly allows its users to complete calls.

The problem

The consequences described above may not seem that big of a problem. And, as described, you can indeed reduce the problem quite a bit. This may be why this approach is still acceptable for many existing APIs.

The problem becomes more clear when you consider that multiple clients will be accessing the API. Each of these clients would include some logic for when an action is allowed. Inevitably, many of those clients will contain outdated or simply incorrect logic.

This becomes a problem with the independent evolvability of your API. Every client needs to duplicate some of the server’s logic, the part that determines when an action is allowed. This increased coupling makes it costly to change that logic on the server.

The solution

The solution was already explained at the beginning of this article. You adhere to the REST constraint of HATEOAS by providing only the allowed actions as part of your resource representation.

The resource no longer explicitly shows what state it is in. It only shows the actions that are allowed. This means that clients never have to include their own logic that determines when an action is allowed. This decouples the client from the API, allowing the API to evolve independently.

If the client still wants to explicitly describe the resource state, it can deduce this from the allowed actions in the resource. The actual text used to represent the resource state in the client would be no different from any other user-facing text defined by the client. So the client can use its existing translation mechanism (like i18next for Javascript frontends) to ensure that the resource state is described correctly in different languages.

Is it worth it?

This solution might seem like only a minor improvement. The consequences of the problem can be very bad but you can improve your situation to a certain extent. Emphasis on “to a certain extent“. There may come a point at which you need to solve this problem to allow your API to continue evolving and improve its scalability. But the problem is actually a design flaw. you would have to redesign your API to solve it. This would be very costly and have a huge impact. So if you want to avoid that, it should be worth it to get this right from the start.

Overcoming the difficulties

Here are some tips that may help you overcome this difficulty during your next API resource design:

  • When designing your resource, create a state diagram that describes its states and state transitions (actions).
  • Include only the allowed actions in each resource representation.
  • Use a resource representation format (i.e. media type) that allows clients to clearly identify the actions on each resource.

Difficulties in API design: Domain

Posted by Riaas Mokiem on April 24, 2020

In this series of articles, I’d like to address some of the difficulties that occur during the design of REST APIs. This article is about difficulties with determining and understanding the domain. In the previous article, I discussed the difficulties with resources, which you should express in terms of the domain. This requires you to determine what domain is relevant to your API and understand it fairly well. This makes the domain an important part of API design.

What is a domain?

In general, a domain refers to an area of expertise or knowledge. In practice, the domain provides the most specific, commonly understood description of functionality. You can see that there is a balance here. A domain is both specific and commonly understood. You can also look at how the domain is defined, to better understand what it is.

How do you define a domain?

One way to define a domain is through practical experience. This requires people that have achieved a common understanding of the specifics in their domain. They can achieve this by spending a long time interacting with many different people in that domain. You typically call these people domain experts. The resulting domain would still be somewhat vague as it is a personal interpretation of each domain expert. If you consult multiple such domain experts, you can still gain a good understanding of the domain.

Another way to define a domain is through standardization. There may be commonly used standards that express their functionality in specific ways. Since people commonly use these standards, they will also understand its terminology. This allows you to define the domain much more concretely as it is based on standards. You can also understand the domain much faster as it only requires an understanding of the relevant standards and protocols, making it easier to become a domain expert.

In practice though, you often define the domain through a combination of the two. There are often competing standards for the same functionality. This means there isn’t a single standard that is common enough to provide a common understanding of the domain. So it requires people that can abstract away what these standards have in common, how they are applied in practice and what may still be missing from them. These people can use their experience to then define the domain. This also shows that the actual definition of a domain will usually be inherently vague, sometimes to the extent that you can’t even provide a name for the domain (though this is usually not a problem).

Why use the domain for REST APIs?

The resources in REST APIs define a contract to its consumers. It is important to describe them in a way that is understandable to these API consumers and that does not violate the contract. Since you can express functionality in a way that people will commonly understand by using the domain, we can allow our API consumers to understand our resources by expressing them that same way.

The domain also provides a very wide view of its functionality. It is not limited to the interpretation of a single person or software implementation. This means that the domain includes all functionality that is relevant within the domain, regardless of whether your own product includes that functionality as well. When you express your resources in terms of the domain this has another benefit. If you need to implement more functionality from that domain, your resources will allow for this. This means you do not need to violate the contract with your API consumers to evolve your API. In practice, it is impossible to ensure that you never have to violate your contract but it’s still good to reduce how often you need to do this as much as possible.

Another reason to use the domain when designing REST APIs is to easily learn how things are supposed to work in that domain. This provides a better understanding of what you need to implement in your product and for your API and should improve the implementation as a whole.

Domain Engineering

Domain engineering (or product line engineering) is a method of software development that has a strong focus on achieving software reuse. The domain is the key to this software reuse, with each system only providing a variation on the domain-specific implementation. This aligns nicely with the REST constraint of Uniformity of Interface which requires you to create your architecture as general as possible. This allows for as much software reuse as possible, just like with domain engineering. This software reuse has the added benefit of not only increased productivity but also higher consistency.

The design process for REST API resources follows a process similar to domain engineering. This shows that some of the techniques from domain engineering are applicable to our design process. But it also shows that the domain is not just relevant for REST APIs. You can apply it to the design and implementation of any product. So it’s clearly beneficial to gain a deeper understanding of the domain whether you’re implementing a REST API, a standalone application or something else entirely.

What is your domain?

When you apply the software engineering principle of Separation of Concerns to your product, its API will likely only cover a single domain. It will usually also need to interact with other domains though. In practice, you may not always be able to name your domain. If you can name or describe the domain, you can look for domain experts for that domain. If you can’t name or describe the domain, you can look for people with a lot of experience with the functionality you’re implementing, from a wide range of use cases (like consultants or specialists from a support department, not the end-users of the functionality). These people are likely to be domain experts and you can rely on them as such. These domain experts can help you define the boundaries of your domain and determine exactly what types of interactions you may have with other domains.

Overcoming the difficulties

Here are some tips that may help you overcome this difficulty during your next API resource design:

  • Identify the main domain for your API early on.
  • Find and involve domain experts early on.
  • Understand your main domain early on.
  • Look into the techniques used in domain engineering to improve your design process.