Notes on Roy Fielding's REST dissertation

Software Architecture has two unique but alternately useful meanings. The first is how the in-the-silicon processing that the programs in a system perform, called the run-time abstraction [Ch1.1]. The second, which Roy Fielding uses in his dissertation to motivate the development of HTTP, is the layering of system constraints to create increasingly focused architectural styles [Ch1]. As a practicing engineer, the approach to software architecture as descriptions of constraints and styles allows projects to consider their designs and possible implementations before the expensive task of coding. After a system is deployed and as it grows, teams necessarily begin using the runtime architecture model to understand, monitor, and troubleshoot the application.

Fielding’s REST

Fielding’s research was motivated by the first decade of the Internet, to his writing in 2000. Through this time, the Internet transitioned from a network of research institutes to a network of the general public. This distributed (and in many ways anarchic) collection of actors pushed Fielding to look for architectural patterns and styles that could be resilient given such a shared resource. The Internet is a global, distributed, hypermedia system. The Internet is defined by its distributed nature, variable latency & low reliability network connections, multiple trust boundaries, and large grain data objects [Pg3]. In creating REST, Fielding is looking to describe a set of constraints that will create a useful style for the Internet, its components, and their connectors [Ch1.2].

Fielding provides a number of criteria by which to evaluate constraints when layering new architectural styles [Ch3]. Many of these criteria revolve around system performance, both end-user perception [Ch2.3.1] as well as system-as-a-whole global resource management [Ch2.3.2]. Other criteria aim to reduce the burden to engineers of Internet software. Engineers find value in "architectural styles as a method of abstraction" [Pg15], and modifiability constraints seek to capture those engineering practices that are replicable [Ch2.3.4].

Examining Network-based Architectural Styles with these criteria in mind guide a systems architect to possible formulations of a hypermedia system. Beginning with the null constraint, every additional layer must first meet the functional requirements of a hypermedia system before adding additional benefits to the Internet. Because the system is widely geographically distributed, a client/server architecture is a natural first constraint [Ch3.4.1,5.1.3]. This extends directly to larger systems using a layered approach, where in the process of serving one client request, the server itself becomes a client and requests data from a second server. From here, REST begins differentiating itself from other network styles like CORBA by adopting the stateless constraint [Ch5.1.2]. This is deemed a critical constraint for Internet systems because of the (lack of) reliability of underlying systems in the WWW. While being stateless does increase both message size and number of messages, it allows adopting a number of styles that make up the performance penalty, primarily Cache [Ch5.1.4]. Other constraints potentially harm performance, but allow additional capabilities of the system which are worth the potential costs. Providing Code on Demand styles (via javascript, css, and similar progressive enhancement tools) gives Internet authors many multimedia options for their content. The performance gains from the constraining styles in many ways allow the Internet to have the resources for progressive enhancement.

There are, however, criteria and constraints not discussed in the paper that are conspicuous in their absence. Implicit throughout the paper is that publishing happens out of band on the Internet [Ch4.1], "since write actions using the Web are extremely rare" [Pg129]. Security is not considered a core architectural constraint, and instead "participants in an application interaction should either assume that any information received is untrusted or require some additional authentication' [Pg70]. Realizing this is left entirely to the mediating connectors [Ch2.3.5, 3.4.2, 4.1.4, 5.2.3]. Privacy is mentioned briefly in describing why cookies violate REST [Ch6.3.4.2], but no discussion is provided to describe why privacy would be desirable and only the briefest note on how it might be achieved.

Specific Notes:

Ch 2.3 Software Architecture Features: Fielding presents too many overlapping -ilities, while missing important -illities (Accessibility, Security, Privacy).
Ch 3.1 Architectural Styles: Overall a good approach, but the per-ility analysis is lacking because of the poor choice of non-functional requirements.
Ch 3.2.2 Uniform Pipe: UNIX Pipes have a uniform interface at the Presentation layer equivalent (line-oriented streams), but do not have a uniform interface at the Application layer (what’s in the lines). This is the first of several times Fielding makes this mistake conflating Presentation and Application concerns, and unfortunately it is a pernicious oversight in the industry.
Ch 3.4.3 Client Stateless Server: As defined, has issues with privacy and security, though they can be sort of rectified with shimming in Encryption and secure access tokens. It’s also ambiguous whether it’s Client (Stateless Transport) Server, or Client (Stateless Server). The former, including all necessary state in the transport method, is a widely adopted constraint. The latter depends on whether the layering is integral. This pattern should work very well for a serverless API Gateway architecture with a backing event broker.
Ch 3.4.4 Client Caching Stateless Server. As defined, cannot have privacy and security, as that would prevent caching. Several variants could allow privacy and security, but would probably need to drop “Stateless” from the architecture.
Ch 3.4.5 Layered Client Caching Stateless Server: Here we get to the reality of most network distributed system, with the caching being “optional” (and, really, relegated to content delivery network subsets of the network architecture). Most API design isn’t naively cacheable.
Ch 3.4.7 Remote Data Access: This is, on top of LC$SS, most API design “in the wild.” “The disadvantages are the client needs to understand the same database manipulation concepts as the server implementation” is not correct. The client must understand the facade the API exposes. For an SQL server, that certainly is the same as the manipulation itself, but that’s not the only way to implement RDA.
Ch 3.5 Mobile Code: originally, this described including JavaScript resources for the server to run on the client; this has largely been superseded by Single Page Application bundles and distributions (again showing the separation between “Content” REST and “API” Rest)
Ch 5.1.5 Uniform Interface: uniform at the transfer (presentation) layer, not at the application layer.
Ch 5.1.6 Layered: this is a good architecture to point out, but in practice, we’ve moved far past the three-tier + caching architecture and far into distributed meshes. Though from the point of view of a trace, it’s still LCS$?S?S.
Ch 5.1.7 Code on Demand: again, only for the client. See notes on Ch 3.5.
Ch 5.2.1.1 Resources and Resource Identifiers: Resource identifiers do not allow for ad-hoc querying or client-directed resource derivation. However, that may better describe GraphQL as a different architecture given the constraints. Still, URIs are probably the most fundamental feature of a REST-like API. Note: This also constrains servers that once a URI, always a URI. (Do all URIs exist in the platonic ideal of URIs, just some are 404s on some servers? 🤯)
Ch 6.2.2 Manipulating Shadows: There’s a very interesting distinction between “Resources” and “Representations”. This is probably the thing that is most overlooked, but after URIs, most fundamental to “Representation” in REST.
6.2 REST Applied to URI: this is really the set of problems an API Generator tackles. Allowing a team to define a domain model for the connector, and generate code for both the client and server (and possibly intermediate caches) to safely marshal into and out of the connector.
6.3 REST Applied to HTTP: this entire section is a red herring. REST is an L7 Application concern; HTTP (and gRPC) are L6 Presentation concerns. Or: REST describes the interface of the connector, HTTP (and gRPC) are the implementation of the connector.
6.5.2 HTTP is not RPC: of course not. See above. RPC is an application concern, and the implementation details of the connector being HTTP or RMI or gRPC are unrelated concerns.
6.5.2b REST is not RPC: “Requests are directed to resources using a generic interface with standard semantics that can be interpreted by intermediaries”. It is not unreasonable to approach a procedure call as a resource. The identifier is the procedure and the input arguments; the resource is the algorithm it wraps, and the representation is the return value of the procedure. Describing this using REST can certainly be contortions - not all RPCs easily fit into REST’s architecture, especially highly stateful procedures. This model works wonderfully, though, when applied to 6.2.3 Remote Authoring, as Remote Authoring is a well-understood set of RPC actions that fit the REST architectural model.

REST as RPC

As laid out, REST is intended as a hypermedia retrieval system. With the introduction of AJAX and Web2.0, this paradigm is flipped on its head. It is interesting that without significant effort, HTTP as designed following the REST style still works in Web 2.0. I find myself most satisfied when I come across a question that I had never before considered for a software system, and the existing architecture has a clear and direct solution. While the details of POST/PUT/PATCH semantics cause headaches, that they existed in the HTTP spec to begin with provides a clear and direct solution for using HTTP as a Resource RPC tool. With only a few more constraints on how the HTTP primitives are assembled, REST has naturally grown from the document distribution system Fielding describes in 2000 into the core of 21st century Internet.

Resource Identifiers

REST URIs are left as-is. A URI when using REST as an RPC still "identifies a concept rather than a document" [Pg111]. The first constraint moving to REST as RPC is applying a structure to the path portion of a URI. This structure creates a resource hierarchy, creating an alternating pattern of resource_type and resource_id. Each resource_type in the hierarchy serves as a collection, possibly constrained in a tree structure of its parents, and a resource_type/resource_id path component pair identify a resource uniquely. When requesting this resource, the client and server can treat it in any number of formats. The representation of the resource, as transmitted, is negotiated but retains "the semantics of what the author [or service] intends to identify" [Pg111]. These representations can contain additional metadata about the resource.

Verbs, Common and Otherwise

HTTP request methods GET and HEAD are, like much of HTTP1.0, designed with distributed document retrieval in mind. Thus, they are designed with the REST constraints described by Fielding. What isn’t described is how the modifying methods POST, PUT, PATCH, and DELETE are RESTful. Using REST as an RPC mechanism requires the modifying methods to also be stateless. This is achieved by treating the body of the method as a representation of the resource that is identified by the request’s URI. When a client wishes to modify the server’s understanding of a resource, it does so by preparing an appropriate HTTP request and filling the payload with a representation of the data in a format the service understands. It then sends that request to the server, which (if the request is allowed via authentication policies), the server can update its internal storage for the identifier with the new data. While the semantics require care when implementing, it is straightforward to map common RPC commands of Create, Retrieve, Update, and Delete to POST, GET, PUT/PATH, and DELETE directly. Custom verbs can be achieved by defining resource types specific to that action, and rephrasing it in the common commands.

HATEOAS vs Resource Discovery

Hypertext as the Engine Of Application State [roy] and Resource Discovery [disc] are alternative architectural styles to describe the shape of a REST API. In Resource Discovery, a REST RPC Service publishes a single document that contains the schema of the entire API. This includes data types, resource hierarchies, available RPC methods (both common and custom), formats for constructing URIs, and other metadata like access control rules. HATEOAS distinguishes itself by including resource information in the API responses themselves. When representing a resource, HATEOAS includes structured metadata about related resources. This includes what the resource is intended to be used as, and critically, the URI of the resource. By including this information with every resource, HATEOAS can in effect include valid business rules available to the resource at a given point in time. Discovery APIs allow a client to retrieve the schema a single time, and make requests with a known structure against a stable API. This can reduce the payload size of each request, as the discovery schema contains all the information necessary to know what valid URIs may be, but is unable to encode business logic like HATEAOS. HATEAOS provides increased flexibility (in that representations may skew separately from their identifier) and increased accuracy (in that only valid transitions are included in the representation), but does have increased overhead in larger message sizes for every request.

REST is many things

REST started as a motivating philosophy for developing the HTTP specification. Its foundation is the separation from identifier to content, and the separation of content and representation. These architectural constraints allowed the Internet to anneal around a set of protocols that could handle its distributed, anarchic nature gracefully and resiliently. Since that formulation, the Internet has seen a radical transformation, from a document sharing system requiring moderate technical investment into an application delivery platform powering a significant portion of the worldwide economy & communications. With that growth, so to has the concept of the REST architecture grown from a hypermedia retrieval architecture into a complete RPC mechanism. Through that growth, its core constraint of stateless requests meets the needs of the ongoing, evolving Internet.

But if you need a definition for a textbook, I propose this:

REST is a range of software architectures that harness and exploit the mechanics of HTTP/1.1 and related transfer protocols. These include as the minimum architectural constraints layered client/server, URI addressing, & state management being restricted to authentication proof, and may include caching and advanced flow control from HTTP/2,3 or gRPC to include multiple Resource Representations in a single transport Request.

References

"Architectural Styles and the Design of Network-based Software Architectures." Fielding, Roy. 2000.
"REST APIs must be hypertext-driven." Fielding, Roy. 20 Oct. 2008.
"Discovery Document" Google, 28 Oct. 2020, developers.google.com/discovery/v1/reference/apis.