PSR-7 is imminent, and here's my issues with it.
PSR-7 is pretty close to completion. PSR-7 is a new ‘PHP standard recommendation’, put out by the PHP-FIG group, of which I’m a member of.
It describes how to create PHP representations of a HTTP Request and a HTTP response. I think the potential impact of PSR-7 can be quite large. If large PHP framework authors are on board (and at least some of them are), it means that these interfaces might in the future be used indirectly or directly by an extremely large portion of the PHP community.
PSR-7 gets a lot of things right, and is very close to nailing the abstract data model behind HTTP, better than many other implementations in many programming languages.
But it’s not perfect. I’ve been pretty vocal about a few issues I have with the approach. Most of this has fallen on deaf ears. I accept that I might be a minority in feeling these are problems, but I feel compelled to share my issues here anyway. Perhaps as a last attempt to sollicit change, or maybe just to get it off my chest.
If anything, it will allow me to say ‘I told you so’ when people start to using it and run into the edge cases that it doesn’t cover well.
PSR-7 doesn’t just represent a HTTP request and HTTP response in PHP, it tells you how to build your HTTP application.
Immutability
More recently in the process the decision has been made to make the objects immutable. This means that after the objects have been created, they are set in stone and cannot be changed.
In practice, this means instead of this:
<?php
$response->setHeader('X-Powered-By', 'Captain Planet');
?>
We need to do:
<?php
$response = $response->withHeader('X-Powered-By', 'Captain Planet');
?>
The difference is small in this isolated example, but the impact is massive.
One obvious issue is that for every change that you want to make to request or response objects, an entirely new instance needs to be created.
This bit of code creates a total of 4 copies of the request.
<?php
$request = $request
->withMethod('POST')
->withUrl(new Url('http://example.org/')
->withHeader('Content-Type', 'text/plain');
?>
The real impact in ‘time spent’ was proven to be quite low, so this part of the argument doesn’t really bother me. Cloning objects is apparently pretty cheap in PHP.
What bothers me a bit more is that this is a pretty major departure of how we are used to using these objects. Most PHP frameworks will have some type of representation of the request and response object, and many APIs that use those objects. By forcing immutability, most of this APIs will have to change.
This decision has been made for sake of robustness. This apparently would “remove a whole class of bugs”. Well, I believe the confusion that comes with an unusual API will definitely open the doors to a whole new class of bugs as well ;).
Silex
To give you an example of an API that is forced to change, here’s an example from Silex. Silex has a set of events that allows a user to alter request and response objects:
<?php
$app->before(function (Request $request, Application $app) {
// Change request and optionally return a response early.
});
$app->after(function (Request $request, Response $response) {
// Change the response before its sent
});
?>
The before method is used to alter the request object, and can potentially
be used to send an early response and ‘bypass’ the normal request flow.
Likewise, after is used to modify a response before it’s being sent to a
client.
Altering to make this work with PSR-7 implies that these relatively simple methods need a new way to send back their altered responses.
One way to do that, is with references:
<?php
$app->before(function (Request &$request, Application $app) {
// ...
});
$app->after(function (Request &$request, Response &$response) {
// ...
});
?>
People tend to not really love references though, because it’s not really obvious from the code that calls the function that it’s going to alter the value of the argument.
I imagine that the functional folks (from which this immutable API stems from) would feel better if these functions didn’t alter their arguments, but instead emit function results.
A better way might be to define an all-new object that is mutable but contains references to both the request and response:
<?php
interface RequestContext {
function getRequest();
function setRequest();
function getResponse();
function setResponse();
}
?>
Although if we want to stick to the PSR-7 philosophy, this RequestContext object should itself also probably be immutable and return modified clones of itself.
To sum up my first issue: Nearly anyone who does something useful with Request and Response objects today and wants to switch to PSR-7, will likely have to rethink large parts of their application and APIs. These objects and the style chosen for them are infectious in nature and very opiniated in their design.
The issue with streams
This is where things get a little bit out of hand. One nice feature of both HTTP in a PHP server and HTTP clients, is that we can work with very large objects, without having to use a large amount of memory.
If, for instance, I would like to send back a file from my filesystem via PHP to a client, I can do so:
<?php
stream_copy_to_stream(
fopen('massive_movie.mp4', 'r'),
fopen('php://output')
);
?>
Likewise, it’s possible to
- read
php://inputfor very large requests - use streams for when making a very large http request using a client.
- use streams to read large http responses (in guzzle for example).
An important aspect of these streams is that they are often only readable once. What this means in reality, is that this breaks one of the core concepts of immutability.
<?php
// Given that $immutable is some immutable object
$immutable = '...';
any_function($immutable);
// Imutable _must_ still be in the exact same state now.
?>
Well, PSR-7 requests and responses are not like that:
Example
<?php
// Returns a request body
$request->getBody()->getContents();
// Returns an empty string.
$request->getBody()->getContents();
?>
In short this means that these objects are actually not immutable. This poses bigger problems, because when somebody changes a header:
<?php
$newRequest = $oldRequest->withHeader('X-Powered-By', 'Don Cheadle');
?>
After this operation, changes to $oldRequest can actually influence changes
to $newRequest. Not really according to the spec, but at least to the most
prominent current implementation of PSR-7, phly/http:
<?php
$newRequest = $oldRequest->withHeader('X-Powered-By', 'Don Cheadle');
// Request body
$newRequest->getBody()->getContents();
// Empty string!
$oldRequest->getBody()->getContents();
?>
This is fixable though, phly/http could in theory ensure that the request
body is also ‘cloned’ when withHeader is called, but this is also an issue.
We already established that every mutation requires a new instance. Requiring a new instance of the stream object for every mutations means that either:
- It needs to always be a string for efficient copying, thus increasing memory requirements.
- Keep it a PHP stream under the hood, and do an expensive copy operation for every ‘mutation’. This increases CPU requirements greatly.
Technically, neither of these are very good solutions.
Big responses
Lastly, it’s not clear to me in this new architecture how we can easily generate big responses on the fly.
This function, taken from StackPHP (sorry guys) is how PSR-7 wants you to think about HTTP applications:
req → λ → rep
The idea is that a function (your application) takes a request (req) and emits a response (rep).
After the response is created, it could be sent off to a client.
In PHP though, sometimes we just want to send a lot of bytes to php://output
or even just call echo() or readfile(). Sometimes we want to generate
large streams of XML or JSON and not keep it in memory, or we may want to do
an EventSource implementation.
I feel that the PSR-7 architecture forces us to buffer entire responses before
we can send them off. In my ideal world, a $request object literally wraps
php://output and writing to its body instantly emits the data, but unless
I’m mistaken this is in direct conflict with the philosophy of PSR-7.
Am I going to use PSR-7?
I’m very much on the fence. sabre/dav is an application that heavily makes use of all things HTTP, and is the type of application that would run into these edge cases.
Request and Responses are everywhere and are also modified often. To adopt PSR-7 means that the plugin api needs to be almost completely rewritten, which also means that a lot of people will be affected by it.
Furthermore, I can not easily emit large responses (yes we actually need to optmize for cases where we’re sending 450MB worth of XML data for a single response), and PSR-7 does just not handle this well, at least not well without ‘working around’ how the API thinks I should build the application. Buffered responses would clearly be a big issue here.
On the other hand this would make my application more like others and this would potentially open the door for people adding generic PSR-7 middleware, which is really cool. Currently we have both a ‘PHP League’ and a Symfony adapter for requests and responses and it would be nice to not need that at one point.
The impact is severe though. Perhaps I’m too specialized for PSR-7 to make sense? Regardless, if we don’t end up with PSR-7, we’ll definitely take inspiration from it. It does contain more than a few other great ideas.
We can always just create adapters for PSR-7 objects so we can both be compatible with PSR-7 and not deal with its drawbacks.
Comments
Alex •
Hi. Thank you for this interesting post. I think you have some valid points.
Could the immutability problem while returning the content from the response be solved by not returning the content istsself but only a reference to the steam?
I agree with you that returning large amounts of data is a quite common usecase and should be supported somehow. Maybe also the solution here would be to make the output stream accessible. When sending data to the stream you do not alter the response object and do not break immutability IMO.
Cheers,
Alex
Emeric Kasbarian •
Hello there,
I think that technically, an immutable object could not contain a reference to a mutable object. Otherwise, the Response object's state could be different following the state of the Stream object, and that would make the Response not really immutable.
I wonder which category of bugs can be avoided when having these immutable objects? Because I never saw an HTTP abstraction considering response as immutable. I would be interested in reading their arguments, and compare them against the ability to compose the Response.
Rasmus Schultz •
I guess they were shooting for consistency - which is important, but in this case doesn't really accomplish what we need. I say make the whole thing consistently mutable - if you think making objects immutable helps prevent bugs, I'm going to suggest maybe PHP isn't the language for you; there are languages where all objects are immutable values, but those languages, and more importantly all the software written in them, were designed with that in mind. PHP code, as shown in this post, generally is not written with that expectation, and immutable types only really exist for very special cases such as timestamps, which are naturally value types.
Richard Remer •
The Response is immutable regardless of whether the associated stream is. The response will always refer to the same stream, no matter what, and cannot be changed. Now, the state of that stream can change, but that doesn't mean the request was mutated. This forces the application to only read the request stream once, which is more efficient.
Evert •
This makes absolutely no sense.
Olivier Laviale •
I share your option about the PSR-7. Although I also implemented ICanBoogie's Request class as a somewhat immutable instance, parameters (request, path, query) are the one thing that should be mutable. Because it's often the case that the parameters of a request need to be altered during the process of a request. This includes sanitization, or controllers adding default values, or removing others.
Also I don't like these withXxx() methods, if you need to change multiple things you create a lot of intermediate instances for nothing. I elected to implement a change() method that allows me to obtain an altered copy of a request with many changed aspects in a single step.
Contrary to Request instances, ICanBoogie's Response instances are completely mutable. And I don't see it any other way because the instance is often altered during the life time of the application, by controller actions or event hooks for instance. I can always supply a different Response instance if I need to completely replace a response.
Regarding streams, I chose to use Closures. They are not seekable or anything, but really who cares about that? Thus, if the response body is a Closure I send the headers then invoke the closure to output the body. Pretty simple and efficient.
This is ICanBoogie's implementation of the Request/Response pattern: https://github.com/ICanBoog...
Fusty •
Could anyone enlighten me as to the theory behind making the object immutable? What would stop people from making a mutable response object that generates the immutable response object just before the end of execution (or would this simply be bypassing PSR-7)?
Thanks for the write up!
Evert •
It would totally be possible to take this approach, but then you would only really take partial advantage of what PSR-7 offers. You lose out on things like middlewares.
David Lundgren •
StackPHP, laravel, and I'm sure others have had middleware for quite a
while now, I don't think there is a real loss here. Middleware that is PSR-7 compliant should work ok. The exception I can see is as mentioned in another comment where forensic analysis is being done AFTER other middleware, at which point I'd suggest to move it, just as Rasmus suggested.
lucifurious •
Immutability at the response level is retarded. Obviously these alleged experts haven't coded many HTTP applications with REST APIs.
It sounds like they want to turn PHP into Java. Ew...
Olivier Laviale •
+1 for your comment about turning PHP into Java. Most PHP frameworks nowadays don't even use properties, it's all about setThis(), getThat()…
Phil •
We often joke about how symfony started turning php into java at work.
And it's not said as a compliment.
Jakub Zalas •
Attributes are application and request specific, and actually CAN be mutable.
Tomasz Kowalczyk •
How about solving immutability issue with ResponseBuilder? When someone needs to change Response object, ResponseBuilder::fromResponse($response)->setHeader('name', 'value')->getResponse() comes in handy and PSR-7 principles stand. I assume that when creating Response object constructor will be sufficient.
Vincent van Dijk •
I think builders and factories are always a good idea. A lot of PSR-7 implementations use static factories on the request and response object. Classes get very bloated and violate SRP.
Miles Johnson •
I definitely agree with what you're talking about. I love immutability when it is useful and I feel like this situation isn't one of them.
Alex Sears •
I completely agree that the Request object should be immutable. However, the Response object should definitely be mutable. The whole point is that it will constantly be changing until the server sends it back to the client. Continually creating new objects seems like a waste of overhead and memory.
I'm not sure if this is possible, but could the request object, on instantiation, do some sort of checking to see if the request is over some threshold? If it is, then it streams the request in when running $request->getBody()->getContent(). This could be given a "sensible" default in PSR-7 but is open enough to be able to be changed by people or frameworks that feel it should be a higher or lower threshold. A user could check if it's streaming using $request->isStreaming() and then run a loop over the object - if it implemented the Traversable interface of course! This also means the $request->getBody()->getContent() method could check for streaming, loop over it and save it locally, and then return the whole thing to the application. This way, when using the Request object, the developer doesn't need to worry about all the streaming issues if they don't want to.
.....oooooooor create a subclass of the Request object called StreamingRequest or something that can be used if a developer knows they are streaming in content.
Evert •
An immutable response was one of the earlier proposals, but that's also an issue in the client-situation where you want the exact opposite. It also ignores an (imho valid) use-case to allow plugins or middlewares (or whatever) to make small alterations to a request.
Lastly, by introducing a split non-streaming/streaming behavior would just make any streaming-related problem more complicated because you need to expect both situations. You can't just support one of them.
Rasmus Schultz •
agree, middleware in PHP is up and coming, and it's going to be hugely important any day now.
Pies •
Middlewares can very much alter the request information, just not by modifying the actual object. That means that any alterations to the request object must be done in a proper way - you can easily provide an interface that allows a module to interject itself into the Request object creation pipeline.
And if you're doing something completely non-standard that requires you to allow Request to be mutable, don't use the PSR-7 standard :) It's not obligatory.
Rasmus Schultz •
My own request and response models in the past have always been mutable. Why? Because these are objects, not values, unlike e.g. DateTime which desperately needed an immutable counterpart. Requests and responses? Definitely not values. Objects for sure. In PHP, objects are mutable, the only exception being value objects such as time stamps, because changing the time stamp makes it inherently a new value. Every case I can think, with immutable request and response models, I will have to simply throw away the previous instance after making a change - which makes immutability rather pointless, if in every case, the new object is going to replace the old, the net effect is equivalent to simply changing the object in the first place. So you will be fighting immutability every step of the way, and, as you pointed out, APIs will end up looking rather kinky.
As for issues with streaming larger volumes of data, I agree completely. A common use case for me is a long running CSV or XML export, which has to stream and cannot be buffered because it won't fit in memory. Even if it fit, buffering is unnecessary and a waste of resources, and requires the end user to wait for the whole data stream to be prepared before they can start downloading the result. That's crazy. My own response models in the past have always simply had a send() method, which would write to the output buffer - a simple approach that doesn't get in the way of testing, as you can simply turn on another output buffer, if needed. Whether it's sending a 100MB binary or a partial response or an HTML document or something else, has never been an issue for me with that model. Here is the simple response model I have been using:
https://github.com/mindplay...
frostymarvelous •
If I am still holding on to the original object, your changes won't affect me.
While you throw away, I'm still keeping it.
I don't make invalid assumptions about state this way.
Rasmus Schultz •
Yeah, that's fine in a lot of cases, but not this one - due to the life-cycle of a PHP request, which matches that of an HTTP request, the number of Request and Response objects required during the life-cycle of a single isolated script run (one HTTP request) is precisely one; there's never more than one incoming Request, or more than one Response returned, during the same script run.
That's not set to change in PHP 7, nor likely to change in any future version of PHP, so the assumption about state here seems correct to me.
The exception would be unit-testing, but I design for the use-case first, not for testing.
Evert •
A lot of people need the use of 'sub requests'. I certainly do.
Rasmus Schultz •
Again, that's one Request per sub-request, you don't need more than one Request instance to handle one sub-request. The parity remains the same - there is no case in which you would need more than one Request instance per request or sub-request.
IMO, the only case where immutable Request objects may be an advantage, is for testing - in every other case it's weird and unnatural and leads to strange code trying to "emulate" normal (in PHP) mutable objects by designing APIs that enable you to replace the Request object, leading essentially to garbled, messy code that effectively accomplishes the same thing.
I don't get it. This approach does nothing useful for me, it just complicates something that should be simple, and would be if you would follow the normal, natural object life-cycle generally used in PHP code.
Evert •
I wasn't arguing against mutability. I'm obviously a proponent. I'm just saying that this statement:
> The number of Request and Response objects required during the life-cycle
of a single isolated script run (one HTTP request) is precisely one
Is very far from the truth for many applications. Sub-requests is one of them.
Rasmus Schultz •
What I'm trying to say is that each individual request or sub-request requires one request/response object pair - the request that actually gets dispatched, and the response that actually gets returned. The intermediary objects (or object states if mutable) are uninteresting, except maybe for testing or diagnostics. Each immutable object is identical to a mutable object state, and only the final state or object is actually dispatched. Immutability in this case just means more fighting for control - in the end, what we want is to be able to alter the request in stages; mutable objects enable that, while immutable objects hinder it.
frostymarvelous •
Let me add that I'm not a proponent for immutability. I like php the old fashioned way.
I'm just playing the Devils advocate trying to find leaks in the immutability argument.
frostymarvelous •
Exactly! If the various middleware keep modifying the request, e.g. A middleware that cleans up the input etc then the actual request changes. Another middleware down the line might want to inspect the actual request that came in say for forensics or even logging. What happens then?
Rasmus Schultz •
So just inject your middle-ware in the correct order? If you want to inspect the original request, inject that middle-ware first - why would you inject it last and then try to work your way backwards to the original request? That seems backwards.
The only scenario I can think of, where this is useful, is if the request processing chain were to fork for some reason - e.g. creating two different, parallel request creation pipelines. In that particular case (assuming there are any real use-cases) that particular middleware could simple use the clone keyword to create an identical request object - that makes more sense to me than cloning everything by default with every incremental change, when most of the time the previous object simply falls out of scope and gets garbage-collected anyway.
I just don't see immutability solving a real problem here. For example, have you heard a lot of Symfony developers complain about mutability of the Request object? Does it cause real-world problems that are addressed by immutability? I don't get the impression that there's a lot of that going around.
frostymarvelous •
Your assumption being I know the workings of all my middleware in and out to know which one does what?
Remember, the example is simplistic. All my middleware can be modifying while trying to consume the original request.
And yes, the idea of psr7 is to allow us to reuse middleware from other frameworks.
nickshanks •
I would like to see a community-accepted extension to the PHP-FIG spec, that adds mutability to Response, and middleware written to *that* API instead. Then everyone would be happy, the purists can use the spec as it was written, and the realists can use the spec that works in the real world.
Bishop •
Sure, PSR-7 seems to give users the finger, but remember that the FIG PSR aim to foster interoperability. In this case, interoperability between middleware stacks. Despite holding state, request and response are shared amongst a pool of middleware components who may know nothing of one another and who may be able to affect one another. Viewed in this light, the default behavior of immutability makes sense, though it means FIG has more of a sales job to do to hook developers into this mentality.
Yes, immutability seems contrary to the common application use case, but I think it fosters a pluggable middleware ecosystem. And that, in my mind, is the biggest win. Because if we have a robust selection of middleware, we can eschew monolithic frameworks and instead start from gaunt micro-frameworks that applications fatten with suitable middleware.
bruce •
I have not tried psr-7 but i think immutable is not a problem. I think immutable objects can be copied simply by making a copy of a reference to it instead of copying the entire object or we could implement 'copy on write' technique.
Matrix AI •
You need generators, the response object should store a generator expression. That is controllers within the http kernel are all lazy, response transformations gets mapped to each other as an expression, and only at the final point of emission do you perform the task of fetching the content and emitting the response to the client. This way you get the flexibility of having out-of-order construction of responses, and ordered execution of the response.
Vincent van Dijk •
Any change in thoughts nowadays? PSR-7 is all about PHP 5.3, what about looking forward with such nice things as type hinting and return types in PHP 7?