Add support for headers in functions | Voters

Gatsby

Create

Roadmap

Gatsby Cloud 🌤

Add support for headers in functions

planned

Michael Zetterberg

A discussion at GitHub has all the details here https://github.com/gatsbyjs/gatsby/discussions/31842 and originally here https://github.com/gatsbyjs/gatsby/discussions/30735#discussioncomment-757061

Basically it would allow for better control over client caches and CDN caches to control how often functions are called and cache leading to improved handling of quota.

June 9, 2021

Shruti Chaturvedula

marked this post as

planned

Joel Smith

Hi Michael Zetterberg, I've just updated the docs to show how you can control cache headers on Gatsby Cloud. Let me know if that takes care of your specific needs or if there's more that you'd be looking to do with headers!

https://support.gatsbyjs.com/hc/en-us/articles/1500009143521-Deploying-Functions-on-Gatsby-Cloud

Michael Zetterberg

Joel Smith: Thank you! My test function at https://gswpgatsby.gatsbyjs.io/api/hello still works as before, incorrect cache-headers. Maybe a redeploy is needed? Also I see in the docs that it says to use res.set(...)
 and not res.setHeader(...)
 (which my test function uses, I will change and check again).
I'm not seeing any specific mention of support for s-max-age
 either, but the statement "to ensure that functions invocations are appropriately handled when served to CDN users" seems to imply it.
I will give it a new go this weekend and will report back with my findings here! Thank you for now :)

Michael Zetterberg

Joel Smith: OK I gave it a go and these are my findings. I used this repo https://gitlab.com/weahead/gswp-gatsby deployed here https://gswpgatsby.gatsbyjs.io/

Both
res.set(...)
and
res.setHeader(...)
work (I had to redeploy though, my previously deployed function did not work). But
set
gives me "Property 'set' does not exist on type 'GatsbyFunctionResponse<any>'." in TypeScript. So I would propose to update the documentation to use
setHeader
instead (or update the typings, but that seems more overkill).
If calling a function from my "HOSTED ON GATSBY CLOUD" domain: https://gswpgatsby.gatsbyjs.io/ it is served through Varnish and the function is invoked correctly based on
s-maxage
header. I verified this by checking the function log in admin (which by the way is real time! Awesome, what a pleasant surprise!). It does not show a new entry until
s-maxage
has passed.
stale-while-revalidate
also works as expected on the CDN level. My function has a
new Date().toString()
call, so I can verify the responses. I verified by invoking the function and then waiting
s-maxage
to pass and then invoke again. It gave me a HTTP 304 stale response with an
age
larger than
s-maxage
. Fetching again gave me a response where
age
was not 0 and the response had increased its seconds correctly. Waiting
s-maxage
+
stale-whilte-revalidate
and fetching again I get a HTTP 200 with an
age
of 0. All good!
Regarding
stale-while-revalidate
, when a stale response is returned there is no way to identify if the response is stale or not without some guess work and math. It would be nice to get a
x-cache-stale
header with
1
or
0
or similar.
I checked browser cache as well, works nice too. Forcing a refetch does trigger a revalidation (I get 304) but only on the CDN level, it does NOT trigger the function to revalidate if the function actually has any new content to return. This is precisely how I expect it all to work. So just noting it here as "documentation".
If calling a function from my build url https://build-d6c01ead-0788-4b56-80b5-af543f47cbee.gtsb.io/ it is NOT served through Varnish and the function is NOT invoked correctly based on s-maxage. This makes sense to me as I believe there is no shared cache/reverse proxy between the client and gtsb.io deployments. Could you confirm? My gut feeling tells me this might be a nice thing, having a URL that can bypass the CDN, but of course it is up to you if that is the intended behaviour. Some users might find it unexpected, maybe explicit documentation might help clarify for them.
Since
s-maxage
doesn't apply to browser caches, you could (should?) strip it at CDN level so it isn't sent to the client. Maybe add it as an X-Stripped-Headers or something so developers can know that it is recognised but stripped.
It is not as clear what to do with
stale-while-revalidate
, as browsers do support it. I wasn't expecting it on my function invocations during testing, but I think that is mainly because I'm used to how Vercel does it. I'm not sure what the "correct" way is, but I would assume that full flexibility would be allowing the developer to choose whether to strip the header in the CDN or not per function, either through configuration (in admin or code) or through code in the function itself. Having said that, personally, I have never thought of having
stale-while-revalidate
header in my client responses over at Vercel (we use it for CDN level only), given that logic I would not miss them on Gatsby Cloud.
The responses from functions contain the
x-robots-tag
header with a value of
none
. I could not find any Gatsby Cloud option or documentation on it, but this header with a value of
none
would request search engines to not index its contents. Some users might find that unexpected. I tried to override it from the function headers but it did not work as expected. Looking at the gtsb.io deployment it shows two instances of the
x-robots-tag
header, both with the value of
none
. Checking through the CDN seems to "squash" these identical headers into a single instance, which is good. What's weird though is that when I try to override it from the function I end up with two instance of the
x-robots-tag
header again, but this time one has the value of
none
and the other the value of
all
(which is what I set in the function). I expect a single instance of
x-robots-tag
with the value of
all
. I would like the CDN to use the headers set from the function always if possible.
I did not try to remove any headers, so I haven't done any testing related to that. I would expect that headers from the function itself would be removable but the headers from the CDN would not be available for manipulation as they are applied after the function invocation itself.

Keep up the good work! You are doing great!

Michael Zetterberg

Joel Smith: I'm not sure my reply to myself above triggered a notification for you. So just to be on the safe side I'm replying your comment again to make sure you see my comments. Thanks!

Kyle Mathews

Michael Zetterberg: I updated the docs to use

res.setHeader

—

set

is an express-ism and not as good as API as what's in node.js core already with

setHeader

. 👍

Thanks for the great testing! Glad it's working for the most part. Looks like some more stuff to clean up here.