Update MAX_COOKIE_SIZE

[felix-tjernberg]

Describe the problem

const MAX_COOKIE_SIZE = 4196 might be set by an inaccurate assumption

From what I have gathered it seems that chromium and firefox sets max byte size to 4096 for name + value attributes, chromium also seems to limit all the other attributes to 1024 each whilst firefox does not have a limit for the other attributes. webkit however seems to have set 5000 maximum for the whole string (I'm not to sure on if this webkit reference is correct though as I had problems understanding that code base, however it seems weird that they have some internal thing called curl that is not curl it self, so not sure if this code is even used)

The extra 100 bytes assumption that came from when rich tested the maximums might have been from attributes that are not name & value

Describe the proposed solution

• Check if name + value attributes is maximally 4096 bytes to match chromium & firefox & upcoming cookie consumer specification behavior
• Check if each other attribute is maximally 1024 bytes to match chromium & upcoming cookie consumer specification behavior
• Check if the whole string is maximally 5000 bytes to match webkit behavior (however suggestion is to also add a monitor on the webkit code base to see if they change their behavior after the new specification comes into effect, see comment below comment for more information)
• console.warn when limits are hit instead of throwing to follow the cookie producer specification behavior (see below comment for more information)

Alternatives considered

No response

Importance

nice to have

Additional Information

I have not tested these limits myself after I found the references in code

I personally do no have a need for this change, however I think it is important to match the browsers behavior, so maybe something worth adding in 3.0 as I saw you guys are planning on enforcing that the string only contains valid characters in 3.0

Also it might be worth documenting in the sveltekit docs that the standard encourages the server to design as few and as small states as small as possible, or in funnier words please do not treat cookies as a database and if you do please use a sane encoding to make the data small

Servers SHOULD use as few and as small cookies as possible to avoid reaching these implementation limits and to minimize network bandwidth due to the Cookie header being included in every request
RFC 6265

Another thing worth documenting in the sveltekit docs is that the original purpose for "cooparative programs keys", is in the general web case to share a database key in requests, which lets the website become stateful as the server can serve the user specific content for each request, usually a shopping cart. But because the cookie limits are large enough you can treat them as small databases that you can modify on the client without making a request when javascript is available using httpOnly=false, which allows for keeping track of user settings without using a "real" server only database

Something I love about your docs is that that it helped me "become a better developer" (a phrase I think was in a previous version of the docs), not sure though if above is in the spirit of what you want to teach about the web in your docs

Another thing I might as well note is that the maximum number of cookies seems to be 150 in chromium & firefox (I do not really understand the quota business, but I'm guessing 150 is max where you can have 30 more for some edge case where cookies have not yet been garbage collected) and 80 is max in webkit, so if you are going to warn about minimum number of cookies I would suggest adding a maximum warning at 80 (if you decide to add my documentation suggestions in sveltekit docs, I would also emphasize that the server should not come close to this limit, and if it does please consider combining cookies to not get close to the limit, ie you have 10 user settings, consider serializing them into 1 cookie with some split token)

Whilst at it I could also inform that the total cookie limit seems to be 3000 in both chromium & firefox, however I could not find a reference in the webkit code base so I'm guessing they do not have a limit (however not finding a reference might have been a skill issue from my part) (also ~96gb (if they all are maxed out) of cookie data is however a resonable maximum xD)

Hopefully you find this rabbit hole information useful :D

[felix-tjernberg]

I recently came across the latest IETF http state management mechanism draft and they have codified the 4096 limit for name + value & codified the 1024 limit for each other attribute

When it comes to the total limit my understanding is that there should not be one explicitly, however they have noted that there is commonly a 8kb web server header limit, so limiting the total to 5000 might have to change in the future depending on if webkit decides to follow the specification (added suggestion to monitor webkit codebase for this). Also it might be worth documenting the 8kb header limit because users might think that it is ok to set multiple large cookies and hits that limit instead

Notably curl also has a 5000 total limit but will probably change theirs quicker because they likely consider themselves as a general-use user agent, which should in practice not have any limits at all if I understood the specification correctly)

Something that also is very interesting about the new specification is that they have split the implementation into two distinct types, so I urge the implementor of this issue to read that part of the specification and discuss with the team what implementation type SvelteKit should follow

From my perspective SvelteKit is a cookie producer and should follow the section 4 specification, however that would mean that you would not have any limits at all

No limits is however not really in line with the practical helpful spirit that SvelteKit has to it's users, so my suggestion is that you console.warn when the limit is hit whilst trying to set the cookie (not throwing like the current implementation), this would help the user diagnose what went wrong if cookies are not set, whilst not limit users that are serving user agents that does not have limits (maybe a option to surpress this logging statement might be needed for those users)

[benmccann]

We currently have const MAX_COOKIE_SIZE = 4129. Is your proposal to make it const MAX_COOKIE_SIZE = 4196 const MAX_COOKIE_SIZE = 4096? And then also to change from an Error to a warning to allow communicating with clients that don't follow the spec?

[felix-tjernberg]

I split my response in two difficulty levels, because I realised that what I think is important nuance to account for, might not be important accounting for your team

My intention is to share all my cards on the table, and hope that it is useful for your team, and I totally get if the hard part is ignored because of time constraints

Easy to implement response

If you want to loosely follow the consumer part of the cookie specification, your response is the most straight forward thing to do, however I'm curious though how 4196 translates to 4096 bytes which only accounts for the name+value limit in the consumer part of the specification (guessing that it was a typo)

However after recontextualizing myself with the issue, I want to give an alternative easy to implement proposal:

• Lower MAX_COOKIE_SIZE to 2000 bytes
  • And change the variable name to for example ABNORMAL_COOKIE_SIZE_THRESHOLD
• Change error to warning
  • Whist providing a configuration options to update the threshold or turning the warning off

The 2000 threshold I picked arbitrarily, because I have no data of what is normal, for some arbitrary reference, JWT tokens are apparently normally around 500 bytes, so allowing 4 of those might be a resonable default

The warning should just be a friendly reminder:

• Informing users that their cookie was abnormally large and they should reconsider their cookie data structure
  • For example telling them to create a custom byte based data structure instead of dumping JSON
• Whilst referring to the new cookie specification point, that web servers normally set http header size to 8000 bytes

This is hopefully inline with sveltekits helpful informative nature

I reckon that few users will actually see this warning, and for those that do, are probably trying to doing something unorthodox with cookies

For example treating the cookie as a client side stored database for insane amounts of end user preferences

So users like that will probably find it useful to be informed about what perils such behavior entails, as it helps them think about the limits early, and then if they decide to proceed have the option to turn off the warning

Hard to implement response

But the easy implementation does not solve the "hidden bonus" issue for sveltekit, which is deciding what parts of the cookie specification should be accounted for

This check is giving me the impression that the framework want to find an opinionated general solution, which is a ven diagram of both the consumer & producer parts of the specification

From my perspective, that requires the implementor to spend a lot of time in the saddle reading the specification and discussing with the team, as the specification paint two different pictures (the producer part seem a bit more relaxed, and the consumer part is arbitrarily complicated whilst giving subject to change vibes)

From my understanding of the specification a framework like sveltekit is only considered to be a producer, however because sveltekit is an opinionated framework and concern itself with client behavior, the framework could state opinions that are based on the consumer part (where cookie max size is just a small part)

So my hard proposal is that the implementor starts a team discussion, of what the framework should state opinions about, in order to help users design resonable cookie data structures that account for all limits

Personally I would start with the following proposal (which only covers parts of the full specification):

• For each individual cookie limit name+value fields to 4096 & other fields to 1024 which follows the upcoming consumer specification (as mentioned above)
  • With the option to configure these checks (which is inline with the specification as the definitions are minimum limits)
  • And the option to turn off these checks, for users that are confident in their cookie data structure, and or wants a small performance boost
• Alternatively for each individual cookie remove limits and refer to the fact that the specification has defined minimum limits
• For the http cookie header warn when size is at 4000 and set hard limit at 5000
  • This check happens both request & response
  • Both are limits are configurable, and able to be turned off (because the specification only points to the fact that 8000 is a normal web server http header limit)
• Check if each byte is a cookie octet (which there already exists a todo comment for in the source code, however could not find an open issue for that)

Additional maybe useful context rant

I'm not sure if this rant is helpful, but left it here in hopes to show case where my understanding of cookie size limits nuances comes from

Cookie limits being complicated is a result of the specification defining arbitrary minimums limits historically, and browser vendors deciding to implemented those limits as maximums, without allowing the end users to configure these limits

This is bad from my understanding, because the limits are set as a minimums in order to not restrict clients & servers that are capable of sending and receiving large amounts of data, so there should exist end user configuration options for such use cases

But I also understand that browser vendors wanted to limit the total storage on disk, however arbitrarily deciding for all end users without allowing for configuration feels irresponsible

Personally I think that specification authors opted for too much specificity, which caused unduly amounts of confusion, and resulted in people forgetting what the original problem is: accounting for http header size and client disk storage size

The specification authors defined:

• A shared total storage minimum limit for all cookies
• A per domain minimum limit, that seems to account arbitrarily for the total storage limit
• An individual cookie minimum limit, that seems to arbitrarily account for the domain and total storage limits

Those are quite abstracted from accounting of http header size and client disk storage size, and I get why you would as a browser implementor opt for the irresponsible solution, because the specification authors did not carry out due diligence in a responsible manner

So from my limited perspective defining a client disk storage size limit for each domain name, is a possible resonable thing to do because:

• It would account for http header size and disk storage size
• It would be reinforced by the other already existing definition: that all cookies connected to the domain name, should be sent in the cookie header field for each request
• Bonus: For clarity define that servers can infer how much domain storage is left on the client by looking at the incoming cookie header size

However I could come up with one possible downside, of only defining one storage limit for each domain name:

• Because the definition that all cookies connected to a domain name will be sent on each request
• A 3 level deep domain name could maximally send 10KB per request which will exceed the normal 8KB web server http header limit, if you set the limit to for example 5KB per domain name
  • (to clarify 10KB could be sent if example.com allows for its cookie to sent for all sub domains by settings the domain field, in other words sub.example.com could send two maximally large cookies because of example.com)

However however that theoretical limit is dwarfed to the current specification cookie header minimum limit per domain which is 200KiB(4KiB * 50), and the practical implementation has a maximum limit of 400KB(5K * 80) (because of webkit being the lowest common denominator, which I have stated in an earlier comment)

So I do really understand why there is so much confusion around the current cookie limitations

Hopefully though the specification is already moving in the direction of removing most of the limits, and just pointing to the fact that the normal http header limit is 8KB, and that there should not be any maximum limits in practice, it is up to the server author to take responsibility for the client

The specification could maybe to help end users out with storage space, by defining that browser vendors have to ask or inform end users how much storage is allowed on the device, however I'm not sure if a protocol definition can take such responsibility

[benmccann]

Have you hit an issue with this in practice or is the motivation to follow the spec?

[felix-tjernberg]

I have yet to hit an issue in practice, because I have planned for the current limits, by reading the specification, browser vendors source codes, and sveltekit source code

Which made me realise that sveltekit arbitrarily sets a maximum that does not match the specification or browser vendors

So I though that it would be helpful to point sveltekit in a direction that follow the specification more closely, especially as the newest one is defining consumer behavior, which hopefully will normalise behavior across browsers

In other words helping the framework prepare for the normalisation

Was it wrong of me to open an issue for that? Was it to early to open an issue for that?

[benmccann]

I do think we should change the value in SvelteKit to match the spec, so thanks for the issue! I'd probably lean towards the simplest possible solution as it will make it easy for us and users to understand and I think be sufficient in practice

[benmccann]

Doing some digging of where the current number comes from. In #8584 (comment):

Though oddly the limit in all three browsers appears to be 4,129 bytes (for the whole thing — attributes and value) rather than 4,096.

Here's what Gemini says we're doing:

Probably it would be better to change this limit to 4096 and check without the options here:

https://github.com/sveltejs/kit/blame/d8b6133e98ea5642f96e716330811c55d66f6e0a/packages/kit/src/runtime/server/cookie.js#L219C46-L219C57

I'm going to untag this as a breaking change since it should be equivalent behavior and only affects dev mode

[benmccann]

PR welcome for the change, but I won't personally prioritize working on it for 3.0 since there's not a lot of impact to users