Maybe return null or otherwise there is no way to know if it "failed".

Also decompressStream should probably not live under /web/

@ronag, can it fail though? The intention was that, if there's no known compression provided, the stream is returned as-is.

You can still handle failures as you normally would:

decompressStream(myStream).on('error', callback)

if there's no known compression provided

The logic seems to be that if an unknown coding is sent, the body isn't decompressed at all. I'm not sure if that sounds right either.

Content-Encoding: gzip, zstd, br -> ignored (zstd is not supported)
Content-Encoding: gzip, br -> decompressed

As a consumer, I expect the body stream to be returned as-is if:

1. It has no Content-Encoding set (no codings provided as an argument);
2. It has unknown Content-Encoding set or an unknown coding(s) is provided as an argument.

I wouldn't expect it to throw because that's a disruptive behavior that implies I need to add a guard before trying to call decompress/decompressStream. At least, not at a level this low.

If the API is potentially disruptive, I need to have an extra check in place to prevent scenarios where decompression is impossible. This puts extra work on me as a consumer but also results in a more frail API since the list of supported encodings is baked in Undici and may change across releases.

I strongly suggest to return the body stream as-is in those two cases I described above. decompressString() must never error, save for the streams errors itself (and those are irrelevant in the context of this function).

@metcoder95 @KhafraDev do you agree with my reasoning in the post above?

Hah, so there's an existing expectation that decompression is skipped if encountered unknown encoding:

 ✖ should skip decompression if unsupported codings

I suppose that answers the question.

throwing an error will make fetch slower

Regarding the decoders customization discussion, I don't think it justifies the cost of making decompressStream more complex. The only thing you need is the list of decodings, which you can get using a one-liner:

 const codings = contentEncoding
    .toLowerCase()
    .split(',')
    .map((coding) => coding.trim())

The purpose of decompressStream is to grab the exact behavior Undici has under the hood. If you need a different behavior, you should (1) parse the Content-Encoding header; (2) map encodings to decompression streams with custom options.

I can see how you'd want to extend or override certain things from the default decompression, and for that we, perhaps, can consider exporting those options separately?

module.exports = {
  gzipDecompressionOptions: {
    flush: zlib.constants.Z_SYNC_FLUSH,
    finishFlush: zlib.constants.Z_SYNC_FLUSH
  }
}

Even then, that's all the options Undici uses right now. To me this looks like a nice thing to have without substantial use case argumentation. I lean toward iterating on this when we have more of the latter.

Regarding erroring or throwing; As you suggest @kettanaito, I'd like to fail faster and inform the implementer the encoding is not supported without even disturbing the input stream, so fallbacks can be applied if they want to.

For fetch, as @KhafraDev points out, the throwing might be slower, so we can disable by default, and not throw at all, but rather ignore the non-supported encodings.

The issue I'd like to avoid, is putting the implementer in the spot of all or nothing.

Even then, that's all the options Undici uses right now. To me this looks like a nice thing to have without substantial use case argumentation. I lean toward iterating on this when we have more of the latter.

Agree, let's do that 👍

I think decompressStream is sufficient. It should already be redirected.

I'd like to have a designated utility that performs the decompression including the handling of redirect responses, etc. decompressStream implies you handle that yourself.

It looks to me like you are targeting the Fetch api response.
In this case, have you already been redirected?

@tsctx, I'm moving the existing content encoding logic to this utility function. In the existing logic, Undici does accept the response, which is the redirect response, not the redirected response. This is correct. This utility must also decide if decompression is unnecessary if the response is a redirect response.

headersList is not an instance of Headers

Technically, not, but its APIs are compatible, from what I can see. At least, the entire test suite hasn't proven me wrong.

I can construct Headers instance out of headersList but it may have performance implications. Would you advise me to do that?

This is the result of running npm run lint:fix on this branch. You may have something misconfigured if it reports diffs.

Or this branch is behind and main has applied this formatting.

please undo them