feat: bundle manifests in code instead of loading from fs

[ghostdevv]

I realised when deploying code that imported module-replacements to Cloudflare that it was reading the manifests from the file system, which of course didn't work. This is the minimum change I could get to instead have the json become javascript without_ removing tshy or introducing a breaking change. I couldn't find if this had been talked about before, so apologies for the impromptu PR

[43081j]

i think this makes sense but it probably needs to happen in the next major since we have the json files in our package exports

maybe you can help me get the new manifest structure over the line in #266 👀

and then the new major would have both

[ghostdevv]

i think this makes sense but it probably needs to happen in the next major since we have the json files in our package exports

They're still there - from a consumer point this shouldn't be a breaking change as only how the main.ts imports the manifests has changed. Before, read from disk at runtime, now simply imported.

maybe you can help me get the new manifest structure over the line in #266 👀

👀

[gameroman]

I think this makes sense and although this should not be a breaking change, even if it is, since we are releasing schema v3 we can merge this

[43081j]

I think it makes sense. Can you resolve the conflicts?

@bluwy @paoloricciuti any thoughts?

[paoloricciuti]

Since we already can make a breaking should we make it for good and remove the script? It might came in handy if later we want to do more stuff in the module and we could make the exports more generic by dropping the .json so that the only thing the user knows is that it's importing an object.

[43081j]

do you mean move all the manifests to be typescript in source?

it'd be a much bigger change but may not affect consumers much since we already expose it as a typed object

[gameroman]

I think he means to remove the .json export from package.json

[bluwy]

This doubles the package size, but I suppose it doesn't matter a lot if they're compressed anyways. But this can be workaround today though by directly importing the json files, e.g. import data from 'module-replacements/manifests/native.json' with { type: 'json' } (or with require if compat is required), so I think I'd prefer making the proper refactor altogether in the next major. One thing it'd be good to preserve is being able to import a single manifest only.

[paoloricciuti]

do you mean move all the manifests to be typescript in source?

it'd be a much bigger change but may not affect consumers much since we already expose it as a typed object

Exactly what I meant...just have one single source with the data and drop the .json from the exports (I guess we could even keep the .json but that would be weird). This would solve the "double in size" problem and having to add an extra step to the build

[gameroman]

Do we want to keep CJS?

[gameroman]

I think we should drop CJS support completely

[43081j]

This doubles the package size

how come?

you mean if we convert the manifests to typescript? if we do that, there's one copy, no? the same size as we have now (roughly).

[gameroman]

I am thinking of something like this
https://github.com/gameroman/module-replacements/tree/tsdown

[43081j]

@bluwy and @paoloricciuti can you two be clear: rewrite the manifests as typescript, yes or no?

Not automatically, but in source. i.e. No more JSON anywhere.

That will decide if we need something like Roman's branch or not. If we need a transform/bundler or not etc

[paoloricciuti]

For me it make sense to have the TS modules directly

[bluwy]

how come?

I mean this PR, which makes the data exist in TS and JSON.

rewrite the manifests as typescript, yes or no?

I don't really mind either as long as it's only in TS or JSON, and it's possible to only import a single type of replacement data (not load the others in memory).

I have a slight preference to JSON though if the data would always be serializable, and would be easier to be consumed by non-JS/native tooling.

[gameroman]

I think it makes sense to leave it as JSON in source

[43081j]

i agree i think it is currently nice that we have simple JSON files anything can load. typescript adds a barrier to that.

this is why they are not bundled and are loaded from disk. if we bundle them and want to keep JSON, we end up with 2x the size of course.

could we not use actual JSON imports via import attributes? would cloudflare not support that?

[bluwy]

I think we can use import attributes but might want to set an explicit node engines range for that support. I believe cli-boxes have a similar setup.

[43081j]

it turns out cloudflare can't do import attributes. so we would have to decide if that's ok or not.

most workers will be bundled so its probably not a problem. but just FYI

[ghostdevv]

most workers will be bundled so its probably not a problem.

I think since they have to be bundled as module-replacements is a package which isn't a thing when you actually deploy to workers, then we can just use import attribs - assuming there isn't some other thing that is weird with them? The only downside is dropping Node 18