Synchronous API ?

[mcarbonneaux]

with promise there no way to get synchronous resulte for now...
it means not to have synchronous api in parallel of the asynchronous API?
beceause some time they need synchronous, and some time asynchronous ...

[daviddias]

@mcarbonneaux I believe the answer here is that you want to avoid any sync operation that is CPU bound.

A Sync API of the WebCrypto API would be useful inside a WebWorker context though.

[sideshowbarker]

I’m gonna make a judgement call that based on what I know, there’s not gonna be any browser-engine implementor interest in this. If anybody feels strongly about it otherwise, and is ready to commit to driving implementor discussion around it, then let me know and we can re-open this.

[threema-lenny]

I'm convinced synchronous operations are needed, at least in a WebWorker context. The problem with async operations is, as you all know, they make the whole surrounding context async. This introduces a significant step up in complexity in a logic section where races are absolutely paramount to avoid. Think end-to-end messaging for example.

To give you a concrete example: In Threema, a large pipeline is responsible for processing a message. This pipeline is currently synchronous, and intentionally so to avoid races, but has to fall back to libraries such as tweetnacl for crypto because it is sync. Making this pipeline async would not only require refactoring a huge API surface but also reach a level of complexity that is almost unmanageable. And for no reason, I might add, because it already runs in a WebWorker and the crypto operations are very fast over smaller amounts of data.

Being able to choose between sync and async is important. I think it should be a goal make (the most commonly used) cryptographic libraries on npm.js redundant and ultimately replace them with guaranteed safe, audited and performant operations of this spec. But this will be impossible without having synchronous operations.

Sorry for reviving this ancient issue. Is this a good place to converse or should I open a new issue?

[Frosne]

I'm convinced synchronous operations are needed, at least in a WebWorker context. The problem with async operations is, as you all know, they make the whole surrounding context async. This introduces a significant step up in complexity in a logic section where races are absolutely paramount to avoid. Think end-to-end messaging for example.

To give you a concrete example: In Threema, a large pipeline is responsible for processing a message. This pipeline is currently synchronous, and intentionally so to avoid races, but has to fall back to libraries such as tweetnacl for crypto because it is sync. Making this pipeline async would not only require refactoring a huge API surface but also reach a level of complexity that is almost unmanageable. And for no reason, I might add, because it already runs in a WebWorker and the crypto operations are very fast over smaller amounts of data.

Being able to choose between sync and async is important. I think it should be a goal make (the most commonly used) cryptographic libraries on npm.js redundant and ultimately replace them with guaranteed safe, audited and performant operations of this spec. But this will be impossible without having synchronous operations.

Sorry for reviving this ancient issue. Is this a good place to converse or should I open a new issue?

You might want to take a look at #389 ?

[twiss]

I'll reopen this issue as well since #389 is more about how the existing API should behave, and @davidben has also stated (there and elsewhere) that he'd prefer a sync API for performance reasons.

I think having a synchronous API would be valuable also because it makes it much easier to call out to from WebAssembly. Obviously wasm code can also ship its own crypto operations, but that increases the code size and somewhat defeats the purpose of having WebCrypto at all. It's also not guaranteed to be constant-time (as long as we don't have constant-time instructions in wasm yet).

I may take a stab at drafting something at some point. The most significant open question is whether we should copy+paste the existing API and remove the async keywords and such (and add some restrictions such as being WebWorker-only), or whether we should design a new API from scratch. The latter is likely to be a lot more work to spec and implement, but allows us to start with a clean slate.

There are also possibly advantages to keeping a similar API, though; such as making it easier to switch over. We could even make them interoperable, such as by reusing the CryptoKey object (which is especially useful if you have those stored in IndexedDB or something, though that may not be particularly common). That might complicate some other ideas such as #263, though (but we could just say that hardware-backed keys can only be used asynchronously, if that's something we ever end up adding).

In any case, I think it's worth discussing and it would be nice to hear what the other implementers think :)

[annevk]

Would this only be for dedicated workers? Will the inputs be constrained to a certain size?

cc @pascoej

[twiss]

I think limiting it to Web Workers would be simplest, yeah. In that case, there's no need to specifically constrain input sizes (more than we do today), since it shouldn't affect the UI responsiveness anyway.

We could also expose certain operations on the main thread (with input size limits), but what constitutes a "reasonable input size" is probably quite dependent on the device capabilities. Also, encrypting a bunch of small files in a tight loop on the main thread is probably not a great idea either, so forcing people to use a worker may lead to better UX in some cases.

[threema-lenny]

I think there is a bit of a niche spot where sync operations also make sense on the main thread. Any artificial size limit would be arbitrary and result in workarounds, like workarounds that are applied to circumvent quota limits for getRandomValues.
That being said, I don't feel strongly about the main thread being forced to async only and personally have not seen a use case where that would bite me.

When it comes to APIs, I'm full of opinions as the next person. But I think what would be really beneficial is to expose the existing functions as sync first and then (or in parallel) think about a new generation API including sync as a requirement.

Happy to provide feedback when desired!

[davidben]

Ah yeah, I'm definitely supportive of WebCrypto getting a sync API. We're paying a lot of overhead from being async today:

• The thread hop is more expensive than the operations themselves
• As noted here, it's hard to integrate WebCrypto in any other API; no one else makes cryptography, a CPU-bound operation, async
• We have to copy every buffer before doing the operation, whereas some of our primitives are fast enough that people design in-place APIs to avoid even a copy in the encryption itself
• Even if we wanted to, we could not design in-place APIs without race conditions
• All intermediate state in calling functions up the stack must be pickled into a closure object, either explicitly, or implicitly with async function

The API shape itself, with everything mashed into a small set of verbs and JS object parameters, also costs us:

• By jamming ECDH and HKDF into the same thing, we introduce security pitfalls like providing a truncation option for ECDH
• The JS object parameters must account for custom getters (which might mutate everything), which means the exact order that parameters are accessed must be specified. As specified, the input buffers are copied before reading the getters, which is another forced copy
• It is impossible for developers to feature detect an algorithm. They have to just try it and see if it works

If we're able to, I think it would be quite valuable to get away from all that. Though interoperability with CryptoKey would be valuable because it's often easier to incrementally migrate functions on shared types than types. Then again, if we were model cryptography in JS more naturally, we probably would have had methods on algorithm-specific types. That makes it easier to do things like add PSS-specific methods that take extra parameters to account for PSS's overparameterization, etc. Not sure how best to square that.

Regarding main thread ops, TBH almost all of our ops can run just fine on the main thread. There are a few exceptions (RSA keygen is so, PBKDF2 takes an iteration count that can get large). It's true that hashing 1 GiB will take a while, but that's also true of Array.sort or for and no one wants to make those async. On the Chrome side, we currently run almost all the async ops immediately on the calling thread anyway, because it turns out the thread hop is more expensive than the cryptography. (But we're still very far from the cost of the actual operation due to the API problems above.)

[twiss]

Yeah. FWIW, it's a bit tangential but regarding

• It is impossible for developers to feature detect an algorithm. They have to just try it and see if it works

https://wicg.github.io/webcrypto-modern-algos/#SubtleCrypto-method-supports proposes a SubtleCrypto.supports() method for the existing API. Obviously that's not as nice as just being able to check whether a method exists, but at least it's better than try/catch.

• By jamming ECDH and HKDF into the same thing, we introduce security pitfalls like providing a truncation option for ECDH
• The JS object parameters must account for custom getters (which might mutate everything), which means the exact order that parameters are accessed must be specified. As specified, the input buffers are copied before reading the getters, which is another forced copy

Perhaps these are things we could (and should) fix in the existing API as well? E.g. with #351 and by changing the order of operations. I strongly doubt that any user of the API is modifying the input data in algorithm object getters.

That's not to say I don't agree that we could have (had) a better API, but I'm tempted to think that if we could more or less copy+paste the existing API (without the asyncness and with minor improvements) we could have something useful much faster than having to design & discuss a new API for every primitive individually.

• We have to copy every buffer before doing the operation, whereas some of our primitives are fast enough that people design in-place APIs to avoid even a copy in the encryption itself

And then after we have a generic sync API (modeled after the existing one) we could always add specific methods for in-place operations, where that makes sense (e.g. for symmetric encryption and decryption)?

[davidben]

by changing the order of operations. I strongly doubt that any user of the API is modifying the input data in algorithm object getters.

I would also be surprised if anything but (I hope!) WPT is sensitive to this. Though resolving the algorithm first means we tolerate arbitrary modifications to the buffers before we do the operation. I don't know if it's possible for those buffers to somehow stop being buffers or do something extra weird in between the two, such that some check by the IDL layer becomes invalidated.

(I actually don't know. I'm not an expert on JS state and the assumptions made by IDL layers. Probably need to talk to an expert there. What happens if the algorithm object has a custom property that, on access, transfers the underlying buffer to a message port. Does that put the object in some kind of bad state? Or if it messes with the prototype?)