I'm able to implement a JS-like Promise system, but this doesn't help with running asynchronous C# functions, for example sending a HTTP request.
Async/await is syntax sugar for promises.
You'll need to familiarize yourself with Continuation Passing Style (CPS). Basically, CPS is a way to write regular, sequential ("procedural") code as a chain of callbacks. Ex:
function() {
let x = foo("a", "b")
let y = bar("c", "d")
let z = baz(1, 2, x)
return y + z
}
converted into CPS is
function(k) {
foo("a", "b", function(x) {
bar("c", "d", function(y) {
baz(1, 2, x, function(z) k(y + z))
})
})
}
This is roughly what your interpreter needs to do with async functions before you interpret them. Convert:
async function() {
let x = await foo("a", "b");
let y = await bar("c", "d");
let z = await baz(1, 2, x);
return y + z;
}
into
function() { return new Promise(resolve =>
foo("a", "b").then(function(x) {
bar("c", "d").then(function(y) {
baz(1, 2, x).then(function(z) resolve(y + z))
})
})
}
Specifically:
- After parsing an
async function or other kind of aysnc block, desugar it into a regular function which returns a promise, where all the awaits are replaced with .then and .catch callbacks.
- If your language supports top-level async, you can parse and desugar the entire source to get one big promise which you then execute. Alternatively, to avoid parsing the entire file at once, you can parse and desugar each individual statement into a promise, waiting to parse the next statement until the promise resolves. Basically, write an async function
evalRemaining(int pos), which lexes/parses and desugars the first statement into a promise, then adds a .then callback to that promise which recursively calls evalRemaining with the position right after that statement's end.
- If your language supports top-level async in a REPL you handle it similarly: read/lex/parse/desugar the current line into a promise, then wait until that promise resolves before asking the user for the next line, by putting the code to do so in the promise callback.
Now, your goal is to desugar a statement or expression with awaits, into a promise expression where the awaits are replaced with .then and .catch callbacks, and the promise resolves into the expression result. This is essentially the same as converting a block of procedural code into a function written in CPS.
There's a lot of existing literature on continuation passing style. You mentioned having trouble finding resources to implement async/await, but if you can rephrase your queries about implementing async/await into implementing CPS conversion, I guarantee you'll find online guides and other resources.
Rule-based async/await (CPS-style) conversion via preorder traversal
Like type-checking, computing the derivative in calculus, and other algorithms, one way to do async/await conversion is to create a fixed set of rules which handle general cases and compose into whatever code you may encounter. Specifically, create a rule for every kind of statement which desugars in a special way, and then to desugar an arbitrary statement, run a preorder traversal, processing each sub-statement or expression with the applicable rule.
Here are some of these rules. If your language has unique semantics, you may need to come up with other rules or modify these rules on your own. There are also be optimized rules for more specific cases which will generate faster / smaller code. Lastly, I didn't check these rules, so there may be bugs:
Async function/block
async function f(...) -> T {
...
<... throw x; ...>
<... return y; ...>
}
converts into
function f(...) -> Promise<T> {
return new Promise((resolve, reject) => {
...
<... resolve(x); ...>
<... reject(y); ...>
// In case the function doesn't return anything,
// add this to the end of the Promise or it'll never resolve
resolve()
});
}
(e.g. return foo gets converted into resolve(foo), throw bar gets converted into reject(bar), otherwise code is desugared with the following rules and placed into the promise closure).
Async blocks, if your language supports them (like Rust), are similar:
async {
...
result
}
converts into
Promise::new(|resolve| {
...
resolve(result)
})
If a statement contains a nested await but it's in an async function or block, desugar the inner async function/block before desugaring the outer one. This also applies to async expressions and other rules.
Statements or expressions without await
A sequence of statements without an await is unchanged.
async function() {
a;
while (b) { c; d; }
try { e; } catch { f; } finally { g; }
}
converts into
function() -> Promise<()> {
return new Promise(resolve => {
a;
while (b) { c; d; }
try { e; } catch { f; } finally { g; }
resolve();
});
}
Expression or expression statement
before;
context <... await expr ...> context;
after;
becomes
before;
expr.then(e => {
context <... e ...> context;
after
});
e.g.
let a = foo();
let b = bar(await baz()) * 2;
let c = a + b;
becomes
let a = foo();
baz().then(e => {
let b = e * 2;
let c = a + b;
});
If statement
if (cond) {
then;
} else {
else;
}
rest;
If either then or else contain an await, the entire if must be converted into a new promise
new Promise(resolveTheIf => {
if (cond) {
then.then(resolveTheIf);
} else {
else.then(resolveTheIf);
}
}).then(() => {
rest;
})
(of course, make sure resolveTheIf doesn't shadow the outer resolve in case then or else contains a return statement).
If cond contains an await, you can convert into the equivalent:
let c = cond;
if (c) {
then...
} else {
else...
}
rest...
and then convert from there.
If either then or else does not contain an await, replace then.then(resolveTheIf) or else.then(resolveTheIf) with then; resolveTheIf() or else; resolveTheIf() (respectively).
If your if/else returns an expression like Rust, convert the zero argument call/closure into a one-argument one. Ex:
let foo = if (cond) {
await then()
} else {
else()
}
...
converts into
Promise::new(|resolveTheIf| {
if (cond) {
then.then(resolveTheIf);
} else {
resolveTheIf(else)
}
}).then(|foo| {
...
})
While statement
Similar to the above
while (cond) {
body;
}
rest;
if body contains an `await, this converts into
const recurse = resolveTheWhile => {
if (cond) {
body.then(() => recurse(resolveTheWhile));
} else {
resolveTheWhile();
}
};
new Promise(recurse)
If cond contains an await, you can resolve it before the while block.
Like return and throw when desugaring an async function, break desugars into resolveTheWhile(). If you have nested whiles and allow labeling them, so that users can call break outer from within the inner loop to break the outer one, you have to call resolveTheWhile of outer to break the outer loop.
Switch and for statements
These can be deduced from the if and while statements. I'll show an example of a complicated for statement.
for (let i = await init(); await cond(i); await update(i)) {
await body;
}
rest;
converts into
init().then(i => {
const recurse = resolveTheFor => {
cond(i).then(c => {
if (c) {
body.then(() => update(i).then(() => recurse(resolveTheFor)));
} else {
resolveTheFor();
}
})
};
}
Try/catch/finally statement
This is similar to the if statement, except that the catch block corresponds to a .catch promise callback.
try {
tryCode;
} catch (e) {
catchCode;
} finally {
finallyCode;
}
...
If either tryCode, catchCode, or finallyCode contain an await, you need to convert the entire statement into a promise.
If tryCode, catchCode, and finallyCode all contain awaits, the entire statement converts into:
new Promise((resolveTheTry, rejectTheTry) => {
// In general, you should be careful duplicating code when desugaring
// because it can lead to exponential blowup (imagine what happens if you
// put finallyCode in the desugared output twice, and then the user writes
// 32 nested try/finally statements. Language users write that kind of
// garbage, then if you're not careful, your interpreter will desugar it
// into billions or more syntax nodes.
const finally = (rethrown, e) => {
finallyCode;
if (!rethrown) {
resolveTheTry();
} else {
rejectTryTry(e);
}
};
tryCode.then(() => finally(false)).catch(e => {
catchCode.then(() => finally(false)).catch(e2 => finally(true, e2))
})
}
If any of tryCode, catchCode, or finallyCode don't contain an await, you can those into new Promise(resolve => { code; resolve() }). There may be optimized variants, but since there are 6 others, I'm not going to compute them all.
A block without a catch case
try {
tryCode;
} finally {
finallyCode;
}
converts into
new Promise((resolveTheTry, rejectTheTry) => {
const finally = (rethrown, e) => {
finallyCode;
if (!rethrown) {
resolveTheTry();
} else {
rejectTheTry(e);
}
};
tryCode.then(() => finally(false)).catch(e => finally(true, e))
});
Convert a statement or expression without await into one with await
If necessary, you can convert an expression which doesn't contain await
expr
into one that does
await new Promise(resolve => resolve(expr))
Or a statement
stmt;
into
await new Promise(resolve => { stmt; resolve() });
Out of the above rules, this is only necessary in the try/catch/finally one.
