SES on moddable's XS

[dckc]

I followed the Getting Started docs; got the dev tools built and the hello world app running

cc @dtribble @erights

2018-09-30 04:04 PM moddable 9b53f14 boilerplate

[dckc]

doesn't look good for our hero:

XS does not store the source code of functions so Function.prototype.toString always fails.
-- XS Conformance Revised: March 19, 2018

[erights]

Hi @dckc , this page is very useful.

We should mark which of our functions actually need their source. Perhaps something like a "use the source" directive at the top of those functions. (Note that there are recent tc39 proposals that suggest directives in the other direction.) We could then do a source-to-source transform copying the source of those functions into literal strings we attach to those functions.

I'm surprised at some of the other non-conformances mentioned, in particular, the lack of template string cache. However, unlike the source string issue, none of these others jump out at me as an urgent problem.

cc @warner

[dckc]

So we only need to be able to stringify a statically-known set of functions?

That surprises me, but now that I think about it, I'm not sure why.

[erights]

See https://github.com/Agoric/SES/issues/19#issuecomment-451362331

[dckc]

picking up on this again... pulling c9a7519..3763532

(and from SwingSet: 15ecf15..3763532)

[dckc]

ag-solo on xs

initial goal: wire up the kernel and bootstrap stuff to the point where it asks for its bundle of stuff using the mailbox... the initial mailbox will just scribble stuff to a file or something so that we can see that it did something.

This comes from a chat with @warner and @dtribble . It probably fits better in its own issue, but until monorepo stuff calms down, I'll just leave this here.

https://github.com/Agoric/cosmic-swingset/tree/master/lib/ag-solo

[dckc]

I ended up emulating much of the SES API using xs's compartment API tonight:

https://github.com/dckc/cosmic-swingset/blob/xs-platform/xs-platform/emulate/ses.js
5f07a15

[dckc]

I'm struggling to understand (or remember?) how endowments are novel and not just short-hand for passing args to a function.

In my port of SES to xs, I turned endowments into function args.
https://github.com/dckc/agoric-sdk/blob/xs-platform/packages/cosmic-swingset/xs-platform/emulate/ses.js#L10-L27

@jfparadis @erights @warner can you explain how this doesn't suffice?

Hmm... maybe you already did... in that one bit of documentation...
p.s. the thing I'm thinking of isn't in https://github.com/Agoric/SES/blob/master/docs/secure-coding-guide.md ... maybe it was a code comment...

[dckc]

ah... there it is: https://github.com/Agoric/SES/issues/67#issuecomment-466705822 from Feb 23
squirreled away in https://github.com/dckc/do-your-worst/blob/master/server.js

I think that's more about what sorts of things should be passed between realms/compartments, whether as endowments or function arguments.

Hmm... I'm starting to see why my approach might be a problem:the Function constructor (or eval) carries a per-compartment environment with it. This looks like another place where my E/monte habits don't serve. In monte, the entire environment is passed in to eval, including the safeScope or primordials or whatever.

Hmm...

[jfparadis]

@dckc, endowments are almost lile function arguments, but not quite:

• in XS, they are properties added to the global object.
• in realm-shim, they almost partly like function arguments, partly like a with target under the global object.

[dckc]

My code is running in XS and it does not make endowments into properties on the global object. My question is: does this suffice? And if not, why not?

https://github.com/dckc/agoric-sdk/blob/xs-platform/packages/cosmic-swingset/xs-platform/emulate/ses.js#L10-L27

[jfparadis]

A function argument is only a subset of what is possible with the other form. For example, an accessor will not be properly reproduced, but that's a small case, there are others.

Does this suffice? It depends. It depends which code is being evaluated, which endowments, which caller.

The one case I think is important to resolve is assignments. If endowments = { x: 1 }, and the code being evaluated does { x = 2 }, then the endowment object will become endowments = { x: 2 }.

We do leverage that behavior in may areas, for example in the transforms.

[dckc]

I suppose the tests should tell me whether this approach suffices.

I went looking for a test that demonstrated writeable endowments; I don't see one. The closest I found was a comment to the contrary:

  // SES.confine accepts endowments, which are made available in the global
  // lexical scope (*not* copied onto the global object, which is frozen
  // anyways), so they'll be available for only the duration of the eval, and
  // only as unbound names (so they could be found statically in the AST)

-- https://github.com/Agoric/SES/blob/master/test/test.js#L41-L43

Would you please make a PR with a test for this assignable endowment feature?

Is there a spec for SES? Can somebody help me find this bit about assignable endowments? (maybe I should raise a separate issue?)

[erights]

Given the conversations in various other threads, I'd like to clarify this one. @dckc is your concern legacy SES-compatible code not written with SES in mind? (The LavaMoat and npm usecase) or code we're writing, to run under SES? If the latter, compartments and endowments should hardly ever be necessary or relevant. We should generally be writing only pure modules. And most of them should be pure Jessie modules, which is unaware of the global object.

The testing frameworks, tape etc, is a legacy compat issue that does apply to us. Also, as discussed in other threads, so can the platform code that gathers initial authorities from non-SES prior platform APIs. Are these the legacy issues of concern?

[dckc]

My concern is cosmic-swingset. Vats, devices, etc.

In particular: am I leaking access to scopes that I shouldn't be?

[erights]

My concern is cosmic-swingset. Vats, devices, etc.

Yeah, to the degree that these are free of legacy code we cannot fix, they should not be using Compartments, global objects, or endowments. Of course, they should be using these to restrict the authority of guest SES code, but that's another matter.

[dckc]

Hm. I thought that was exactly this matter: the use of SES in controller.js in SwingSet to make a new realm for the kernel and such. I suspect we're reaching the limits of this communications medium and I should give you a call some time this week.

[kriskowal]

My current understanding is that we won’t use SES-shim on XS, as-is. An Agoric runtime on XS will need some parts of SES like harden, but not lockdown. We will need to build CommonJS and run-time ESM support out of SES parts too, but won’t use the SES Compartment. We will need to tool to precompile ESM in Agoric application archives.

I would like to continue tracking that project on #400. Please reopen here if this conversation is in fact ongoing.