Arguing refcaps

[hovsater]

In this Playground example (also attached below), table have been captured by the lambda. The code does not compile, but I'm pretty sure it should. The issue was also shortly discussed on Zulip prior to this issue.

Playground code

use "collections"
use "itertools"

class Foo is (Equatable[Foo])
  let _numbers: Set[U32]
  
  new create(numbers: Set[U32]) =>
    _numbers = numbers
    
actor Main
  new create(env: Env) =>
    let table: Map[U32, Set[U32]] = table.create()
    let container: Array[Foo] =
      Iter[U32]([as U32: 1; 2; 3].values())
        .map[Foo]({(n: U32) => Foo(table.get_or_else(0, Set[U32]))})
        .collect(Array[Foo])

[SeanTAllen]

Ok so looking at this, there's a few things going on with what @hovsater is trying to do:

First this works:

use "collections"

class Foo is (Equatable[Foo])
  let _numbers: Set[U32]

  new create(numbers: Set[U32]) =>
    _numbers = numbers

class Bob
  let t: Map[U32, Set[U32]] = t.create()

  fun ref apply(n: U32): Foo =>
    let empty = Set[U32]
    let a = t.get_or_else(n, empty)
    Foo(a)

But that isn't what @hovsater's code desugars to effectively. His lambda is creating a fun box apply and then we get the error from get_or_else as the "else" and return type from it are this->V.

You can see a similar error from this simpler code:

use "collections"

class Foo is (Equatable[Foo])
  let _numbers: Set[U32]

  new create(numbers: Set[U32]) =>
    _numbers = numbers

class Bob
  let t: Map[U32, Set[U32]] = t.create()

  fun apply(n: U32): Foo =>
    let empty = Set[U32]
    let a = t.get_or_else(n, empty)
    Foo(a)

But that leads to a different issue with itertools which can be seen in this slightly simplified example:

use "collections"
use "itertools"

class Foo is (Equatable[Foo])
  let _numbers: Set[U32]

  new create(numbers: Set[U32]) =>
    _numbers = numbers

actor Main
  new create(env: Env) =>
    let container: Array[Foo] =
      Iter[U32]([as U32: 1; 2; 3].values())
        .map[Foo](
          object ref
            let t: Map[U32, Set[U32]] = Map[U32, Set[U32]]
            fun ref apply(n: U32): Foo =>
              let empty = Set[U32]
              let a = t.get_or_else(n, empty)
              Foo(a)
          end)
        .collect(Array[Foo])

[SeanTAllen]

@jemc I won't be around for the next sync, but I hope that I've left enough crumbs for this to start being discussed more fully.

[SeanTAllen]

@hovsater noted in Zulip that map_stateful works. I am not very educated about Itertools. Perhaps that was created for situations such as this?

[SeanTAllen]

i discussed with @jemc who knows Iter much better than I. This issue and similar is indeed why map_stateful exists. It is the proper solution.

[hovsater]

Alright. I'm still not entirely following why we need map_stateful here. Would an explanation be possible? Perhaps for the sole reason to understand reference capabilities better.

[SeanTAllen]

Let's chat at an upcoming Office Hours (I wont be there next week). I laid everything out above so I think it is an issue of walking through it together and answering questions.

I'll be there on the 16th.

[jemc]

Here's a quick summary from my end:

• map takes a fun box apply object and map_stateful takes a fun ref apply object
• since you're using map, your function is box / read-only, and everything it sees in its closure environment is via box viewpoint-adaptation (i.e. everything is seen as read-only) (see also this table on viewpoint adaptation)
• You cannot obtain a mutable Set[U32]'ref from a read-only Map[U32, Set[U32]'ref]'box because of viewpoint adaptation - again, it's read-only all the way down
• you need map_stateful because it will give you a read/write view of its closure environment