Merge branch 'nightly' into fix-iter-ergonomics

.github/workflows/standard_library_tests_and_examples.yml

@@ -63,5 +63,5 @@ jobs:
         env:
           MOJO_ENABLE_ASSERTIONS_IN_TESTS: ${{ matrix.mojo-enable-assertions }}
         run: |
-          magic run tests
-          magic run examples
+          magic run --frozen tests
+          magic run --frozen examples

.gitignore

@@ -17,7 +17,6 @@ ENV/
 env.bak/
 venv.bak/
 .magic/
-magic.lock
 
 # MacOS
 .DS_Store

README.md

@@ -15,7 +15,7 @@
 Mojo is a new programming language that bridges the gap between research
 and production by combining Python syntax and ecosystem with systems
 programming and metaprogramming features. Mojo is still young, but it is designed
-to become a superset of Python over time.
+to become the best way to extend Python over time.
 
 This repo includes source code for:
 

docs/changelog.md

@@ -21,10 +21,11 @@ what we publish.
   [Issue #933](https://github.com/modularml/mojo/issues/933).
 
 - The destructor insertion logic in Mojo is now aware that types that take an
-  `AnyLifetime` as part of their signature could potentially access any live
-  value that destructor insertion is tracking, eliminating a significant
-  usability issue with unsafe APIs like `UnsafePointer`.  Consider a typical
-  example working with strings before this change:
+  `MutableAnyOrigin` or `ImmutableAnyOrigin` as part of their signature could
+  potentially access any live value that destructor insertion is tracking,
+  eliminating a significant usability issue with unsafe APIs like
+  `UnsafePointer`.  Consider a typical example working with strings before this
+   change:
 
   ```mojo
   var str = String(...)
@@ -35,20 +36,20 @@ what we publish.
 
   The `_ = str^` pattern was formerly required because the Mojo compiler has no
   idea what "ptr" might reference.  As a consequence, it had no idea that
-  `some_low_level_api` might access `str` and therefore thought it was ok to
+  `some_low_level_api()` might access `str` and therefore thought it was ok to
   destroy the `String` before the call - this is why the explicit lifetime
   extension was required.
 
-  Mojo now knows that `UnsafePointer` may access the `AnyLifetime` lifetime,
-  and now assumes that any API that uses that lifetime could use live values.
-  In this case, it assumes that `some_low_level_api` might access `str` and
+  Mojo now knows that `UnsafePointer` may access the `MutableAnyOrigin` origin,
+  and now assumes that any API that uses that origin could use live values.
+  In this case, it assumes that `some_low_level_api()` might access `str` and
   because it might be using it, it cannot destroy `str` until after the call.
   The consequence of this is that the old hack is no longer needed for these
   cases!
 
-- The `UnsafePointer` type now has a `lifetime` parameter that can be used when
-  the `UnsafePointer` is known to point into some lifetime.  This lifetime is
-  propagated through the `ptr[]` indirection operation.
+- The `UnsafePointer` type now has an `origin` parameter that can be used when
+  the `UnsafePointer` is known to point to a value with a known origin. This
+  origin is propagated through the `ptr[]` indirection operation.
 
 - The VS Code Mojo Debugger now has a `buildArgs` JSON debug configuration
   setting that can be used in conjunction with `mojoFile` to define the build
@@ -123,9 +124,9 @@ what we publish.
     pass
   ```
 
-- Function types now accept a lifetime set parameter. This parameter represents
-  the lifetimes of values captured by a parameter closure. The compiler
-  automatically tags parameter closures with the right set of lifetimes. This
+- Function types now accept an origin set parameter. This parameter represents
+  the origins of values captured by a parameter closure. The compiler
+  automatically tags parameter closures with the right set of origins. This
   enables lifetimes and parameter closures to correctly compose.
 
   ```mojo
@@ -144,8 +145,8 @@ what we publish.
   ```
 
   Note that this only works for higher-order functions which have explicitly
-  added `[_]` as the capture lifetimes. By default, the compiler still assumes
-  a `capturing` closure does not reference any lifetimes. This will soon change.
+  added `[_]` as the capture origins. By default, the compiler still assumes
+  a `capturing` closure does not reference any origins. This will soon change.
 
 - The VS Code extension now has the `mojo.run.focusOnTerminalAfterLaunch`
   setting, which controls whether to focus on the terminal used by the
@@ -157,18 +158,19 @@ what we publish.
   determining a default SDK to use. The user can select the default SDK to use
   with the `Mojo: Select the default MAX SDK` command.
 
-- Added a new [`Box`](/mojo/stdlib/memory/box/Box) type as a safe, single-owner,
-  non-nullable smart pointer with similar semantics to Rust's
+- Added a new [`OwnedPointer`](/mojo/stdlib/memory/owned_pointer/OwnedPointer)
+  type as a safe, single-owner, non-nullable smart pointer with similar
+  semantics to Rust's
   [`Box<>`](https://doc.rust-lang.org/std/boxed/struct.Box.html) and C++'s
   [`std::unique_ptr`](https://en.cppreference.com/w/cpp/memory/unique_ptr).
 
   ([PR #3524](https://github.com/modularml/mojo/pull/3524) by [@szbergeron](https://github.com/szbergeron))
 
 - `ref` argument and result specifiers now allow providing a memory value
-  directly in the lifetime specifier, rather than requiring the use of
-  `__origin_of`.  It is still fine to use `__origin_of` explicitly though,
-  and this is required when specifying lifetimes for parameters (e.g. to the
-  `Reference` type). For example, this is now valid without `__origin_of`:
+  directly in the origin specifier, rather than requiring the use of
+  `__origin_of()`.  It is still fine to use `__origin_of()` explicitly though,
+  and this is required when specifying origins for parameters (e.g. to the
+  `Pointer` type). For example, this is now valid without `__origin_of()`:
 
   ```mojo
   fn return_ref(a: String) -> ref [a] String:
@@ -214,6 +216,19 @@ what we publish.
 - The Mojo LLDB debugger now supports symbol breakpoints, e.g. `b main` or
   `b my_module::main`.
 
+- The VS Code extension now allows cancelling the installation of its private
+  MAX SDK.
+
+- The VS Code extension now opens the Run and Debug tab automatically whenever
+  a debug session starts.
+
+- The `mojo debug --vscode` command now support the `--init-command` and
+  `--stop-on-entry` flags. Execute `mojo debug --help` for more information.
+
+- The Mojo LLDB debugger on VS Code now supports inspecting the raw attributes
+  of variables that are handled as synthetic types, e.g. `List` from Mojo or
+  `std::vector` from C++.
+
 ### 🦋 Changed
 
 - More things have been removed from the auto-exported set of entities in the `prelude`
@@ -234,8 +249,8 @@ what we publish.
   specifies to the compiler that the resultant pointer is a distinct
   identifiable object that does not alias any other memory in the local scope.
 
-- The `AnyLifetime` type (useful for declaring lifetime types as parameters) has
-  been renamed to `Lifetime`.
+- The `AnyLifetime` type (useful for declaring origin types as parameters) has
+  been renamed to `Origin`.
 
 - Restore implicit copyability of `Tuple` and `ListLiteral`.
 
@@ -341,12 +356,50 @@ what we publish.
   been consolidated under `s.as_bytes` to return a `Span[Byte]`, you can convert
   it to a `List` if you require a copy with `List(s.as_bytes())`.
 
-- `Lifetime` and related types has been renamed to `Origin` in the standard
+- `Lifetime` and related types have been renamed to `Origin` in the standard
   library to better clarify that parameters of this type indicate where a
   reference is derived from, not the more complicated notion of where a variable
   is initialized and destroyed.  Please see [the proposal](https://github.com/modularml/mojo/blob/main/proposals/lifetimes-keyword-renaming.md)
-  for more information and rationale.  As a consequence `__lifetime_of` is now
-  named `__origin_of`.
+  for more information and rationale.  As a consequence the `__lifetime_of()`
+  operator is now named `__origin_of()`.
+
+- You can now use the `+=` and `*` operators on a `StringLiteral` at compile
+  time using the `alias` keyword:
+
+  ```mojo
+  alias original = "mojo"
+  alias concat = original * 3
+  assert_equal("mojomojomojo", concat)
+  ```
+
+  Or inside a `fn` that is being evaluated at compile time:
+
+  ```mojo
+  fn add_literal(
+      owned original: StringLiteral, add: StringLiteral, n: Int
+  ) -> StringLiteral:
+      for _ in range(n):
+          original += add
+      return original
+
+
+  fn main():
+      alias original = "mojo"
+      alias concat = add_literal(original, "!", 4)
+      assert_equal("mojo!!!!", concat)
+  ```
+
+  These operators can't be evaluated at runtime, as a `StringLiteral` must be
+  written into the binary during compilation.
+
+  - You can now index into `UnsafePointer` using SIMD scalar integral types:
+
+  ```mojo
+  p = UnsafePointer[Int].alloc(1)
+  i = UInt8(1)
+  p[i] = 42
+  print(p[i])
+  ```
 
 ### ❌ Removed
 
@@ -365,11 +418,14 @@ what we publish.
   doesn't extend the lifetimes of the values it references.
 
 - [Issue #3627](https://github.com/modularml/mojo/issues/3627) - Compiler
-  overlooked exclusivity violation caused by `ref [MutableAnyLifetime] T`
+  overlooked exclusivity violation caused by `ref [MutableAnyOrigin] T`
 
 - The VS Code extension now auto-updates its private copy of the MAX SDK.
 
 - The variadic initializer for `SIMD` now works in parameter expressions.
 
 - The VS Code extension now downloads its private copy of the MAX SDK in a way
   that prevents ETXTBSY errors on Linux.
+
+- The VS Code extension now allows invoking a mojo formatter from SDK
+  installations that contain white spaces in their path.

docs/manual/basics.ipynb

@@ -25,7 +25,7 @@
     "world\"](/mojo/manual/get-started). Now let's talk about how\n",
     "to write Mojo code.\n",
     "\n",
-    "You probably already know that Mojo is designed as a superset of Python. So if\n",
+    "If\n",
     "you know Python, then a lot of Mojo code will look familiar. However, Mojo\n",
     "is—first and foremost—designed for high-performance systems programming, with\n",
     "features like strong type checking, memory safety, next-generation compiler\n",
@@ -643,7 +643,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Mojo is not yet a full superset of Python, but we've built a mechanism to import\n",
+    "Mojo does not yet adopt the full syntax of Python, but we've built a mechanism to import\n",
     "Python modules as-is, so you can leverage existing Python code right away.\n",
     "\n",
     "For example, here's how you can import and use NumPy (you must have Python\n",

docs/manual/decorators/parameter.ipynb

@@ -127,7 +127,7 @@
     }
    ],
    "source": [
-    "fn use_closure[func: fn(Int) capturing -> Int](num: Int) -> Int:\n",
+    "fn use_closure[func: fn(Int) capturing [_] -> Int](num: Int) -> Int:\n",
     "    return func(num)\n",
     "\n",
     "fn create_closure():\n",
@@ -150,20 +150,19 @@
    "source": [
     "Without the `@parameter` decorator, you'll get a compiler error that says you\n",
     "\"cannot use a dynamic value in call parameter\"—referring to the\n",
-    "`use_closure[add](2)` call—because the `add()` closure would still be dynamic."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
+    "`use_closure[add](2)` call—because the `add()` closure would still be dynamic.\n",
     "\n",
-    ":::caution\n",
+    "Note the `[_]` in the function type:\n",
     "\n",
-    "This is an unsafe feature because we currently do not model the lifetimes of\n",
-    "capture-by-reference.\n",
+    "```mojo\n",
+    "fn use_closure[func: fn(Int) capturing [_] -> Int](num: Int) -> Int:\n",
+    "```\n",
     "\n",
-    ":::"
+    "This origin specifier represents the set of origins for the values captured by\n",
+    "the parametric closure. This allows the compiler to correctly extend the\n",
+    "lifetimes of those values. For more information on lifetimes and origins, see\n",
+    "[Lifetimes, origins and references](/mojo/manual/values/lifetimes).\n",
+    "\n"
    ]
   },
   {

docs/manual/index.md

@@ -10,7 +10,7 @@ Mojo is designed to solve a variety of AI development challenges that no other
 language can, because Mojo is the first programming language built from the
 ground-up with [MLIR](https://mlir.llvm.org/) (a compiler infrastructure that's
 ideal for heterogeneous hardware, from CPUs and GPUs, to various AI ASICs). We
-also designed Mojo as a superset of Python because we love Python and its
+also designed Mojo as the best way to extend Python because we love Python and its
 community, but we couldn't realistically enhance Python to do all the things we
 wanted. For a longer discussion on this topic, read [Why
 Mojo](/mojo/why-mojo).

docs/manual/lifecycle/index.ipynb

@@ -71,20 +71,21 @@
         "constructor (`__copyinit__()`), and the move constructor (`__moveinit__()`).\n",
         "All values that are declared with the same type have the same lifecycle.\n",
         "\n",
-        "- The \"lifetime\" of a value is defined by the span of time during \n",
-        "program execution in which each value is considered valid. The life of a value \n",
-        "begins when it is initialized (via `__init__()`, `__copyinit__()` or \n",
-        "`__moveinit__()`) and ends when it is destroyed (`__del__()`), or consumed in\n",
-        "some other way (for example, as part of a `__moveinit__()` call). \n",
+        "- The \"lifetime\" of a variable is defined by the span of time during \n",
+        "program execution in which the variable is considered valid. The life of a \n",
+        "variable begins when its value is initialized (via `__init__()`, \n",
+        "`__copyinit__()` or `__moveinit__()`) and ends when the value is destroyed \n",
+        "(`__del__()`), or consumed in some other way (for example, as part of a \n",
+        "`__moveinit__()` call). \n",
         "\n",
-        "No two values have the exact same life span, because every value is created and \n",
+        "No two values have the exact same lifetime, because every value is created and \n",
         "destroyed at a different point in time (even if the difference is imperceptible).\n",
         "\n",
-        ":::note Lifetime type\n",
+        ":::note Origin type\n",
         "\n",
         "The concept of lifetimes is related to the `origin` type, a Mojo primitive\n",
         "used to track ownership. For most Mojo programming, you won't need to work with\n",
-        "`origin` values directly. For information, see [Lifetimes and\n",
+        "`origin` values directly. For information, see [Lifetimes, origins and\n",
         "references](/mojo/manual/values/lifetimes).\n",
         "\n",
         ":::\n",