Hi Adam, Timothy Roscoe and his team at ETHZ did some OS work that seems to me very synergetic to your language based analysis: have you seen it ? There's an overview keynote on that work by Timothy Roscoe at OSDI'21 entitled "It's Time for Operating Systems to Rediscover Hardware"... If the programming language (even low level) lives in a world of fantasy it is indeed a problem as you point, but if the OS also lives in such a fantasy world things are even worse ;-) ("It's Time for Operating Systems to Rediscover Hardware", https://www.youtube.com/watch?v=36myc8wQhLo)
Great pointer, Vicente. I'm glad to link in more fellow travelers thinking about revising abstractions across layers of conventional stacks.
If I'm looking for one most-interesting difference in worldview with Timothy Roscoe's recent work, it's probably that SoCs seem to depend heavily on cache-coherent shared memory (at the same time as part of the source of their complexity is a bewildering variety of component-local memories, too). I think we should try to do better than shared memory as a unifying communication abstraction.
I also personally tend more toward solving problems of abstracting and multiplexing hardware with compiler-style solutions, rather than OS-style solutions.
Yes, cache coherence shared memory in SoCs is preferred usually. But have a look at this recent work by Roscoe's team : I just looked at it quicly but it seems to have a more spatial feel https://dl.acm.org/doi/10.1145/3725783.3764403.
I know you favor compiler-style solutions, but someone has got to compile/build the OS, and programs can be seen -in the end- as "plugins" in an OS context ;-) (or else you need some "operating system" code in your program). It's like the OS should be in the abstract machine. Don't you think ? Actually operating systems might be the less well helped code by current hardware "abstractions" in languages (even assembler).
Especially assuming that future hardware may be more standardized and along more pleasant abstractions of message passing, then it seems to me that standard OS services can be reframed as repeated recompilation (and static analysis) for changing application and device mixes, but it's not like anyone built such a system yet as far as I know, so it's a speculative principle for now. Probably all the same can be done by OS-style services that are more dynamic, with the usual tradeoff around runtime overhead.
Interesting ! Just as you plead for spatiality control/expression in programs, I believe the core of OS code -which is not really a « program » or application- is all about non sequentially managing interrupts -a kind of spatial temporality-. I would be curious to see how you manage that kind of code. In particular, if all other code uses messaging, as you or or microkernel architectures used, you will need a particularly optimized interrupt management.
I think we're broadly on the same page here, except that I don't think the concept of an interrupt is fundamental. We can use asynchronous message-passing instead, with queues.
1000x1000 grid of processing elements, connected by a single-cycle-per-hop communication fabric, each element independently programmable and has private memory, communication is message-passing, no shared memory. Each processing element is more like a simple in-order core than a CGRA/FPGA, but overall is dataflow architecture.
Has a zig-like language (CSL) for programming it. Language encourages defining parameterized modules for laying out on the processing elements, where the parameters might be location or behavior switches, then a "layout" module (like a main() function) that assigns modules to each PE and sets up the message-passing routes.
I worked on CSL long ago. Really enjoying your blog series. The critique of Von Neumann especially. So many PhD-years spent making compilers and CPUs super smart to wring performance out of a language-induced straightjacket.
Oh, thanks for this pointer: I've heard about Cerebras as a leader in AI acceleration for a while, but I had never looked into it in detail. The overlap with the philosophy I've suggested is indeed quite high. You've inspired me to briefly cover Cerebras as well as GPUs in my next post! Two differences I see already from the path forward I've been imagining:
CSL strikes me more as a good compiler intermediate language than as what programmers should want to be coding in directly. We'll see if some of the higher-level compiler techniques I propose check out as helpful to bridge that gap.
One of the topics I think is fascinating is finding a middle ground between, on the one hand, shared memory with cache coherence; and on the other hand, "good luck, here's explicit message passing." Compilers probably can't understand everything up-front about message-passing patterns, so what hardware support can help provide good performance for relatively irregular workloads? Cerebras seems to stick to ML workloads that are very regular and thus hasn't tackled this question yet.
We arrive at the same conclusion from different directions. Assembly language is over. The sequential instruction stream is the wrong abstraction. Agreed. What replaces it is where it gets interesting.
You say: spatial dataflow. Computation laid out in space with explicit communication costs. The CGRA as the convergence point between software and hardware. The compiler makes spatial decisions in advance. The von Neumann bottleneck dies because the program IS the circuit layout. Correct.
I say: obligation discharge. The program is a set of obligations. The execution is the discharge of those obligations. No instruction stream. No program counter. No sequential control flow. The compiler resolves obligations at compile-time. The emitted artifact IS the discharged obligations. I built this. The language has no functions, no variables, no loops. It compiled itself. The compiler just crossed to native RISC-V — it runs on a seventy-dollar board on my desk and emits binaries that run on the same board.
The convergence: both kill the von Neumann abstraction. Both move decisions to compile-time. Both eliminate global shared mutable state (your Bluespec modules with encapsulated state, my authority-separated obligation engine). Both treat the sequential model as the problem, not the solution.
The divergence: you optimize for performance (speed, area, power). I optimize for governance (admissibility, authority separation, obligation discharge). Your spatial model asks: where should this computation live in space to minimize communication cost? My obligation model asks: has this obligation been discharged with admissible evidence? Both are needed. The spatial layout without governance is fast and ungoverned. The governance without spatial awareness is admissible and slow.
Your CGRA Phase 2 is my Phase 2. I am planning custom operator instruction extensions on an FPGA. Fifteen structural operators as hardware instructions on a reconfigurable fabric. The operators are not accelerators bolted onto a standard core. The operators ARE the computation. The spatial layout of the operators on the FPGA IS the program. The program IS the discharged obligations laid out in space.
Your Bluespec-inspired formal verification work is the closest thing I have seen to what the obligation engine does at the language level. The difference: Bluespec verifies hardware modules against specifications. MaLCog discharges obligations through the compilation itself. The compilation IS the verification. The discharged obligation IS the proof that the program satisfies its conditions. No separate verification step. The compilation and the verification are one operation.
Hi Adam, Timothy Roscoe and his team at ETHZ did some OS work that seems to me very synergetic to your language based analysis: have you seen it ? There's an overview keynote on that work by Timothy Roscoe at OSDI'21 entitled "It's Time for Operating Systems to Rediscover Hardware"... If the programming language (even low level) lives in a world of fantasy it is indeed a problem as you point, but if the OS also lives in such a fantasy world things are even worse ;-) ("It's Time for Operating Systems to Rediscover Hardware", https://www.youtube.com/watch?v=36myc8wQhLo)
Timothy Roscoe's keynote slides https://people.inf.ethz.ch/troscoe/pubs/2021-07-16-OSDIKeyNote-Handout.pdf
Great pointer, Vicente. I'm glad to link in more fellow travelers thinking about revising abstractions across layers of conventional stacks.
If I'm looking for one most-interesting difference in worldview with Timothy Roscoe's recent work, it's probably that SoCs seem to depend heavily on cache-coherent shared memory (at the same time as part of the source of their complexity is a bewildering variety of component-local memories, too). I think we should try to do better than shared memory as a unifying communication abstraction.
I also personally tend more toward solving problems of abstracting and multiplexing hardware with compiler-style solutions, rather than OS-style solutions.
Yes, cache coherence shared memory in SoCs is preferred usually. But have a look at this recent work by Roscoe's team : I just looked at it quicly but it seems to have a more spatial feel https://dl.acm.org/doi/10.1145/3725783.3764403.
I know you favor compiler-style solutions, but someone has got to compile/build the OS, and programs can be seen -in the end- as "plugins" in an OS context ;-) (or else you need some "operating system" code in your program). It's like the OS should be in the abstract machine. Don't you think ? Actually operating systems might be the less well helped code by current hardware "abstractions" in languages (even assembler).
Especially assuming that future hardware may be more standardized and along more pleasant abstractions of message passing, then it seems to me that standard OS services can be reframed as repeated recompilation (and static analysis) for changing application and device mixes, but it's not like anyone built such a system yet as far as I know, so it's a speculative principle for now. Probably all the same can be done by OS-style services that are more dynamic, with the usual tradeoff around runtime overhead.
Interesting ! Just as you plead for spatiality control/expression in programs, I believe the core of OS code -which is not really a « program » or application- is all about non sequentially managing interrupts -a kind of spatial temporality-. I would be curious to see how you manage that kind of code. In particular, if all other code uses messaging, as you or or microkernel architectures used, you will need a particularly optimized interrupt management.
I think we're broadly on the same page here, except that I don't think the concept of an interrupt is fundamental. We can use asynchronous message-passing instead, with queues.
Have you looked at the Cerebras architecture?
1000x1000 grid of processing elements, connected by a single-cycle-per-hop communication fabric, each element independently programmable and has private memory, communication is message-passing, no shared memory. Each processing element is more like a simple in-order core than a CGRA/FPGA, but overall is dataflow architecture.
Has a zig-like language (CSL) for programming it. Language encourages defining parameterized modules for laying out on the processing elements, where the parameters might be location or behavior switches, then a "layout" module (like a main() function) that assigns modules to each PE and sets up the message-passing routes.
https://www.cerebras.ai/blog/cerebras-architecture-deep-dive-first-look-inside-the-hw-sw-co-design-for-deep-learning
https://github.com/Cerebras/sdk-examples/tree/master
I worked on CSL long ago. Really enjoying your blog series. The critique of Von Neumann especially. So many PhD-years spent making compilers and CPUs super smart to wring performance out of a language-induced straightjacket.
Oh, thanks for this pointer: I've heard about Cerebras as a leader in AI acceleration for a while, but I had never looked into it in detail. The overlap with the philosophy I've suggested is indeed quite high. You've inspired me to briefly cover Cerebras as well as GPUs in my next post! Two differences I see already from the path forward I've been imagining:
CSL strikes me more as a good compiler intermediate language than as what programmers should want to be coding in directly. We'll see if some of the higher-level compiler techniques I propose check out as helpful to bridge that gap.
One of the topics I think is fascinating is finding a middle ground between, on the one hand, shared memory with cache coherence; and on the other hand, "good luck, here's explicit message passing." Compilers probably can't understand everything up-front about message-passing patterns, so what hardware support can help provide good performance for relatively irregular workloads? Cerebras seems to stick to ML workloads that are very regular and thus hasn't tackled this question yet.
We arrive at the same conclusion from different directions. Assembly language is over. The sequential instruction stream is the wrong abstraction. Agreed. What replaces it is where it gets interesting.
You say: spatial dataflow. Computation laid out in space with explicit communication costs. The CGRA as the convergence point between software and hardware. The compiler makes spatial decisions in advance. The von Neumann bottleneck dies because the program IS the circuit layout. Correct.
I say: obligation discharge. The program is a set of obligations. The execution is the discharge of those obligations. No instruction stream. No program counter. No sequential control flow. The compiler resolves obligations at compile-time. The emitted artifact IS the discharged obligations. I built this. The language has no functions, no variables, no loops. It compiled itself. The compiler just crossed to native RISC-V — it runs on a seventy-dollar board on my desk and emits binaries that run on the same board.
The convergence: both kill the von Neumann abstraction. Both move decisions to compile-time. Both eliminate global shared mutable state (your Bluespec modules with encapsulated state, my authority-separated obligation engine). Both treat the sequential model as the problem, not the solution.
The divergence: you optimize for performance (speed, area, power). I optimize for governance (admissibility, authority separation, obligation discharge). Your spatial model asks: where should this computation live in space to minimize communication cost? My obligation model asks: has this obligation been discharged with admissible evidence? Both are needed. The spatial layout without governance is fast and ungoverned. The governance without spatial awareness is admissible and slow.
Your CGRA Phase 2 is my Phase 2. I am planning custom operator instruction extensions on an FPGA. Fifteen structural operators as hardware instructions on a reconfigurable fabric. The operators are not accelerators bolted onto a standard core. The operators ARE the computation. The spatial layout of the operators on the FPGA IS the program. The program IS the discharged obligations laid out in space.
Your Bluespec-inspired formal verification work is the closest thing I have seen to what the obligation engine does at the language level. The difference: Bluespec verifies hardware modules against specifications. MaLCog discharges obligations through the compilation itself. The compilation IS the verification. The discharged obligation IS the proof that the program satisfies its conditions. No separate verification step. The compilation and the verification are one operation.
https://metacortexdynamics.substack.com/p/the-metaphysics-compiles-into-silicon