Skip to content

Commit e9f0d3d

Browse files
adamgerhantadamgerhant
committed
incremental compilation
1 parent 08c2d05 commit e9f0d3d

File tree

1 file changed

+148
-0
lines changed

1 file changed

+148
-0
lines changed

‎node-graph/rfcs/incremental-compilation.md

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
2+
# Incremental Compilation
3+
Any time a NodeInput is changed in the network the following changes to the borrow tree are necessary:
4+
5+
1. Insert the new upstream nodes into the borrow tree based on their SNI
6+
2. Remove upstream SNI's which are now orphaned
7+
3. Update the SNI of the downstream nodes (Not necessary if ghostcell is used and downstream nodes can have their inputs mutated)
8+
4. Reset any downstream caches
9+
10+
We currently clone and recompile the entire network, which is very expensive. As networks grow larger and functionality gets split into separate nodes, a reasonable network may have hundreds of thousands of nodes. I believe with the following data structure and algorithm changes, it will be possible to implement incremental compilation, where a diff of changes is sent to the borrow tree. The goal is to only recompile the network upstream of the input change, and then reconstruct all downstream nodes. However, this is complicated by deduplication and compilation caching requirements, which requires storing a global cache state in the editor for compilation metadata.
11+
12+
# Editor changes:
13+
The editor now performs all compilation, type resolution, and stable node id generation.
14+
15+
```rust
16+
struct SNI(u64);
17+
type ProtonodeInput = (Vec<NodeId>, usize);
18+
```
19+
SNI represents the stable node id of a protonode, which is calculated based on the hash of the inputs SNI + implementation string, or the hash of the value. Protonode SNI's may become stale, in which case the counter in CompiledProtonodeMetadata is incremented
20+
21+
ProtonodeInput is the path to the protonode DocumentNode in the recursive NodeNetwork structure, as well as the input index.
22+
23+
```rust
24+
PortfolioMessageHandler {
25+
pub compiler_metadata: HashMap<SNI, CompilerMetadata>
26+
}
27+
28+
// Used by the compiler and editor to send diff based changes to the runtime and cache the types so the constructor can be easily looked up
29+
pub struct CompilerMetadata {
30+
resolved_type: NodeIOTypes,
31+
// How many document nodes with this SNI exist. Could possibly be moved to the executor
32+
usages: usize,
33+
}
34+
```
35+
36+
The portfolio message handler stores a mapping of types for each SNI in the borrow tree. This is shared across documents, which also share a runtime. It represents which nodes already exist in the borrow tree, and thus can be replaced with ProtonodeEntry::Existing(SNI). It also stores the usages of each SNI. If it drops to zero after a compilation, then it is removed from the borrow tree.
37+
38+
39+
```rust
40+
DocumentNodeImplementation::ProtoNode(DocumentProtonode)
41+
42+
enum DocumentProtonode {
43+
stable_node_id: Option<SNI>,
44+
cache_output: bool,
45+
identifier: ProtonodeIdentifier,
46+
// Generated during compile time, and used to unload downstream SNI's which relied on this protonode
47+
callers: Vec<ProtonodePath>,
48+
}
49+
50+
```
51+
52+
The editor protonode now stores its SNI. This is used to get its type information, thumbnail information, and other metadata during compilation. It is set during compilation. If it already exists, then compilation can skip any upstream nodes. This is similar to the MemoHash for value inputs, which is also changed to follow the same pattern for clarity.
53+
54+
Cache protonodes no longer exist, instead the protonode is annotated to insert a cache on its output. The current cache node can be replaced with an identity node with this field set to true. This field is set to true during compilation whenever the algorithm decides to add a cache node. Currently, this is whenever context nullification is added, but more advanced rules can be implemented. The overall goal is to add a cache node when a protonode has a high likely hood of being called multiple times with the same Context, takes a long time to evaluate, and returns a small amount of data. A similar algorithm could also be used to be determine when an input should be evaluated in another thread.
55+
56+
```rust
57+
pub enum NodeInput {
58+
Value { ValueInput },
59+
...
60+
}
61+
62+
pub struct ValueInput {
63+
value: TaggedValue,
64+
stable_node_id: SNI,
65+
...metadata
66+
}
67+
```
68+
This is a simple rename to change the current memo hash to SNI, which makes it more clear that it is the same concept as for protonodes.
69+
70+
# Editor -> Runtime changes
71+
These structs are used to transfer data from the editor to the runtime in CompilationRequest. Eventually the goal is move the entire NodeNetwork into the runtime, in which case the editor will send requests such as AddNode and SetInput. Then it will send queries to get data from the network.
72+
73+
```rust
74+
pub struct ProtoNetworkUpdate {
75+
// Topologically sorted
76+
nodes: Vec<ProtonodeEntry>,
77+
}
78+
```
79+
This represents the compiled proto network which is sent to the runtime and used to update borrow tree
80+
81+
```rust
82+
pub enum ProtonodeEntry {
83+
// A new SNI that does not exist in the borrow tree or in the ProtoNetwork
84+
NewProtonode(SNI, ConstructionArgs),
85+
// A protonode's SNI already exists in the protonetwork update
86+
Deduplicated,
87+
// Remove a node from the borrow tree when it has no callers
88+
Remove(SNI)
89+
}
90+
```
91+
92+
Used to transfer information from the editor to the runtime.
93+
94+
```rust
95+
pub enum ConstructionArgs {
96+
/// A value of a type that is known, allowing serialization (serde::Deserialize is not object safe)
97+
Value(TaggedValue),
98+
Nodes(NodeConstructionArgs),
99+
/// Used for GPU computation to work around the limitations of rust-gpu.
100+
Inline(InlineRust),
101+
}
102+
```
103+
Im not sure what Inline is
104+
105+
```rust
106+
pub Struct NodeConstructionArgs {
107+
//used to get the constructor
108+
pub type: NodeIOTypes,
109+
pub id: ProtonodeIdentifier,
110+
/// A mapping of each input to the SNI, used to get the upstream SharedNodeContainers for the constructor
111+
inputs: Vec<SNI>,
112+
}
113+
```
114+
The goal of NodeConstruction args is to create the SharedNodeContainer that can be inserted into the borrow tree. It does this by using the id/types to get the implementation constructor, then using the vec of input SNI's to get references to the upstream inserted nodes.
115+
116+
117+
## Runtime Changes
118+
The runtime only contains the borrow tree, which has to have some way of iterating in topological order from any point.
119+
```rust
120+
pub struct BorrowTree {
121+
nodes: HashMap<SNI, (SharedNodeContainer,NodeConstructor)>,
122+
}
123+
124+
The SNI is used to perform the correct updates/introspection to the borrow tree by the editor. It doesnt have to be correct, it just has to match the editor state.
125+
126+
127+
# Full Compilation Process
128+
129+
Every time an input changes in the editor, it first performs a downstream traversal using the callers field in the protonode, and sets all Downstream SNI's to None. It also decrements the usages of that SNI by one. If it reaches 0, then it must no longer exist since there has been an upstream change, so its SNI must have changed. Save a vec of SNI to the protonetwork. It then performs an upstream traversal to create a topologically sorted ProtoNetwork. This would ideally be done non recursively, as networks are typically very deep.
130+
131+
The traversal starts at the root export, and gets the SNI of all inputs. gets the SNI of the upstream node. If it is None, then continue traversal. When coming up the callstack, compute the SNI based on the inputs and compute/save the type if it is not saved. Then, increment the usages by one.
132+
If the type already exists, then add a Deduplicated entry to the network. Continue to the root.
133+
134+
Next, iterate over the old input and decrement all usages by one. If it reaches 0, push Remove(SNI) to the protonetwork. Finally, push the saved vec of downstream nodes to remove.
135+
136+
it is now sent to the runtime which inserts or removes the corresponding nodes.
137+
138+
139+
Overall Benefits:
140+
-Compilation is now performed in the editor, which means typing information/SNI generation is all moved into the editor, and nothing has to be returned.
141+
142+
- NodeNetwork doesnt have to be hashed/cloned for every compilation request. It doesnt have to be hashed, since every set compilation request is guaranteed to change the network.
143+
144+
- Much easier to keep runtime in sync and prevent unnecessary compilations since compilation requests don't have to be manually added
145+
146+
- The mirrored metadata structure in network interface can be removed, and all metadata can be stored in `DocumentNode`/`NodeInput`, as they are now editor only.
147+
148+
- Compilation only has to be performed on nodes which do not have an SNI, and can stop early if the SNI already exists in `compiler_metadata`

0 commit comments

Comments
 (0)