Skip to content

Commit 05e8882

Browse files
dboyliaodboyliao
committed
Merge branch 're-arch-support' into re-arch-support-extra-ops
2 parents 0b4af2d + 7c4b0fe commit 05e8882

File tree

33 files changed

+1043
-90
lines changed

33 files changed

+1043
-90
lines changed

‎Makefile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,9 @@ tests:
77

88
test_%:
99
@if [ -d .venv ]; then \
10-
pipenv run pytest tests/$@ -vv -s | tee -a tests_log.txt; \
10+
pipenv run pytest tests/$@ -m 'not slow_test and not deprecated' -vv -s | tee -a tests_log.txt; \
1111
else \
12-
pytest tests/$@ -vv -s | tee -a tests_log.txt; \
12+
pytest tests/$@ -vv -s -m 'not slow_test and not deprecated' | tee -a tests_log.txt; \
1313
fi;
1414

1515
package:

‎tutorials/README.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Overview
2+
========
3+
4+
- `end2end_training`: notebooks for end-to-end training with `utensor_cgen`
5+
- `component_registration`: `utensor_cgen` is highly customizable, which means you can register your own frontends, transformers, backends and more.

‎tutorials/write_plugins/.gitignore renamed to ‎tutorials/component_registration/.gitignore

File renamed without changes.

‎tutorials/component_registration/README.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
## The Architecture of `utensor_cgen`
2+
3+
![utensor-cli-components](images/utensor-cli-components.drawio.svg)
4+
5+
- `utensor_cgen` consists of multiple **components**, such as frontend component and backend component
6+
- Each component consists of multiple **parts**. For example, a backend may be composed of a graph lower part and a code generator part
7+
- Most of the compoents found in the graph above is customizable. The user can register their own component to `utensor_cgen`
8+
9+
## Examples
10+
11+
- `backend_registration.ipynb`: example notebook for backend registration, this allows you to inject customized backend to the `utensor-cli`
12+
- `frontend_registration.ipynb`: example notebook for frontend parser registration, this allows you to inject customized frontend parser to the `utensor-cli`
13+
- `transformer_registration.ipynb`: example notebook for transformer registration, this allows you to inject customized transformer into the transformation pipeline in `utensor-cli`
14+
15+
## Load Plugin
16+
17+
- with `utensor-cli`
18+
- You can use `--plugin` flag to load the plugins
19+
- It's a multiple flag. So you can pass multiple values to the cli
20+
- ex: `utensor-cli --plugin plugin1 --plugin plugin2 [convert|generate-config|...]`
21+
- both `plugin1` and `plugin2` will be loaded
22+
- Note that `--plugin` must be passed before any subcommand such as `convert`
23+
- with script
24+
- simply import your plugins and it should work
25+
26+
```python
27+
import plugin1, plugin2
28+
import utensor_cgen
29+
30+
# do stuffs with utensor_cgen and your plugins
31+
```

‎tutorials/write_plugins/backend_registration.ipynb renamed to ‎tutorials/component_registration/backend_registration.ipynb

Lines changed: 68 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -12,13 +12,29 @@
1212
" - `apply` method, which is responsible for generating files or anything you want with the given `ugraph`"
1313
]
1414
},
15+
{
16+
"cell_type": "markdown",
17+
"metadata": {},
18+
"source": [
19+
"### The `Backend` interface\n",
20+
"\n",
21+
"1. the class attribute `TARGET` should be overwriten and be of type `str`.\n",
22+
" - the value of `TARGET` should be unique or `ValueError` will be raised\n",
23+
"2. one should overwrite the `default_config` class property\n",
24+
"3. the first argument of `__init__` should be the config dictionary\n",
25+
" - this dictionary can be generated via a toml file (recommended)\n",
26+
" - this dictionary should be of the same format as the value returned by `default_config` class property\n",
27+
"\n",
28+
"Here is a simple example:"
29+
]
30+
},
1531
{
1632
"cell_type": "code",
1733
"execution_count": 1,
1834
"metadata": {
1935
"ExecuteTime": {
20-
"end_time": "2020-05-15T06:16:52.700065Z",
21-
"start_time": "2020-05-15T06:16:52.266124Z"
36+
"end_time": "2020-05-27T03:50:50.714504Z",
37+
"start_time": "2020-05-27T03:50:50.242148Z"
2238
}
2339
},
2440
"outputs": [],
@@ -59,20 +75,51 @@
5975
" }\n"
6076
]
6177
},
78+
{
79+
"cell_type": "code",
80+
"execution_count": 2,
81+
"metadata": {
82+
"ExecuteTime": {
83+
"end_time": "2020-05-27T03:51:06.947435Z",
84+
"start_time": "2020-05-27T03:51:06.943073Z"
85+
}
86+
},
87+
"outputs": [
88+
{
89+
"name": "stdout",
90+
"output_type": "stream",
91+
"text": [
92+
"dummy-backend backend\n"
93+
]
94+
}
95+
],
96+
"source": [
97+
"print(DummyBackend.TARGET, DummyBackend.COMPONENT)"
98+
]
99+
},
62100
{
63101
"cell_type": "markdown",
64102
"metadata": {},
65103
"source": [
66-
"## Backend Registration"
104+
"Basically, this dummy backend takes a graph and generate a `.c` file which will print out all the operators in the graph."
105+
]
106+
},
107+
{
108+
"cell_type": "markdown",
109+
"metadata": {},
110+
"source": [
111+
"## Backend Registration\n",
112+
"\n",
113+
"Once you create a `Backend`, you can registrate it via `BackendManager` as following:"
67114
]
68115
},
69116
{
70117
"cell_type": "code",
71-
"execution_count": 2,
118+
"execution_count": 3,
72119
"metadata": {
73120
"ExecuteTime": {
74-
"end_time": "2020-05-15T06:16:52.724946Z",
75-
"start_time": "2020-05-15T06:16:52.714719Z"
121+
"end_time": "2020-05-27T03:51:19.277175Z",
122+
"start_time": "2020-05-27T03:51:19.265963Z"
76123
}
77124
},
78125
"outputs": [
@@ -82,7 +129,7 @@
82129
"__main__.DummyBackend"
83130
]
84131
},
85-
"execution_count": 2,
132+
"execution_count": 3,
86133
"metadata": {},
87134
"output_type": "execute_result"
88135
}
@@ -102,11 +149,11 @@
102149
},
103150
{
104151
"cell_type": "code",
105-
"execution_count": 3,
152+
"execution_count": 4,
106153
"metadata": {
107154
"ExecuteTime": {
108-
"end_time": "2020-05-15T06:16:53.407285Z",
109-
"start_time": "2020-05-15T06:16:53.399436Z"
155+
"end_time": "2020-05-27T03:51:22.426714Z",
156+
"start_time": "2020-05-27T03:51:22.416453Z"
110157
}
111158
},
112159
"outputs": [],
@@ -116,11 +163,11 @@
116163
},
117164
{
118165
"cell_type": "code",
119-
"execution_count": 4,
166+
"execution_count": 5,
120167
"metadata": {
121168
"ExecuteTime": {
122-
"end_time": "2020-05-15T06:16:58.423617Z",
123-
"start_time": "2020-05-15T06:16:54.435841Z"
169+
"end_time": "2020-05-27T03:51:28.055665Z",
170+
"start_time": "2020-05-27T03:51:23.803717Z"
124171
}
125172
},
126173
"outputs": [],
@@ -130,11 +177,11 @@
130177
},
131178
{
132179
"cell_type": "code",
133-
"execution_count": 5,
180+
"execution_count": 6,
134181
"metadata": {
135182
"ExecuteTime": {
136-
"end_time": "2020-05-15T06:16:58.447131Z",
137-
"start_time": "2020-05-15T06:16:58.426076Z"
183+
"end_time": "2020-05-27T03:51:30.645094Z",
184+
"start_time": "2020-05-27T03:51:30.638159Z"
138185
}
139186
},
140187
"outputs": [
@@ -158,11 +205,11 @@
158205
},
159206
{
160207
"cell_type": "code",
161-
"execution_count": 6,
208+
"execution_count": 7,
162209
"metadata": {
163210
"ExecuteTime": {
164-
"end_time": "2020-05-15T06:16:58.606631Z",
165-
"start_time": "2020-05-15T06:16:58.457135Z"
211+
"end_time": "2020-05-27T03:52:03.395036Z",
212+
"start_time": "2020-05-27T03:52:03.258055Z"
166213
}
167214
},
168215
"outputs": [
@@ -234,10 +281,8 @@
234281
]
235282
},
236283
{
237-
"cell_type": "code",
238-
"execution_count": null,
284+
"cell_type": "markdown",
239285
"metadata": {},
240-
"outputs": [],
241286
"source": []
242287
}
243288
],
@@ -257,7 +302,7 @@
257302
"name": "python",
258303
"nbconvert_exporter": "python",
259304
"pygments_lexer": "ipython3",
260-
"version": "3.6.8"
305+
"version": "3.7.2"
261306
},
262307
"toc": {
263308
"base_numbering": 1,

0 commit comments

Comments
 (0)