Exp matcher #65

dboyliao · 2019-04-13T07:42:01Z

No description provided.

- permutation aware - compatible ops aware - MetaOpInfo

fails at quantization: missing attribute

…tizedMul

TODO: copy attributes to the fused node

matcher replace in progress

dboyliao · 2019-06-16T01:44:23Z

@mbartling I know.
It takes me a while to pull these code off.
But I suggest not to go too deep into the code, just review the overall interface and pay attention to obvious mistake such as typo, styling or simple performance issue.

Feel free to ask me anything.

mbartling

Over all looks excellent! Great work!

I am a fan of the verbosity and configurability of the matcher architecture. Personally, I'd like to build a convenience layer on top of this (mostly around the subgraph declaration) to simplify general development. And if the simplification is not enough, then we can always fall back to full control mode you built.

mbartling · 2019-06-18T14:02:40Z

README.md

+
+## Subgraph Isomorphic Matcher and Nodes Fusion
+
+With `uTensorGraphMatcher`, performing a subgraph isomorphic match, replace or manipulate the matched subgraph is just a few line of code!


With uTensorGraphMatcher, performing common subgraph tasks such as isomorphic matching along with replacing or manipulating the matched subgraph(s) takes just a few line of code!

We should make a note here that future transforms will be based on our own graph constructor/builder.

mbartling · 2019-06-18T14:03:19Z

README.md

+
+With `uTensorGraphMatcher`, performing a subgraph isomorphic match, replace or manipulate the matched subgraph is just a few line of code!
+
+### Node Fusion


For these few examples, you might want one or two sentences explaining what they user is looking for.

Right. I'll elaborate it more.

mbartling · 2019-06-18T14:08:51Z

utensor_cgen/backend/code_generator.py

@@ -81,8 +81,6 @@ def formatter(name, kwargs):
    )
    quant_ugraph = self._transform_graph(ugraph,
                                         self.trans_methods)
-    from utensor_cgen.ir.misc.graph_viz import viz_graph


Thank you for removing this :)

No problem, bro.

mbartling · 2019-06-18T14:12:48Z

utensor_cgen/backend/operators.py

@@ -18,7 +21,7 @@ class OperatorFactory():
  def createOperatorSnippet(self, op_info, **kwargs):
    op_type = op_info.op_type
    if op_type not in self._operators:
-      err_msg = "unsupported op type in uTensor: {}".format(op_type)
+      err_msg = "unsupported op type in uTensor: {op.name}, {op.op_type}".format(op=op_info)


This is just a thought, but we might consider creating a default "Unsupported Op" that takes on any unknown op. Rather than exiting out, we can have an option for --ignore-unsupported which will do it's best to continue and attempt to generate the code with a ctx.push(new {{op.name}}(....)); bit.

I prefer fail fast in python.
Because it's the whole point of dev in python.
Fail fast, iterate faster.

mbartling · 2019-06-18T14:14:42Z

utensor_cgen/backend/operators.py

 @OperatorFactory.register
+@OpEqualityDelegate.is_compatible_with("Const", _morphism.Inline2ConstMorphism)


I like this format

Not sure if morphism is a good idea.
Not very useful now. Maybe it'll do some cool stuff later (?

Assuming the morphisms can chain this could be super useful. Doesnt seem too dangerous

Might be useful for stuff like architecture search.
However, we should also define the transform functions morphing from one graph to its equivalences. In addition to op_type we should also define how the tensor values need to be modified.
Seems doable with the current setup.

Yes.
But I still have a gut feeling that the current setup may not be good enough.
Need a few iterations of refactoring. We'll see. : )

mbartling · 2019-06-18T14:18:53Z

utensor_cgen/transformer/conv_pool.py

+    super(ConvPoolTransformer, self).__init__(prune_graph=False)
+
+  @property
+  def pattern_ugraph(self):


Eventually it would be nice to figure out a better way of declaring the sub graphs. but the actual matching bit is so much nicer than it used to be

Sure.
But it'll like doing our own Tensorflow python api again.
Not of first priority for now.

Completely agree, let's just add this as a TODO feature in the issue tracker

Right. Will do.

mbartling · 2019-06-18T14:20:15Z

utensor_cgen/transformer/conv_pool.py

+      ugraph=repl_ugraph
+    )
+    topologic_order_graph(repl_ugraph)
+    input_map = {


These lines can be procedurally done right?

I think it's non-trivial how to map the inputs/outputs of pattern graph and matched subgraph.
So I have to leave it to the developers.

mbartling · 2019-06-18T14:21:41Z

utensor_cgen/transformer/linear_reoder.py

+  @property
+  def pattern_ugraph(self):
+    graph = tf.Graph()


Probably an insanely stupid question, but what does this subgraph have to do with LinearReorder?

You mean the naming?
I just follow @neil-tan's naming.

@neil-tan The hell are you doing here? XD

I know this.
Just rearrange the layers to save memory.
relu -> maxpool is equivalent to maxpool -> relu
And @neil-tan call this linear_reorder.

@mbartling this is to reduce the memory needed to store the activation by taking advantage of down-sampling

neil-tan

This is good.
Many of the points have already been mentioned in the discussion. On the graph builder-constructor, we should aim to eliminate framework dependencies in our IR-space (after parsers and before backends). For this PR, we should make this clear to developers in the Readme and source code (transforms).

Please let us know how we can help with the graph-builder.

neil-tan · 2019-07-02T10:02:44Z

utensor_cgen/backend/operators.py

@@ -46,6 +55,9 @@ def snippet(self):


 @OperatorFactory.register
+@OpEqualityDelegate.is_associative(
+  permutations=((0, 1), (1, 0))


Does this support groupings? For example;
q_add(a, a_min, a_max, b, b_min, b_max) == q_add( b, b_min, b_max, a, a_min, a_max)

I don't get it.
The later can be configured as (3, 4, 5, 0, 1, 2).
Well, a tuple is kinda a formal math notation for permutation.

All you have to do is to list all possible permutations in this decorator.

Ok. As long as this supports multiple permutations. It shouldn’t be a problem since the numbers of operants are usually small.

neil-tan · 2019-07-02T10:03:59Z

README.md

+
+## Subgraph Isomorphic Matcher and Nodes Fusion
+
+With `uTensorGraphMatcher`, performing a subgraph isomorphic match, replace or manipulate the matched subgraph is just a few line of code!


We should make a note here that future transforms will be based on our own graph constructor/builder.

neil-tan · 2019-07-02T10:04:52Z

utensor_cgen/backend/operators.py

 @OperatorFactory.register
+@OpEqualityDelegate.is_compatible_with("Const", _morphism.Inline2ConstMorphism)


Might be useful for stuff like architecture search.
However, we should also define the transform functions morphing from one graph to its equivalences. In addition to op_type we should also define how the tensor values need to be modified.
Seems doable with the current setup.

neil-tan · 2019-07-02T10:16:33Z

utensor_cgen/transformer/linear_reoder.py

+  @property
+  def pattern_ugraph(self):
+    graph = tf.Graph()


@mbartling this is to reduce the memory needed to store the activation by taking advantage of down-sampling

mbartling · 2019-07-11T14:01:22Z

Note: This PR passes integration tests with the following PB file
mnist_for_mc.tar.gz

- README.md -> README.rst

dboyliao added 3 commits April 13, 2019 15:39

Simple BFS Graph Matcher

4a6f3ba

- permutation aware - compatible ops aware - MetaOpInfo

Add tests (failed)

6daeb73

Fix bug

2d9d193

dboyliao requested review from Knight-X, mbartling and neil-tan April 13, 2019 07:42

dboyliao and others added 21 commits April 25, 2019 00:20

Syncing develop branch

291f9c7

syncing with develop, fix conflicts

f15048f

Add replace util methods

725e9d9

update tests

ee54a0b

linear_reorder: wip

53065fd

fails at quantization: missing attribute

ugraph seems correct, but tf quantization complains

0f802fb

quantization passed, good until: unsupported op type in uTensor: Quan…

574f4d7

…tizedMul

CONV_POOL_Transformer WIP: matcher graph completed

b046b82

fused, but with graph integrity problem

7870206

output graph topologically correct

55fbf77

TODO: copy attributes to the fused node

attaching op_attribs to the fused-node

eb1a510

Merge branch 'f/conv_pool_fusion' into PR69

324287c

fix up

d232e02

fixing the types

c7687e2

resolving conflict

e66bf54

generates some source files, weights file missing

82f5e96

inline transformer enabler hotfix

51641b8

fixing GatherOpSnippet

428f59c

minor bug fix

4a2cb98

matcher replace in progress

subgraph view

4fc9b14

fix bug

1bf33d5

dboyliao force-pushed the exp-matcher branch from 815b3d0 to 1bf33d5 Compare May 9, 2019 16:33

dboyliao added 2 commits May 10, 2019 00:35

merge develope: sync bug fix and resolve conflict

4a07522

null tensor match support

ad00ee1

dboyliao force-pushed the exp-matcher branch from 9bb77c5 to fae1129 Compare June 11, 2019 15:38

dboyliao added 2 commits June 12, 2019 00:10

Update README and images

e90771a

Add old dropout trans back, update tests and add subcmd

2671be1

dboyliao force-pushed the exp-matcher branch from ac97767 to 2671be1 Compare June 12, 2019 15:20

dboyliao added 5 commits June 12, 2019 23:47

minor fix

1121d72

Add lockfile

0e46059

merge develop and resolve conflicts

18a7697

fix bug and make op name pattern as kwarg to dropout tranformer

9d52464

update doc str

85c1758

dboyliao changed the base branch from develop to ubit June 16, 2019 01:47

mbartling reviewed Jun 18, 2019

View reviewed changes

dboyliao added 2 commits June 21, 2019 02:36

add flags to show subcmd

3648a27

Few bug fixs

5ae22f7

neil-tan requested changes Jul 2, 2019

View reviewed changes

dboyliao added 4 commits July 4, 2019 00:46

Fix bug and update REAME.md

8744bb1

Add explanation to matcher show cases in README.md

665ecfd

Fix typos

f56b40b

Fix quant fused conv op snippet bug

40f846f

dboyliao force-pushed the exp-matcher branch from f25eac0 to 40f846f Compare July 11, 2019 13:34

remove old experimental matcher and cmsis nn transformer

9d4153c

dboyliao changed the base branch from ubit to develop July 14, 2019 17:38

dboyliao added 5 commits July 18, 2019 00:37

update README.md

ca0d37b

- setup basic doc files

ea99036

- README.md -> README.rst

update README

5677dd4

Fix imgs path

c3af9d3

Add ir chapter

5024724

neil-tan merged commit 7094964 into develop Jul 26, 2019

dboyliao deleted the exp-matcher branch August 3, 2019 09:18


		## Subgraph Isomorphic Matcher and Nodes Fusion

		With `uTensorGraphMatcher`, performing a subgraph isomorphic match, replace or manipulate the matched subgraph is just a few line of code!


		With `uTensorGraphMatcher`, performing a subgraph isomorphic match, replace or manipulate the matched subgraph is just a few line of code!

		### Node Fusion

		@OperatorFactory.register
		@OpEqualityDelegate.is_compatible_with("Const", _morphism.Inline2ConstMorphism)

Exp matcher #65

Exp matcher #65

Conversation

dboyliao commented Apr 13, 2019

dboyliao commented Jun 16, 2019

mbartling left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dboyliao Jun 18, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neil-tan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbartling commented Jul 11, 2019

dboyliao Jun 18, 2019 •

edited

Loading