Problems with reshaping

[p-saucedo]

Hi all,
I'm having some problems implementing a very simple model. The problem what I think I'm facing is related on how PyTorch linear layers handle tensors with more than 2D. In this case, I'm extracting some features from my two channels inputs (originally complex input) using several 1D convolutions, but when I get into the final linear layer I need so reshape and flatten (batch and channel dimensions) manually the tensors (to replicate manually the Pytorch way of working). In this point is where I'm stucked, I'm not able to Absorb or reorder the transposes and/or the Flatten node, resulting in a AssertionError: cycle-free graph violated: partition depends on itself during the creation of the dataflow partition.

I provide here the original model:

Here, just before it goes into the step_create_dataflow_partition:

How can I solve this? How can I get my model compiled?
Thanks in advance!!

[fpjentzsch]

Hi,

did you try applying this transformation:

finn/src/finn/transformation/move_reshape.py

Line 9 in 94f887b

class RemoveCNVtoFCFlatten(Transformation):

[p-saucedo]

Hi,

Thanks for your response. I’ve tested it both before and after the streamlining, and it still returns the same error.

Maybe is due to the two Transpose nodes? In fact, I'm using AbsorbConsecutiveTransposes, but still they are not collapsing...

[fpjentzsch]

Yes, the shape looks weird. FINN needs to operate on NHWC data layout.
After the MultiThreshold it looks like you have NHWC (1x448x1x128) -> NCHW (1x128x448x1) -> NHCW (1x448x128x1). The second transpose to NHCW should not be there. Can you reshape and flatten differently in your PyTorch model?

[p-saucedo]

That's exactly from where I think the problem is coming.

The Pytorch forward method of the provided model is the following:

As you can see, the Flatten layer is just to avoid the "trick" that Pytorch Linear layers uses. My original model was exactly the same but without the Flatten layer. If I try to compile it with FINN I get a problem with the shapes in the convert to hw step, caused by the missmath between the MatMul weights (128x4) and its input (1x448x128x1):
onnx.onnx_cpp2py_export.shape_inference.InferenceError: [ShapeInferenceError] (op_type:RandomNormal, node name: MVAU_MatMul_3): [ShapeInferenceError] Inferred shape and existing shape differ in dimension 2: (128) vs (4)
which takes me again to the Flatten solution.

This is the model after the streamlining and before the convert to hw step (where the error is raised):