• State Suggested Answer
  • Locked Locked
  • Replies 4 replies
  • Answers 1 answer
  • Subscribers 17 subscribers
  • Views 3181 views
  • Users 0 members are here
Options
Related
How was your experience today?
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

How to prepare tflite for vela compiler?

Danter

First of all, I'm not sure if it should be posted here.

If there is an error, please forgive me.

I want to try ethus-U NPU. (developer.arm.com/.../The-Vela-compiler

Therefore I need to use the vela compiler (pypi.org/.../)

I use the example of the following website to make a tflite (www.tensorflow.org/.../post_training_integer_quant)

Then use vela compiler to compile tflite, and then see this warning

 Warning: PACK 'sequential/reshape/Reshape/shape' is not supported on the NPU. Placing on CPU instead
  - Input(s) and Output tensors must not be dynamic
   Op has dynamic tensor(s): sequential/reshape/strided_slice2
 Warning: STRIDED_SLICE 'sequential/reshape/strided_slice2' is not supported on the NPU. Placing on CPU instead
  - Input(s) and Output tensors must not be dynamic
   Op has dynamic tensor(s): sequential/reshape/strided_slice2


I'm not sure what does ''dynamic'' mean for this compiler?

This code comes from part of the example mentioned above:

model = tf.keras.Sequential([
        tf.keras.layers.InputLayer(input_shape=(28, 28)),
        tf.keras.layers.Reshape(target_shape=(28,28,1),input_shape=(28,28,1)),
        tf.keras.layers.Conv2D(filters=12, kernel_size=(3,3),activation='relu'),
        tf.keras.layers.MaxPool2D(pool_size=(2, 2)),
        tf.keras.layers.Flatten(),
        tf.keras.layers.Dense(10)
])
l

I’m not sure if this is the case for functions like reshpae, or it’s my coding problem.

Parents
  • 0

    Hi , 

     For some network architectures we don't know at load time, what the necessary sizes will be for all the tensors (input, output and intermediate). For such dynamic tensor size specified to the model specified might be at load time or known based on the size of the inputs etc. This is called dynamic tensor and they are classified as different types. 

    Coming back to the issue, in this case it's just a print message where we are checking the tensor size: ethosu/vela/supported_operators.py

    def constraint_tens_no_dynamic(op):
        "Input(s) and Output tensors must not be dynamic"
        valid = True
        extra = []
        tensors = [tens for tens in op.inputs + op.outputs if tens]
        for tens in tensors:
            if (tens.shape == []) and (tens.values is None):
                valid = False
                extra.append(tens.name)
        extra = ", ".join(extra)
        return valid, f"Op has dynamic tensor(s): {extra}"

    The warning " Warning: PACK 'sequential/reshape/Reshape/shape'" that you are getting is because you are using float 32 and ops falling back on CPU as U55 only supports int8/int16 weights/activations. I suspect the model is not quantised.

    RESHAPE op is supported by U55 for fallback on NPU. Details - review.mlplatform.org/.../SUPPORTED_OPS.md

    Hope this helps. You can also use support-ml <support-ml@arm.com> for your issues if you are Arm License customer. 

Reply
  • 0

    Hi , 

     For some network architectures we don't know at load time, what the necessary sizes will be for all the tensors (input, output and intermediate). For such dynamic tensor size specified to the model specified might be at load time or known based on the size of the inputs etc. This is called dynamic tensor and they are classified as different types. 

    Coming back to the issue, in this case it's just a print message where we are checking the tensor size: ethosu/vela/supported_operators.py

    def constraint_tens_no_dynamic(op):
        "Input(s) and Output tensors must not be dynamic"
        valid = True
        extra = []
        tensors = [tens for tens in op.inputs + op.outputs if tens]
        for tens in tensors:
            if (tens.shape == []) and (tens.values is None):
                valid = False
                extra.append(tens.name)
        extra = ", ".join(extra)
        return valid, f"Op has dynamic tensor(s): {extra}"

    The warning " Warning: PACK 'sequential/reshape/Reshape/shape'" that you are getting is because you are using float 32 and ops falling back on CPU as U55 only supports int8/int16 weights/activations. I suspect the model is not quantised.

    RESHAPE op is supported by U55 for fallback on NPU. Details - review.mlplatform.org/.../SUPPORTED_OPS.md

    Hope this helps. You can also use support-ml <support-ml@arm.com> for your issues if you are Arm License customer. 

Children