Have you tried to not delayed stacking input (Use delayed stacking for generation, but not on input)

[zmy1116]

Hello,

I've gone through the paper and the code. I have question regarding delayed stacking input.

Let's use Y_{c, t} represent the c^th code book for time step t.

I understand that when we delayed stacking generation, we maintain the property that Y_{i,t} is generated after Y_{j,t} for all j>i, while not doing C x T generations (C is num of code books and T is the time steps).

I just don't see the intuition to delay stack the input, adding different residual part of different timestep
[ Y_{1,(t-3)}, Y_{2,{t-2}} Y_{3(t-1)} Y_(4t) ] for input doesn't make too much sense for me.

Is it mainly for ease of implementation? because I understand if the input at each step is sum of all codes at that timestep, since at generation phase, the output of different timestep will be summed and fed back to the transformer as input, we may need to do extra work to differentiate the sum of all codes at the same timestep versus sum of residuals at different timestep.

Thanks