As a guest user you are not logged in or recognized by your IP address. You have
access to the Front Matter, Abstracts, Author Index, Subject Index and the full
text of Open Access publications.
This paper proposes a novel method for sequence modelling which we call Seq2Space. The basic idea is to project sequential information into the channel dimension. The Seq2Space layer outperforms the Transformer on every dataset contained in the Long Range Arena (LRA) benchmark as well as on the WSJ0-2 Mix benchmark for single-channel speech separation. Compared to previous methods which were tested on the LRA, the proposed Seq2Space layer does not quite reach the accuracy of the convolution-based methods. It is, however, more than twice as fast as the next fastest method as well as the most memory efficient, and still reaches an average accuracy of 71.15%. On the WSJ0-2Mix, the Seq2Space layer outperforms all other sequence modelling methods in our experiments except for the MEGA layer. By replacing Transformers with the Seq2Space layer on a current SOTA method, we are able to reach 22.8 dB SI-SDR improvement, which is comparable to current SOTA while being significantly faster and more memory efficient during both training and inference.
This website uses cookies
We use cookies to provide you with the best possible experience. They also allow us to analyze user behavior in order to constantly improve the website for you. Info about the privacy policy of IOS Press.
This website uses cookies
We use cookies to provide you with the best possible experience. They also allow us to analyze user behavior in order to constantly improve the website for you. Info about the privacy policy of IOS Press.