## HIGH-LEVEL CONTROL OF DRUM TRACK GENERATION USING LEARNED PATTERNS OF RHYTHMIC INTERACTION

Stefan Lattner , Maarten Grachten 1 2

1 Sony Computer Science Laboratories (CSL), Paris, France 2 Contractor for Sony CSL, Paris, France

## ABSTRACT

Spurred by the potential of deep learning, computational music generation has gained renewed academic interest. A crucial issue in music generation is that of user control, especially in scenarios where the music generation process is conditioned on existing musical material. Here we propose a model for conditional kick drum track generation that takes existing musical material as input, in addition to a lowdimensional code that encodes the desired relation between the existing material and the new material to be generated. These relational codes are learned in an unsupervised manner from a music dataset. We show that codes can be sampled to create a variety of musically plausible kick drum tracks and that the model can be used to transfer kick drum patterns from one song to another. Lastly, we demonstrate that the learned codes are largely invariant to tempo and time-shift.

## 1. INTRODUCTION

Acrucial issue in music generation is that of user control. Especially for problems where musical material is to be generated conditioned on existing musical material (so-called conditional generation ), it is not desirable for a system to produce its output deterministically. Typically there are multiple valid ways to complement existing material with new material, and a music generation system should reflect that degree of freedom, either by modeling it as a predictive distribution from which samples can be drawn and evaluated by the user, or by letting the generated material depend on some form of user input in addition to the existing material. An intuitive way to address this requirement is to learn a latent space, for example by means of a variational autoencoder (VAE). This approach has been successfully applied to music generation [1, 2], and allows for both generation and manipulation of musical material by sampling from the latent prior, manual exploration of the latent space, or some form of local neighborhood search or interpolation.

In this paper we also take a latent space learning approach to address the issue of control over music generation. More specifically, we propose a model architecture to learn a latent space that encodes rhythmic interactions of the kick drum vs. bass and snare patterns. The architecture is a convolutional variant of a Gated Autoencoder (GAE, see Section 3). This architecture can be thought of as a feed-forward neural network where the weights are modulated by learned mapping codes [3]. Each mapping code captures local relations between kick vs bass and snare inputs, such that an entire track is associated to a sequence of mapping codes.

Since we want mapping codes to capture rhythmic patterns rather than just the instantaneous presence or absence of onsets in the tracks, during training we enforce invariance of mapping codes to (moderate) time shifts and tempo changes in the inputs. The resulting mapping codes remain largely constant throughout sections with a stable rhythm. This provides high-level control over the generated material in the sense that different kick drum patterns for some section can be realized simply by selecting a different mapping code (either by sampling or by inferring them from another section or song), and applying it throughout the section.

To our knowledge this is a novel approach to music generation. It reconciles the notion of user control with the presence of conditioning material in a musically meaningful way: rather than controlling the characteristics of the generated material directly, it offers control over how the generated material relates to the conditioning material.

Apart from quantitative experiments to show the basic validity of our approach, we validate our model by way of a set of sound examples and visualized outputs. We focus on three scenarios specifically. Firstly we demonstrate the ability to create a variety of plausible kick drum tracks for a given snare and bass track pair by sampling from a standard multivariate normal distribution in the mapping space. Secondly, we test the possibility of style-transfer , by applying rhythmic interaction patterns inferred from one song to induce similar patterns in other songs. Finally, we show that the semantics of the mapping space is invariant under changes in tempo.

In continuation we present related work (Section 2), describe the proposed model architecture and data representations (Section 3), and validate the approach (Section 4). Section 5 provides concluding remarks and future work.

## 2. RELATED WORK

In addition to the V AE-based methods for control over music generation processes mentioned above, a number of other studies have applied deep learning methods to address the problem of music generation in general, as reviewed in [4]. Drum track generation has been tackled using recurrent architectures [5, 6], Restricted Boltzmann Machines [7], and Generative Adversarial Networks (GANs) [8]. Approaches

to control the generation process may rely on sampling from some latent representation of the material to be generated [1, 2], possibly in an incremental fashion [9], or conditioning on user-provided information (such as a style label [10], unary [11], or structural [12] constraints). [13] demonstrates style transfer for audio. GANs are used in [8, 14], where the output of the generation process is determined by providing some (time-varying) noise, in combination with conditioning on existing material. Similar to our study, [15] uses a GAE to model relations between musical material in an autoregressive prediction task. To our knowledge this is the first use of GAEs for conditional music generation.

## 3. METHOD

A schematic overview of the proposed model architecture is shown in Figure 1. For time series modeling, we adapt the common dense GAE architecture to 1D convolution in time, yielding a Convolutional Gated Autoencoder (CGAE). We aim to model the rhythmic interactions between input signals x ∈ R M T × and a target signal y ∈ R 1 × T . More precisely, x represents M 1D signals of length T indicating onset functions of instrument tracks and beat- and downbeat information of a song, while y represents the onset function of a target instrument. Then the rhythmic interactions (henceforth referred to as mappings ) between x and y are defined as

$$m = \mathbf W * ( \mathbf U * x \cdot \mathbf V * y ), \quad \quad ( 1 ) \quad ^ { \text{tr} }..$$

where m ∈ R Q T × , and U ∈ R K × M R × , V ∈ R K × × 1 R represent respectively K convolution kernels for M input maps and kernel size R , and W ∈ R Q K × × 1 represents Q convolution kernels for K input maps and kernel size 1 . Furthermore, ∗ is the convolution operator and · is the Hadamard product. For brevity the notation above assumes a CGAE architecture with only one mapping layer and one layer for input and target. In practice we use several convolutional layers, as described in Section 3.1.

Given the rhythmic interactions m and the rhythmic context x , the target onset function is reconstructed as

$$\tilde { y } = V ^ { \top } * ( U * x \cdot W ^ { \top } * m ), \quad \quad ( 2 )$$

where the transposed kernels V glyph[latticetop] and W glyph[latticetop] result in a deconvolution. The model parameters are trained by minimizing the mean squared error L mse ( y ˜ , y ) between the target signal y and its reconstruction ˜ y .

In order to draw samples from the model, we want to impose a Gaussian prior over m resulting in p ( m ) = N (0 , I ) . Additionally, m should apply to any input x , and should therefore not contain any information about the content of x . These conditions are imposed using adversarial training [16]: A discriminator D ( ) · estimates whether its input is drawn from a Gaussian distribution and contains no information about x . To that end, we concatenate ( U x ∗ ) with either actual mappings m or noise drawn from an independent Gaussian distribution η ∼ N (0 , I ) , η ∈ R Q T × . This

Figure 1: The proposed model architecture.

![image](https://ac.marcodsn.me/arxiv-markdown-images/1908.00948v1-figure-6ab6d78e-5fe0-4e6b-ab74-d237c78c3ae2.jpg)

results in D ( m U , ( ∗ x )) and D ( η , ( U ∗ x )) . In adversarial training, the discriminator D ( ) · learns to distinguish between the input containing m and the input containing η . If there is mutual information between ( U ∗ x ) and m the discriminator can exploit this for its classification task. This causes the encoding pathways to remove any information about x from m . Also, we obtain m ∼ N (0 , I ) . Accordingly, the discriminator is trained to minimize the loss

$$\text{ut} _ { Q } \quad \mathcal { L } _ { \text{advers} } = \frac { 1 } { T } \sum _ { t } D ( \mathbf m, ( \mathbf U * \mathbf x ) ) _ { t } - D ( \eta, ( \mathbf U * \mathbf x ) ) _ { t }, \ \ ( 3 )$$

with D ( ) · t being the output of the discriminator at time t .

To make the mappings more constant over time, an additional loss penalizes differences of successive mappings L const = 1 T ∑ t ( m t -m t +1 ) 2 , m t ∈ R Q . A further loss that constrains each map m q ∈ R T to have zero mean and unit variance over time and instances in a batch considerably improves the learning of the CGAE:

$$2 ) \quad \mathcal { L } _ { \text{std} } = \frac { 1 } { Q } \sum _ { q } ^ { Q } \left [ \left ( \frac { 1 } { N } \sum _ { i } ^ { N } \left ( m _ { q, i } - \mu _ { q } \right ) ^ { 2 } \right ) - 1 \right ] ^ { 2 } + \mu _ { q } ^ { 2 }, \ \ ( 4 )$$

where m q,i are the observations of convolutional map m q over all time steps and instances in a batch, and µ q is the mean of m q,i . Optimization is performed in two steps per mini-batch. First, the discriminator is trained to minimize L advers , then the CGAE is trained to minimize L mse ( y ˜ , y ) + L const + L std -L advers .

## 3.1. Architecture and training details

As mentioned above the weight matrices W U , and V in Eqs. 1 and 2 act as placeholders for several convolutional layers. For U and V , 8 convolutional layers are defined, with { 32 32 64 64 64 128 128 256 , , , , , , , } output units, kernel size 2 , and dilations which double for each layer (i.e., 1 2 4 8 , , , , . . . ).

The first 5 layers keep the 4 inputs (onset strength snare, onset strength bass, beats, downbeats) separated (i.e., their units are separated in 4 groups, where each group only pools over 1 / 4 th of the input maps), and the information is combined only in the two top-most layers. W is a placeholder for 6 layers, with sizes { 128 128 64 32 32 16 , , , , , } , and kernel size 1 . The discriminator D ( ) · consists of 5 layers with { 256 128 64 64 1 , , , , } maps and kernel size 1 . All stacks described above have no non-linearity in the output and SELU non-linearity [17] between layers (also for the deconvolution passes).

The model is trained for 2500 epochs with batch size 100 , using 50% dropout on the inputs x . During training, a data augmentation based regularization method is used to make the mappings invariant to time shift and tempo change. To that end we define a transformation function ψ θ ( z ) that shifts and scales a signal z in the time dimension with random shifts between -150 and +150 time steps ( ± 1 75 . s ) and random scale factors between 0 8 . and 1 2 . . Training is then performed as follows. First, the mappings m are inferred according to Eq. 1. Then, the input signals are modified using ψ θ ( ) · resulting in an altered Eq. 2:

$$\tilde { y } _ { \psi _ { \theta } } = V ^ { \top } * ( U * \psi _ { \theta } ( x ) \cdot W ^ { \top } * m ). \quad ( 5 ) \quad \text{$\quad $\quad$\quad}$$

Finally, the mean squared error between the such obtained reconstruction and the transformed target is minimized as L mse ( ψ θ ( y ) , ˜ y ψ θ ) . This approach was first proposed in [18]. Due to the gating mechanism · (activating only pathways with appropriate tempo and time-shift), a CGAE is particularly suited for learning such invariances. By imposing time-shift invariance, we assume that rhythmic interaction patterns (and the respective mappings) in the training data are locally constant. Even if this method introduces some error at positions where rhythmic patterns change, most of the time the assumption of locally constant rhythm is valid.

## 3.2. Data representation

The training/validation sets consist of 665 / 193 pop/rock/ electro songs where the rhythm instruments bass, kick and snare are available as separate 44.1kHz audio tracks. The context signals x consist of two 1D input maps for beat and downbeat probabilities, and two 1D input maps for the onset functions of Snare and Bass. The target signal y consists of a 1D onset function of the Kick drum. The onset functions are extracted using the ComplexDomainOnsetDetection feature of the publicly available Yaafe library 1 with a block size of 1024 , a step size of 512 , and a Hann window function. For the downbeat functions we use the downbeat estimation RNN of the madmom library 2 . Input signals are individually standardized to zero mean and unit variance over each song.

1 http://yaafe.sourceforge.net

2 https://github.com/CPJKU/madmom

## 3.3. Rendering Audio

We create an actual kick drum track from an onset strength curve y using salient peak picking. First, we remove less salient peaks from y by zero-phase filtering with a low-pass Butterworth filter of order two and a critical frequency of half the Nyquist frequency. The local maxima of the smoothed curve are then thresholded, discarding all maxima below a certain proportion (see below) of the maximum of the standardized onset strength curve. The remaining peaks are selected as onset positions. Finally, we render an audio file by placing a 'one-shot' drum sample on all remaining peaks after thresholding. We introduce dynamics by choosing the volume of the sample from 70% for peaks at the threshold to 100% for peaks with the maximum value. For the qualitative experiments in the following section, we manually choose the threshold between 15% and 50% . For the quantitative results in Table 1 we fix the threshold at 25% , but values of 20% and 30% yield similar figures.

## 4. EXPERIMENTS

For the qualitative experiments we use four songs, Gipsy Love , Orgs Waltz , Miss You and Drehscheibe , produced by the first author. We encourage the reader to listen to the results on the accompanying web page 3 . Three scenarios are chosen to show the effectiveness of the proposed approach:

Conditional Generation of Drum Patterns To generate a kick drum track, we sample only one mapping code m t (from a 16-dimensional standard Gaussian), repeat it across the time dimension, and reconstruct y given the resulting m , as well as x . Subsequently, we render 20 audio files as described in Section 3.3 and pick those 10 which together constitute the most varied set. Figure 2 shows some results of the generation task - randomly generated kick drum tracks conditioned on the song Drehscheibe (the sound examples are available online). It is clear from these screenshots that the model generates a wide variety of different rhythmic patterns which adapt to the local context, even though the sampled mapping code is constant (repeated) over time.

Style Transfer First, for a given song, we infer m from x and y . Second, k-means clustering is performed over all m t , using the Davis-Bouldin score [19] for determining the optimal number of clusters (typically yielding an optimal k between 5 and 8). Then we use the cluster center of the largest cluster found as the mapping code, again repeat it over time and use it for another song onto which the style should be transferred. Again the results are available on the accompanying web page (see above).

Tempo-invariance We use the WSOLA time stretching algorithm [20] as implemented in sox , to create four time stretched versions of each song, at 80% , 90% , 110% and 120% of the original tempo, respectively. Then, for a given song in original tempo , we determine a prototypical mapping

3 https://sites.google.com/view/drum-generation

Figure 2: Conditionally sampled kick drum tracks for the song Drehscheibe . Each track is the result of a sampled 'transfer function' m t which is held constant over time.

![image](https://ac.marcodsn.me/arxiv-markdown-images/1908.00948v1-figure-fdd8f700-d847-46f9-be22-f1b8862e09ea.jpg)

Figure 3: Generated drum tracks for different tempos ( 80% , 90% 100% 110% , , and 120% of the original tempo) for the song Orgs Waltz , using the mapping code of the original tempo. Top: the overall song; Bottom: close-up of the first onsets, manually aligned for the purpose of visualization.

![image](https://ac.marcodsn.me/arxiv-markdown-images/1908.00948v1-figure-17f35bc7-438d-4f9a-81a7-ebd5032fb1b8.jpg)

code with the k-means clustering method described above. We repeat that code througout the time-stretched versions of the song and reconstruct y given m and x . Figure 3 shows generated kick drum tracks in the five different tempos (four time-stretched versions plus the original tempo). It is clear from these screenshots that the drum pattern adjusts to the different tempos and does not change its style.

Although it is not obvious how to evaluate the output of the model other than by listening, we can check the validity of basic assumptions about the behavior of the model. One assumption is that the ground truth mappings for a song-as defined in Eq. (1)-allow us to reconstruct the drum track relatively faithfully (Eq. (2)). To test the degree to which reconstruction may be sacrificed to satisfy other constraints (e.g., the adversarial loss) we compute the accuracy of the reconstruction. Given onset strength curves y and ˜ y we determine the onset positions as described in Section 3.3, and compute the precision, recall, and F-score using a 50 ms tolerance window, following MIREX onset detection evaluation criteria [21].

Table 1 (upper half) lists the results for the training and validation sets and shows that the mappings are specific

Table 1: Average precision, recall, and F-score for onset reconstruction using ground truth and style transfer mappings.

|                | Precision   | Recall   | F-Score   |
|----------------|-------------|----------|-----------|
| Ground truth   |             |          |           |
| Training       | 0 . 946     | 0 . 812  | 0 . 865   |
| Validation     | 0 . 943     | 0 . 816  | 0 . 867   |
| Style transfer |             |          |           |
| Training       | 0 . 774     | 0 . 696  | 0 . 712   |
| Validation     | 0 . 781     | 0 . 707  | 0 . 723   |

enough to largely reconstruct the target onsets correctly. The reconstructions are not perfect, likely due to the model's invariance and the adversarial loss on the mappings. Note also that the accuracy for the validation set is similar to that for the training set, implying that no overfitting has occurred. The dominance of precision over recall is likely due to the typical 'conservative' behavior of GAEs [22].

Furthermore we test the validity of the heuristic of taking the largest cluster centroid as a constant mapping vector over time for style transfer (assuming time-invariance). To do so, we apply this heuristic to transfer the style of a song to itself. That is, to reconstruct the kick drum track we use the largest mode of the song in the mapping space as a constant through time, rather than the ground truth mapping-a trajectory through the mapping space. Unsurprisingly, this approximation affects the reconstruction of the original kick drum track negatively, but the F-scores of over 0 7 . still shows that a substantial part of the tracks is reconstructed correctly.

## 5. CONCLUSIONS AND FUTURE WORK

We have presented a model for the conditional generation of kick drums tracks given snare and bass tracks in pop/rock/electro music. The model was trained on a dataset of multi-track recordings, using a custom objective function to capture the relationship between onset patterns in the tracks of the same song in mapping codes. We have shown that the mapping codes are largely tempo and time-invariant and that musically plausible kick drum tracks can be generated given a snare and bass track either by sampling a mapping code or through style transfer, by inferring the mapping code from another song.

Importantly, two basic aspects of the chosen approach have been shown to be valid. Firstly, the ground-truth mapping codes are able to faithfully reconstruct the original kick drum track. Secondly, the style transfer heuristic of applying a constant mapping code through time was shown to be largely valid, by comparing the original kick drum track of a song to the result of applying the style of a song to itself.

Although the current work is limited in the sense that the model has only been demonstrated for kick drum track generation, we believe this approach is applicable to other content. We are currently applying the same approach to snare drum generation and f 0 -generation for bass tracks.

## 6. ACKNOWLEDGEMENTS

We thank Cyran Aouameur for his valuable support, as well as Adonis Storr, Tegan Koster, Stefan Weißenberger and Clemens Riedl for their contribution in producing the example tracks.

## 7. REFERENCES

- [1] A. Roberts, J. Engel, C. Raffel, C. Hawthorne, and D. Eck, 'A hierarchical latent vector model for learning long-term structure in music,' in Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018 , 2018, pp. 4361-4370.
- [2] I. Simon, A. Roberts, C. Raffel, J. Engel, C. Hawthorne, and D. Eck, 'Learning a latent space of multitrack measures,' arXiv preprint arXiv:1806.00195 , 2018.
- [3] R. Memisevic, 'Gradient-based learning of higherorder image features,' in IEEE International Conf. on Computer Vision, ICCV 2011, Barcelona, Spain, November 6-13, 2011 , 2011, pp. 1591-1598.
- [4] J. Briot, G. Hadjeres, and F. Pachet, 'Deep learning techniques for music generation - A survey,' CoRR , vol. abs/1709.01620, 2017. [Online]. Available: http: //arxiv.org/abs/1709.01620
- [5] D. Makris, M. Kaliakatsos-Papakostas, I. Karydis, and K. L. Kermanidis, 'Conditional neural sequence learners for generating drums' rhythms,' Neural Computing and Applications , pp. 1-12, 2018.
- [6] D. Makris, M. A. Kaliakatsos-Papakostas, I. Karydis, and K. L. Kermanidis, 'Combining LSTM and feed forward neural networks for conditional rhythm composition,' in EANN , ser. Communications in Computer and Information Science, vol. 744. Springer, 2017, pp. 570-582.
- [7] R. Vogl and P. Knees, 'An intelligent drum machine for electronic dance music production and performance,' in 17th International Conference on New Interfaces for Musical Expression, NIME 2017, Aalborg University, Copenhagen, Denmark, May 15-18, 2017. , C. Erkut, Ed. nime.org, 2017, pp. 251-256. [Online]. Available: http://www.nime.org/proceedings/ 2017/nime2017\_paper0047.pdf
- [8] H.-W. Dong, W.-Y. Hsiao, L.-C. Yang, and Y.-H. Yang, 'Musegan: Multi-track sequential generative adversarial networks for symbolic music generation and accompaniment,' in Thirty-Second AAAI Conference on Artificial Intelligence , 2018.
- [9] G. Hadjeres, F. Pachet, and F. Nielsen, 'DeepBach: A steerable model for bach chorales generation,' in Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, August 6-11, 2017 , ser. Proceedings of Machine Learning Research, D. Precup and Y. W. Teh, Eds., vol. 70. PMLR, 2017, pp. 1362-1371. [Online]. Available: http://proceedings.mlr.press/v70/ hadjeres17a.html
- [10] H. H. Mao, T. Shin, and G. Cottrell, 'Deepj: Stylespecific music generation,' in 2018 IEEE 12th International Conference on Semantic Computing (ICSC) . IEEE, 2018, pp. 377-382.
- [11] G. Hadjeres and F. Nielsen, 'Anticipation-rnn: enforcing unary constraints in sequence generation, with application to interactive music generation,' Neural Computing and Applications , pp. 1-11, 2018.
- [12] S. Lattner, M. Grachten, and G. Widmer, 'Imposing higher-level structure in polyphonic music generation using convolutional restricted Boltzmann machines and constraints,' Journal of Creative Music Systems , vol. 3(1), 2018. [Online]. Available: http://jcms.org.uk/ issues/Vol3Issue1/
- [13] E. Grinstein, N. Q. K. Duong, A. Ozerov, and P. Pérez, 'Audio style transfer,' in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2018, Calgary, AB, Canada, April 15-20, 2018 . IEEE, 2018, pp. 586-590.
- [14] H. Liu and Y. Yang, 'Lead sheet generation and arrangement by conditional generative adversarial network,' in 17th IEEE International Conference on Machine Learning and Applications (ICMLA) , 2018, pp. 722-727.
- [15] S. Lattner, M. Grachten, and G. Widmer, 'A predictive model for music based on learned interval representations,' in Proceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, September 23-27 , 2018. [Online]. Available: http://ismir2018.ircam.fr/doc/pdfs/179\_Paper.pdf
- [16] A. Makhzani, J. Shlens, N. Jaitly, and I. J. Goodfellow, 'Adversarial autoencoders,' CoRR , vol. abs/1511.05644, 2015. [Online]. Available: http: //arxiv.org/abs/1511.05644
- [17] G. Klambauer, T. Unterthiner, A. Mayr, and S. Hochreiter, 'Self-normalizing neural networks,' in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems

- 2017, 4-9 December 2017, Long Beach, CA, USA , I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, Eds., 2017, pp. 972-981. [Online]. Available: http://papers. nips.cc/paper/6698-self-normalizing-neural-networks
- [18] S. Lattner, M. Grachten, and G. Widmer, 'Learning transposition-invariant interval features from symbolic music and audio,' in Proceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, September 23-27 , 2018. [Online]. Available: http://ismir2018. ircam.fr/doc/pdfs/172\_Paper.pdf
- [19] D. L. Davies and D. W. Bouldin, 'A cluster separation measure,' IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 1, no. 2, pp. 224-227, 1979.
- [20] W. Verhelst and M. Roelands, 'An overlap-add technique based on waveform similarity (WSOLA) for high quality time-scale modification of speech,' in IEEE International Conference on Acoustics, Speech, and Signal Processing , vol. 2, April 1993, pp. 554-557.
- [21] 'MIREX Onset Detection Task,' https://www.music-ir. org/mirex/wiki/2018:Audio\_Onset\_Detection, 2018.
- [22] R. Memisevic, 'On multi-view feature learning,' in Proceedings of the 29th International Conference on Machine Learning (ICML-12) , ser. ICML '12, J. Langford and J. Pineau, Eds. New York, NY, USA: Omnipress, July 2012, pp. 161-168.
Stefan Lattner Maarten Grachten
High-Level Control of Drum Track Generation Using Learned Patterns of Rhythmic Interaction
Spurred by the potential of deep learning, computational music generation has
gained renewed academic interest. A crucial issue in music generation is that
of user control, especially in scenarios where the music generation process is
conditioned on existing musical material. Here we propose a model for
conditional kick drum track generation that takes existing musical material as
input, in addition to a low-dimensional code that encodes the desired relation
between the existing material and the new material to be generated. These
relational codes are learned in an unsupervised manner from a music dataset. We
show that codes can be sampled to create a variety of musically plausible kick
drum tracks and that the model can be used to transfer kick drum patterns from
one song to another. Lastly, we demonstrate that the learned codes are largely
invariant to tempo and time-shift.