Proof of uniform sampling of binary matrices with fixed row sums and column sums for the fast Curveball algorithm

Randomization of binary matrices has become one of the most important quantitative tools in modern computational biology. The equivalent problem of generating random directed networks with fixed degree sequences has also attracted a lot of attention. However, it is very challenging to generate truly unbiased random matrices with fixed row and column sums. Strona et al. [Nat. Commun. 5, 4114 (2014)] introduce the innovative Curveball algorithm and give numerical support for the proposition that it generates truly random matrices. In this paper, we present a rigorous proof of convergence to the uniform distribution. Furthermore, we show the Curveball algorithm must include certain failed trades to ensure uniform sampling.

Figure 1

A trade in the Curveball algorithm consists of (a) converting a binary matrix into lists of indices $A_i$ for each row $i$. In step (b) two rows are selected, in this case rows 1 and 3. (c) The set differences $A_{1-3}$ and $A_{3-1}$ are extracted. (d) Set $B_1$ is formed by removing $A_{1-3}$ from $A_1$ and adding $|A_{1-3}|$ elements randomly chosen from $A_{1-3}\cup A_{3-1}$, in this case $\{2,3,4\}.B_3$ is formed by removing $A_{3-1}$ from $A_3$ and adding the remaining elements of $A_{1-3}\cup A_{3-1}$. Step (f) converts the resulting lists of indices $B_i$ into matrix $B$. In the original description of the Curveball algorithm step (b)-(d) are repeated $N$ times in step (e). This is equivalent to repeating step (a)-(d) and step (f) $N$ times.

Figure 2

A switch in the switching algorithm.

Figure 3

(a) There are six different matrices with row sums (2,2) and column sums (1,1,1,1). (b) The structure of the state graph for the switching method. Notice that there are no edges between states $A$ and $F$, between states $B$ and $E$, and between states $C$ and $D$ since these pairs of matrices differ by two switches. (c) The structure of the state graph of the Curveball algorithm. There are edges between states $A$ and $F$, between states $B$ and $E$, and between states $C$ and $D$ since these pairs of matrices differ by one trade (of size two).

Figure 4

When a matrix $A$ contains two rows $i$ and $j$ that can make a trade and differ at more than two indices, it contains a submatrix of one of the forms above. This implies that there is a path of three trades as well as a path of two trades starting and ending at $A$. Thus state $A$ is aperiodic, and hence the finite Markov chain is aperiodic.

Figure 5

(a) When all repeated states are excluded from the Curveball algorithm, $P_{AB}$ is not always equal to $P_{BA}$. In this example, both $A_{1-2}$ and $A_{2-1}$ contain two elements, that is $|A_{1-2}|=|A_{2-1}|=2$. There are two row pairs in $A$ that can make trades, resulting in $P_{AB}=\frac{1}{10}$. This does not equal $P_{BA}$ since there are three row pairs in $B$ that can make trades and hence $P_{BA}=\frac{1}{15}$. (b) The number of each of the 15 possible binary matrices with row sums (2,2,2) and column sums (2,1,2,1) in a biased sample, generated by the modified Curveball algorithm where all repeated states are excluded. The sample consists of 10 000 matrices sampled at every 1000th trade of the Markov chain.

Figure 6

The transition probabilities of the Curveball algorithm without repeated states for all 15 matrices with row sums (2,2,2) and column sums (2,1,2,1).

Figure 7

Due to the large variance in perturbation scores for different runs of each Markov chain, average perturbation scores over 100 runs are shown. (a) Five random $10\times 10$ binary matrices were created by letting each matrix entry be one with probability 0.1, 0.2, 0.3, 0.4, or 0.5. For each matrix, the perturbation scores of the Curveball and good-shuffle Curveball algorithm stabilize at roughly the same point after about 50 steps. (b) The same experiment was repeated for $100\times 100$ matrices, and here the perturbation scores of the Curveball and good-shuffle Curveball algorithm are indistinguishable.