Determine whether two binary sequences are correlated I have two arrays of 2048 binary values (1 and 0) and I'd like to find out if their content can be considered correlated. I'd like to have less than 1:1000000 chance of a false positive in determining the correlation.
I'm looking for a percentage number as well as a formula.
We can assume the binary values to have been generated by uniform random variables that may or may not be independent. The task is to find out whether they are independent or correlated. I assume with 2048 values, they can be considered uncorrelated if around 50% values are different. But they will never be exactly 50% different because they are random values. They may be 49% different and we may still be quite confident that there is no correlation.
Note: The only value I have as input for the algorithm is the number of positions that are different (denoted popcount(A xor B) in the example).
Let X=popcount(A xor Y)/2048 with A and B independent uniform random variables. I think X would form a Gauss distribution around 0.5 and its standard deviation would probably be the main ingredient of the answer, I think.
Example 1:
Pattern A          100111010101010111111111
Pattern B          100101010001000111111101
A xor B            000010000100010000000010
popcount(A xor B)  4
In percentage      4/24=16.7%
Interpretation     Correlated (only 4 positions differ) 

Example 2:
Pattern A          100111010101010111111111
Pattern B          001100110001000100101101
A xor B            101011100100010011010010
popcount(A xor B)  11
In percentage      11/24=45.8%
Interpretation     Not Correlated (nearly 50% different)


You can compute correlation between vectors/matrices as their dot product scaled by their magnitude (norm). Geometrically, this represents the cosine of the angle formed between them.
Ie:
$\text{corr}(\mathbf{X},\mathbf{Y}) = 
\frac{\left< \mathbf{X}, \mathbf{Y} \right>_F}{\| \mathbf{X} \|_F \| \mathbf{Y} \|_F }$
where the inner product between matrices (from which the norm follows) is the Frobenius norm, that is the $\ell_2$ norm over the matrice viewed as a vector : $\text{vec}(\mathbf{X})$.
