How many coin flips are needed to reliably know a coin of weight w is unfair? I want to find out how many flips I need to flip a coin to reliably know that it is an unfair coin.
The issue is that as the coin becomes closer to 50/50, the more false-negatives you will have if you don't take dramatically more data.
I wrote some python code and numerically found out how many flips are needed to be able to confirm a weighted coin with weight w. As w gets closer to .5, exponentially more flips are needed to have a low false-negative rate, as shown in this figure I made in python:

The x-axis represents the weight of the coin, while the y-axis represents the number of flips to get a false-negative rate of 5% (while also having a 5% false-positive rate).
Is there a way of getting this answer analytically? I'm particularly interested in what approximations can be done to know what will occur as you get closer to .5.
Here's my python code attempt that was used to generate the figure. I have added comments in the code to try to explain the logic:
from scipy import stats
import numpy as np
import matplotlib.pyplot as plt



# The goal of this script is to figure out how many flips are required to
# have a low false-negative rate when testing if a coin is fair for a specific
# p-value. I use this script to understand the relation between
# the unfairness of the coin and the amount of flips needed to get a low false-negative
# rate. 
# Essentially we calculate the false-negative rate for some starting number of flips,
# and if this rate is lower than our desired threshold, then we add more number of flips,
# we continue this in a loop until we are under the false-negative rate (or until we run 
# out of iterations)
# Finally, we run this all inside the function find_numberofFlips_and_false_negatives_vs_fairness
# this function has an input of the coins fairness, and returns a number of flips as an output
# we then use this function to collect a set of points for the coins fairness vs numberofFlips 
# then we plot this

def flipAndDoPTest(numberOfFlips, weight, guessWeight):
    """
    Simulate multiple coinflips and perform a binomial test, 
    returning a pvalue.
    
    Parameters
    ----------
    numberOfFlips : int
                    The number of flips that will be simulated.
    weight        : int
                    The weight of the coin that is flipped
    guessWeight   : float 
        The hypothesized probability of success, i.e. the expected
        proportion of successes. We will typically use .5 for this.
        
    Returns
    -------
    result :
        pvalue : float
            The p-value of the hypothesis test.
    """
        
    flippedCoinSimulation = np.random.binomial(1, weight, numberOfFlips) 
    numberOfHeads = np.sum(flippedCoinSimulation==1)
    numberOfTails = np.sum(flippedCoinSimulation==0)
    pvalue = stats.binom_test(numberOfHeads, numberOfFlips, guessWeight)
    return pvalue

def accurate_pvalue_interval(numberofFlips, desiredConfidenceLevel):
    """
    Creates an new confidence interval which is closest to the desiredConfidenceLevel.
    Because of the discrete nature of coin flips, it's not possible to get any arbitrary
    confidence level. This code finds the closest confidence level that has a pvalue lower
    than the desired confidence level.
    
    Parameters    
    ----------
    numberofFlips          : int
                             The number of flips that will be done for the 
                             relevant confidence level.
    desiredConfidenceLevel : float
                             the desiredConfidenceLevel that we will find the nearest
                             confidence level for
        
    Returns
    -------
    result :
        pValUnderThreshold : float
                             the nearest valid pvalue that is smaller than the desiredConfidenceLevel
            
    """
    
    numberofIntervals = np.int(np.round(numberofFlips/2));
    intervalIndex = np.arange(numberofIntervals);
    intervalList = 2*(stats.binom.cdf(intervalIndex,numberofFlips,.5));
    pValUnderThreshold = np.max(intervalList[intervalList <= desiredConfidenceLevel])
    return pValUnderThreshold


def false_negative_calculator(numberofFlips, unfairCoinWeight, desiredConfidenceLevel):
    """
    loop through each coin and do a simulated p-test to see if for a set number of coinflips 
    if the given coin is verified to be unfair in a confidence window.
    if the coin is a fair coin, and pvalue<=closestConfidenceLevel (meaning it passes the binomial test and 
    the coin is unfair in a certain confidence window), then we add 1 to the false-positive number
    if the coin is an unfair coin, yet is not identified as one, then we add1 to the false-negative numbers
    """
    guessWeight = .5
    falsePositives = 0
    falseNegatives = 0
    numberofTrials = 1000
    
    
    #for our p-test, we want to make sure to pick an interval that fits the discretness of 
    #the problem. Therefore we input a desired confidence interval, and the function finds 
    #the nearest confidence interval that can be used.
   
    closestConfidenceLevel = accurate_pvalue_interval(numberofFlips, desiredConfidenceLevel)
    
    #see how many false negatives are obtained in a number of trials.
    for ithTrial in range(numberofTrials):
        pvalue = flipAndDoPTest(numberofFlips, unfairCoinWeight, guessWeight)     
        if pvalue>closestConfidenceLevel:
            falseNegatives += 1
    falseNegativeRate = falseNegatives/numberofTrials            
    return falseNegativeRate


def find_numberofFlips_and_false_negatives_vs_fairness(unfairCoinWeight):

    numIterations = 1000
    #maximum number of iterations used in the loop
    numberofFlips = 10
    #initial starting number of flips used in the first iteration of loop
    flipNumberIncrement = 50
    #how much the flip number increases by each iteration in loop
    numberofStandardDeviationSeparation = 4
    #decides the amount of standard deviations the unfair coin is from the fair coin that we finding the 
    #number of flips for
    falseNegativeThreshold = .05
    desiredConfidenceLevel = .05

    #Loop explained in the function deviation_separation_calculator()
    for stepi in range(numIterations):
        falseNegativeRate = false_negative_calculator(numberofFlips, unfairCoinWeight, desiredConfidenceLevel)
        if falseNegativeRate <= falseNegativeThreshold:
            print("Completed in", stepi, "total increments and requires", numberofFlips, "number of flips", "with a false negative rate of:", round(falseNegativeRate*100, 4), "%")
            break
        else:
            numberofFlips += flipNumberIncrement
    if falseNegativeRate > falseNegativeThreshold:
        print("Could not find solution in the given number of iterations:", numIterations)
    return numberofFlips



    
weightList = np.array([.9, .8, .7, .6, .55, .54, .53, .525, .520, .515, .510])
#list of weights of the unfair coin; as this gets closer to .5 the more numberofFlips will be needed for
#our condition to be satisfied

numofFlipsList = np.arange(len(weightList))
for i in range(len(weightList)):
    numofFlipsList[i] = find_numberofFlips_and_false_negatives_vs_fairness(weightList[i])
    print("For fairness weight of:", weightList[i], "number of flips required is",numofFlipsList[i])

plt.plot(weightList, numofFlipsList)   


We can simplify the power calculation by

*

*approximating the sample distributions as normal distributions with $$\sigma \approx \sqrt{pq/n}$$

*approximation $pq \approx 0.5^2$ such that
$$\sigma\approx \frac{0.5}{\sqrt{n}}$$

*approximating the power by considering the entire left tail as non-rejection of the hypothesis (which is not entirely true because a tiny part of the tail is below the lower boundary, but this is very small)


So then we need that the distance $p-0.5$ is equal to $(1.96+1.65)\sigma$. Which leads to
$$p-0.5 = (1.96+1.65)\frac{0.5}{\sqrt{n}}$$
or
$$n = \left(\frac{(1.96+1.65)}{2p-1} \right)^2$$
These values $1.96$ and $1.65$ are computed by the quantile function of the normal distribution and relate to the $2.5\%$ and $5\%$ quantiles.
If we get rid of the second approximation (the deviation is computed for both null and alternative hypotheses with $p=0.5$) then the solution will become
$$n = \left(\frac{0.5 \cdot 1.96 + \sqrt{p(1-p)} \cdot 1.65}{p-0.5} \right)^2$$

Computational comparison
In the graph below we compare the approximation with exact computations

The used code is in R but I imagine it is easy to read and can be easily converted into python.
###
### function to compute power 
### for given sample size n
### and given effect p
###
power = function(n,p) {
  ### hypothesis test boundaries based on binomial distribution quantiles
  lower = qbinom(0.025,n,0.5) ### this gives a lower/left tail of at least 2.5%
  upper = n-(lower+1) ### make a symmetric upper/right tail
  
  ### compute power as probabilities of rejection
  ### two parts either we are below the lower boundary or above the upper boundary
  pbelow = pbinom(lower-1,n,p) # reject when below 'lower' 
  pabove = 1-pbinom(upper,n,p) # reject when above 'upper'
  
  ### return total probability of rejection
  return(pbelow+pabove)
}

### function to get required 'n' 
### such that type 2 error is below 5% (or power above 95%)
get_n = function(p,start_n) {
  
  ### get the value of neccesary 'n' with a loop
  ### we start with start_n and keep increasing n untill the power is above 0.95
  n_test = start_n 
  while (power(n_test,p) < 0.95) {
    n_test = n_test + 1
  }
  
  return(n_test)
}


### plot the theoretic curve
n = 1:30000
plot(0.5+(qnorm(0.975)+qnorm(0.95))*sqrt(n*0.5^2)/n,n, type = "l", xlim = c(0.5,1),
     xlab="p", ylab = "", yaxt = "n")
### y axis tags and label
axis(2, at = 5000*c(0:10), las = 2) ## 
mtext("n", 2, line=4, las = 2)

n_current = 1
### add computed points
for (p in c(0.9, 0.8, 0.7, 0.6, 0.55, 0.54, 0.53, 0.525, 0.520, 0.515, 0.510)) {
  ### compute the neccesary n
  ### and use the old n_current to optimize the loop in the get_n function
  n_current = get_n(p,n_current)
  points(p, n_current)
} 

