Basic Sampling - Provide confidence of estimate I tried searching the site but nothing came up.
I have a simple situation for a business problem - We have a population of 50 million files. We want to review a sample and see whether a file contains or does not contain "x". So it's binary. We will calculate a proportion of yes:no.
We want to be able to quantify how accurate our estimate is of the true population. "We are 95.. 99% confident that the true proportion is "x/y" +/- x%. From my research, confidence intervals cannot be interpreted in such a way. Rather, confidence intervals are supposed to be interpreted as "in the long-run, if we conduct many experiments, we are 95% confident that the CIs calculated for those samples will contain the true population parameter". Do we just take an arbitrary "large enough" random sample, calculate the proportion and assume that is the true proportion? I came across the binomial credible interval that can be interpreted as the former, but I know nothing of Bayesian inference unfortunately.

This type of problem is well-suited to a confidence/credible interval.  It is unclear to me why you think a confidence interval is unsuitable.  You ought to be able to get a reasonable inference for the true proportion using a Wilson score interval.  You can compute this confidence interval (for a finite or infinite population) using the CONF.prop function in the stat.extend package in R.  Here is an example where I use some randomly generated mock data to get the interval:
#Set significance level and population size
#In this case the population size is large so it could be ignored if desired
N     <- 50000000
ALPHA <- 0.05

#Generate some (mock) data
p <- 0.03
n <- 3000
set.seed(1)
DATA <- sample(c(0,1), size = n, replace = TRUE, prob = c(1-p,p))

#Compute the confidence interval
stat.extend::CONF.prop(alpha = ALPHA, x = DATA, N = N)

        Confidence Interval (CI) 
 
95.00% CI for proportion for population of size 5e+07 
Interval uses 3000 binary data points from data DATA with sample proportion = 0.0303 

[0.0247716637647717, 0.0370961961781321]

