In most AB test analysis we did and the AB test report generator, we uses bayesian credible interval to evaluate whether the differences between groups are significantly different. However we used power analysis, which is a method of frequentist statistics, to calculate the sample size needed for AB test, as most practitioner did in this industry.

@mpopov and I (@chelsyx) discussed about this a while ago and found some literatures about this topic. We need to read these articles and:

• Find whether there is feasible way to calculate sample size under Bayesian framework
• If not, could we use power analysis to calculate sample size, then use bayesian credible interval to make decision? What could be the problem?