It's closer to a formula than it is to trial and error. There are mathematical methods to estimating square roots, and calculators do not need infinite precision. They iterate over a formula that gets more precise the more you apply it, looking for the desired precision, and then they return the result.