Reformed theologian Abraham Kuyper (1837-1920):

“The only question that is decisive is whether it achieves its goal. Aside from this, everyone is obligated, to apply it with all caution if it is effective, and to reject it if it is not effective.”

He leaves it up to parents:

“Research must continue to determine whether the harmful effects of the vaccination can be prevented with better treatment or more cautious application… On this point each one must decide for himself, and the father for his child, so that no one may coerce us against our will.”

He vehemently rejected vaccine mandates:

“For this reason alone, compulsory cowpox vaccination should be out of the question. Our physicians may be mistaken and government may never stamp a particular medical opinion as orthodox and therefore binding. Moreover, compulsion can never be justified until the illness manifests itself and may therefore never be prescribed as a preventative. A third reason is that government should keep its hands off our bodies. Fourthly, government must respect conscientious objections. In the fifth place, it is one or the other: either it does not itself believe in vaccination, or if it does, it will do redundant work by proceeding to protect once more those already safeguarded against an evil that will no longer have a hold on them anyway. Vaccination certificates will therefore have to go—and will be gone at least from our free schools. The form of tyranny hidden in these vaccination certificates is just as real a threat to the nation’s spiritual resources as a smallpox epidemic itself.” (“Our Program: A Christian Political Manifesto”)

But he says that it would be foolish and immoral to abstain from the vaccine if the evidence really does favor them:

“In the end everything comes down to the question of whether inoculation, or whatever other means, shows empirically that it reduces susceptibility to cowpox and also does not bring other serious damage to the body. This must be tested, researched, and determined experimentally. If the result of this research is unfavorable, it automatically falls by the wayside. But if the results of the research are favorable, then not only does nothing stand in the way of its application from the standpoint of faith, but it would be foolhardy—even immoral—not to apply a means that God has shown us for the protection of the life of our child. We are not advocating coercion on the part of the government.”

But again, he “[wishes] to lay it before the conscience of parents”, and encourages them to be informed:

“It stands to reason that God the Lord would have informed us directly about the nature of this terrible illness and also about the effective means to heal or prevent it.”

For perspective, 300 million people died of smallpox, and its fatality rate was quite different than COVID-19:

“During the 18th century the disease killed an estimated 400,000 Europeans each year, including five reigning monarchs, and was responsible for a third of all blindness. Between 20 and 60% of all those infected—and over 80% of infected children—died from the disease.” (Wikipedia)

Source of most quotes: Abraham Kuyper, “Common Grace“, vol 2. Series written 1902–1905. Contains two chapters on the ethics of vaccines. He favors vaccines in general and addresses ethical concerns that persist today.