Although the following excerpt from theologian Albert Barnes is more judicial than theological in nature, the difference between public justice and private revenge is nevertheless important for certain denominations in the Church, like Moral Government Theology.
“The design of punishment is not revenge or vengeance; for it is not to gratify private feelings or to redress private wrong, – which is the true notion of revenge or vengeance. It is not the infliction of pain for an offence committed against an individual. It is always, though it may be for a wrong done to an individual, inflicted for the offence regarded as perpetrated against the peace of a community, against the lawgiver, against the law itself. When a man is punished for assault and battery, it is not pain inflicted considered as a recompense to the individual who has been injured or wronged: it is a just retribution for a crime against the peace of society and the honour of the law, and the punishment is measured by that consideration alone. When a man is punished for murder, it is not as an act of recompense to the murdered man, – for he is beyond the reach of all such recompense, – but it is for an offence against the law and the peace of the community. The murdered man is in no manner referred to in the case except as one over whom the law was designed to throw its protection; and the purpose is to maintain the honour of that law and to prevent its violation. In the infancy of society, in the days of savage barbarity, when there were no tribunals of justice, a relative of the murdered man – an avenger of blood – might take the matter into his own hands and inflict summary justice on the murderer, and that would be properly revenge; but the arrangements of a civilized community are designed to take the case out of the hands of the individual. The crime is punished, not as a matter of private vengeance or satisfaction, but as due to public justice. The individual who has suffered wrong is not even represented in the transaction. The law only is represented; and the affair is no longer one of a private character, but becomes one of pertaining wholly to the public.”
Source: Albert Barnes, The Atonement (1860), Chapter 6: Necessity of an Atonement, p. 191-192.