Source: http://news.bbc.co.uk/2/hi/americas/4363175.stm
The cabinet chief said Bishop Antonio Baseotto's comments recalled the former military government, which dropped opponents from planes into the sea.
Technically only the Vatican can sack the prelate, who is bishop to Argentina's armed forces.
Abortion is illegal in the country and can be punishable by prison.
'Millstone'
Health Minister Gines Gonzalez Garcia recently said he was in favour of legalising abortion.
Bishop Baseotto replied that the minister's stance made him an apologist for murder.
Quoting a passage from the New Testament, he said Mr Gonzalez Garcia deserved to have a millstone hung around his neck and should be "thrown into the sea".
But the government has said the statement evoked Argentina's controversial past.
During the military regime that ruled the country from 1976-1983, political opponents were thrown into the sea during the so-called "death flights".
The bishop says his comments were taken out of context, but BBC Buenos Aires correspondent Elliott Gotkine says his position now appears untenable.
Bishop Baseotto holds his post because of a 2002 agreement between Argentina and the Vatican.
But a new presidential decree has annulled the earlier agreement.
Cabinet chief Alberto Fernandez said the government would formally withdraw its support for the bishop and stop his salary.
The move follows Rome's decision to ratify the bishop in his position.