A schism among the Republican supporters of the war was with difficulty averted in 1864, when Fremont threatened to lead the radicals in opposition to the "Union" party of the President and his conservative policy.
The breach was widened by the refusal of Congress to admit representatives from Arkansas and Louisiana in 1864 and to count the electoral vote of Louisiana and Tennessee in 1865.

The passage of the Wade-Davis reconstruction bill in July 1864, and the protests of its authors after Lincoln's pocket veto called attention to the growing opposition.

Severe criticism caused Lincoln to withdraw the propositions which he had made in April 1865, with regard to the restoration of Virginia.

In his last public speech, he referred with regret to the growing spirit of vindictiveness toward the South.

Much of the opposition to Lincoln's Southern policy was based not on radicalism, that is, not on any desire for a revolutionary change in the South, but upon a belief that Congress and not the executive should be entrusted with the work of reorganizing the Union.