He said: "Perhaps I am speaking too boldly, but I learned to speak boldly a long time ago.

I will speak my sentiments in the face of any organization; or, if it does not show its face, though its secret mines are beneath my feet, and unseen hands ready to apply the match, I will declare those sentiments that a freeman is bound to utter." The people of Massachusetts elected Gardner Governor in 1854, 1855 and 1856.

But in the autumn of 1857 he was beaten under the leadership of General Banks.

The party lingered until 1856 when there was an attempt to keep it alive in the Presidential campaign of that year when Millard Fillmore was its candidate for the Presidency.
But it was destroyed in the consuming fire kindled by the Civil War, and has not since been heard of by its old name.
The proscriptive and intolerant opposition to Catholicism, especially against men of foreign birth, has shown its head occasionally.

It appeared in its most formidable shape in a secret organization known as the A.P.