U.K. Prime Minister Boris Johnson faced yet another challenge from within his own Conservative Party when Brexit Minister David Frost resigned effective immediately from Johnson’s cabinet Saturday, BBC reports.

Frost, who holds a life peerage that entitles him to sit in the House of Lords, had planned to resign next month, but moved his timetable up to protest Johnson’s new COVID restrictions, including a requirement to show proof of vaccination or a negative test to enter many public venues, according to The Associated Press.

Instead of implementing vaccine passports, Lord Frost wrote in his resignation letter, the U.K. should “learn to live with COVID.”

Frost praised Johnson for his handling of Brexit and for making the “brave decision in July, against considerable opposition, to open up the country again.” He also urged the prime minister not to be “tempted by the kind of coercive measures we have seen elsewhere.”

Earlier this week, 99 Conservative members of Parliament voted against the passports in the largest defection of Johnson’s premiership. Johnson also suffered defeat Thursday when the Liberal Democrats won a by-election in North Shropshire, long considered a safe seat for the Conservatives.

Johnson became prime minister after the resignation of Theresa May in May 2019 and led his party to a massive electoral victory in December of the same year. Now, though, the opposition senses weakness. In response to Frost’s resignation, Labour Deputy Leader Angela Rayner opined that the government seems to be in “total chaos.”