WASHINGTON (Reuters)—About four 4 million people in the United States will be fined for failing to buy health insurance when the health overhaul law is fully in force in 2016, the Congressional Budget Office forecast on Thursday.

Most individuals must buy health insurance under the landmark legislation passed by Congress last month, or face fines that will be phased in. By 2016, those without coverage may be fined up to 2.5% of their income.

The majority of the 21 million expected to lack insurance in 2016 will likely not face the penalties, according to the report. Of the estimated 4 million who will, about 9% will be under the poverty line, which is $11,800 in annual income for an individual and $24,000 for a family of four.

The CBO, a nonpartisan arbiter of budget matters, analyzed expected compliance rates with the new law and also forecast the government would collect about $4 billion annually from related penalties between 2017 and 2019.

Under the legislation, a cornerstone of President Barack Obama's domestic agenda, insurance premiums are waived for those with very low income as well as some other groups, while subsidies aim to help poorer people buy coverage.

The new law also requires states to set up exchanges to allow individuals and small groups to shop for insurance, although 20 states have sued the federal government in an attempt to block the restructuring.

To collect penalties, the Internal Revenue Service is considering withholding tax refunds since it cannot seize assets or levy fines, the IRS commissioner said recently.

The legislation aims to expand coverage to about 32 million out of about 46 million uninsured Americans. President Obama signed the legislation after it passed both houses of Congress despite unanimous opposition from Republicans, the minority party.

Republicans have called the overhaul an over-reach of government power, while Democrats have portrayed it as a key reform that will help keep health care costs from skyrocketing out of control.