Since Ronald Reagan was elected president in 1980, the Republican party has run on the idea of shrinking the federal government to "get it out of people's lives." While this like most campaign promises made by either party is easy to say, it's a more difficult task to actually reduce government size.
Regulations can be cut by legislature and rules can be created to reign in agencies' influence but the bureaucracy is a hard beast to cut down. The executive branch has only grown since the constitution was signed, both in size and in power. Initiatives like Teddy Roosevelt's creation of conservation agencies, Franklin Roosevelt's New Deal and Lyndon Johnson's Great Society programs greatly expanded federal jurisdiction where those presidents saw the need for government intervention. From these programs came the government protection of certain lands, revitalization of a depressed economy, and welfare services for the nation's neediest.
When these agencies are created they answer to the president. Soon after though, they begin to take on a life of their own, growing ties to different interest groups, industry leaders, and representatives in congress. Like a living organism, federal agencies prioritize their own survival very highly. This is where some government inefficiency comes from. These agencies do not want to have any funds left over as they don't want their next amount of funding slashed by the difference. So for the most part, they take their time performing their duties.
This may seem blasphemous from a logical standpoint but it is argued that this was an intention of the founding fathers. A slow, deliberate government is what they wanted. The bureaucracy's pace is mirrored in the total time of the legislative process and the amount of time SCOTUS takes to get through a large constitutional case (as well as the fact that SCOTUS is an entirely reaction based institution and lacks proactive ability by design).
The difference between these three cases that makes the bureaucracy an easy target is its gigantic budget. And with a Republican president and Republican controlled congress, it's likely we'll see some small cuts to big agencies and total cuts to smaller agencies. President Donald Trump has also taken some alternative measures in attempting to shrink the federal government, as almost 500 presidentially appointed positions out of 1,200 remain nominee-less. Civil servants fill most of these smaller roles, but some departments are greatly handicapped by a lack of important staff. We have also seen White House advisor Steve Bannon running around talking about "deoperationalizing" different government institutions.
As tax reforms and budgetary delegation continue, it may be a good show watching what agencies they attempt to shrink. Maybe the GOP will manage to stop the hydra's heads from growing back.