J. Q. Adams went on to become a Representative for MA in the House and I’d argue was the only real demotion post-presidency.
W. Taft went to the Supreme Court after his Presidency (not exactly a lateral move in and of itself) but was the Chief Justice so it’s probably close. Then again institutions powers wax and wane over time, so it could be argued that it was a demotion.
Lastly, kind of a bonus factoid of post-presidency activity – Teddy Roosevelt, after serving a term plus some (edit: he took over the presidency as the VP after McKinley’s assassination) and was a Republican. He later ran in 1912 as a candidate for the Bull-Moose Party after a rift in the Republican party when they nominated the above W. Taft instead of himself or another more Progressive Republican.
Ask your representative(s) to support (more) funding for the IRS.