I don’t know anything about the system in the US, but I know that here in Canada they won’t take you to court instantly if you don’t fill in the census (short or long, similar to the US). Instead they’ll send you a few reminder letters first, and if that doesn’t work they’ll try to send a census working to your home to ask you the questions you missed. AFAIK, this is done to try to prevent a situation where you’re taking to court someone who perhaps can’t read (due to vision or literacy or language problems), or who has other trouble filling out the forms.

So long as you cooperate with the census worker, you won’t see the inside of a courtroom. AFAIK they only take people to court who don’t cooperate with the census taker.