Often the advice is "don't do it - it's far too complicated."
In actual fact, although you do need to be organised (several months in advance), the process itself isn't especially difficult.
Nor is it as expensive as a lot of people make out (if done correctly). If you mess it up, or come ill-equipped, be prepared to spend thousands of your respective currency.
All the information you need for importing or exporting any animal - from a dog to a skunk, from honey bees to beef cattle - can be found at the Japanese Animal Quarantine Service website. The site's in English, and has all the necessary forms to download and requirements listed.
For the basic cat or dog case, you'll need appropriate vaccinations (and a certificate of the lack of rabies antibodies, issued several months after the test) and a micro-chip for starters. You'll also need to submit your intent to travel with your pet 40 days prior. Unless your animals needs to be detained, the cost will be fairly small (quarantine inspection is free, but you may have to pay for any veterinary health checks or immunizations required), and the process should be completed within 12 hours (on the way in, much less time outward-bound). See the AQS website for full details.
Be sure to check which airlines allow pets and what their terms are before buying your ticket. If you have a small dog you may well be able to take it in the cabin rather than let it freeze to death elsewhere.
NB: Pet monkeys aren't allowed into Japan. Only for the lab or zoo.
The rabbit procedure apparently goes like this:
Also be prepared, if you're coming here to live, that finding pet-OK accommodation can be pretty tough.
Successes/failures in the comments, please!