As previous post predicted, it is now legal (since May 21st, 2016) for a non-Japanese foreigner (Gaijin-san) to use your non-Japanese phone in Japan for upto 90 days at a time (days counted since your entry date to Japan), as long as your phone has an FCC (with a phone for US market) or a CE (with a phone for EU market) mark, and and a Wi-Fi Alliance mark, even if your phone does not have a Japanese Giteki mark.
Many of SIM companies (MVNOs) are now offering SIMs for non-Japanese travellers to Japan, at the point of entry (international airports) or phone/electronics shops in large cities.
For those who plan to stay in Japan for more than 90 days and have a VISA to stay there, you should register yourself with the municipal office of your residence and obtain an alien card. Your alien card would let you obtain a postpaid or prepaid cellphone contract with any of cellphone carriers in Japan. Your’re, though, not going to able use any phone without a Giteki mark legally for more than 90 days since your last entry to Japan.
You can download the official government brochures here: