That's odd, how far out in the sticks were you? I've used my european chip card extensively in Japan for the past 3 years with no confusion whatsoever, and most places that take cards have a chip reader that they use immediately. Nobody has ever said my card doesn't work. There's actually way more confusion when I pull out a WAON card or something.
It's been more of a problem in places in Thailand and Singapore where they're not really equipped to hand over the card terminal and sometimes I have to go behind the desk or something.
The first place I had a problem was actually a Honda (I think?) car rental office in Shinjuku, not a place where I expected a problem (and not the easiest thing to pay in cash, though that's what we eventually did).
Then a few hotels and ryokans mostly around Nagano and Toyama. The 7/11 ATMs worked fine though (but the card didn't work to pay directly, it did at Lawson though, IIRC) so we just used cash everywhere, but I liked to keep trying to use the card, as I was really not sure what was happening.
It's in a small hotel in Obama that I finally understood what was happening, after the woman first asked me to "write my secret code" on a piece of paper (she didn't know the keypad was attached to a long cord and that she was supposed to let me enter the PIN myself, her daughter figured - or knew - that).
I don't really remember but I think we just didn't have many issues afterwards, once in the larger cities of the Kansai.
It was kind of a pain to have to use cash so much because my card had a rather low limit on cash withdrawal at foreign ATMs (of course, my other card had been mailed by mistake by my bank to my former place just before I left). I just didn't expect at all that it would be so difficult in Japan.
It's been more of a problem in places in Thailand and Singapore where they're not really equipped to hand over the card terminal and sometimes I have to go behind the desk or something.