Do not ignore it. At every stage, that is what works best. One phone call to the department named on the letter changes the story from "an owner we cannot reach" to "an owner who is dealing with it".
What troubles the city official is less the damaged house itself than being unable to reach the owner. A letter finally sent after tracing the family registers gets no reply; if that continues, the city has no choice but to move to the next stage. If there is a reply and a plan, the official records it and can tell the neighbours the owner is dealing with it. Even from outside the prefecture there is no need to visit; a phone call is enough.
There are four things to say: ① that you are the owner (or an heir); ② whether you know the house's current state, and if not, when you will check; ③ what you intend to do and by when; ④ your contact details. Even without a firm plan, "we are considering whether to sell or demolish and will decide by next month" is enough.
Sample reply (phone)"I am calling about the vacant-house notice dated [date]. I am [name], an heir of the owner [name]. I live outside the prefecture and have not seen the house for some time. I plan to go back and check it during [month], or have a contractor look at it. I will decide what to do, including whether to sell or demolish, and contact you by the end of [month]. You can reach me on this number on weekdays after [time]."
Sample reply (letter or email)"I have received your letter dated [date], 'Request regarding the proper management of a vacant house'. As one of the heirs to the property, I will check its current state and inform you of our course of action (repair, removal or sale) by the end of [month]. If there is any immediate danger (loose tiles, encroachment onto the road), I will arrange a contractor first. Contact: name, address, telephone."
When the letter names something specific, such as tiles about to fall onto the road or a garden tree touching the power lines, the right order is to deal with that one point first, before deciding the overall plan. We can arrange removal of loose parts or cutting back of branches, and if you wish we will contact the city's department on your behalf. This need not be on the basis of a sale. Simply letting the city know that a contractor is scheduled removes its reason to move to the next stage.
Note that the city may enter the lot to inspect it to the extent necessary, and refusing without good reason carries a fine of up to ¥200,000. There is no reason to refuse. If you know the date, we can attend in your place.