Families of Japanese abducted by North Korea say Japan must follow through on the Trump-Kim summit by speaking directly with Pyongyang.