Yamagiwa Daishiro, Japan's minister in charge of economic revitalization, is stepping down amid mounting criticism over his relations with the controversial Unification Church.