Hilary Duff still thought Joel Madden was a good guy after their 2006 breakup—and now she counts his wife Nicole Richie among her dear friends, too.