Using an electric mixer on medium speed, beat granulated sugar and butter in a large bowl, scraping down sides as needed, until light and creamy, about 5 minutes. Add vanilla and yolks in 2 additions, scraping down bowl after each. Reduce mixer speed to low and add dry ingredients, alternating with milk mixture in 2 additions, beginning and ending with dry ingredients and beat just until smooth.