Preheat oven to 350 degrees F. Spray a 9x5-inch loaf pan with non-stick spray or line with foil and spray with non-stick spray to get out easily for slicing.Mix brown sugar and cinnamon together in a bowl. Set aside.In another large bowl, beat white sugar and butter together using an electric mixer until smooth and creamy.Beat in eggs, one at a time, until blended in; add vanilla extract.Combine flour and baking powder together, with a whisk, in another bowl and slowly add into butter mixture. Stir until blended.Mix milk into batter until smooth.Pour half the batter into the prepared loaf pan; add half the apple mixture, then half the brown sugar/cinnamon mixture.Lightly pat apple mixture into batter.Pour the remaining batter over apple layer and top with remaining apple mixture, then the remaining brown sugar/cinnamon mixture.Lightly pat apples into batter; swirl brown sugar mixture through apples using knife or spoon.Bake 50 to 60 minutes or until a toothpick inserted in the center of loaf comes out clean.To make glaze, mix powdered sugar and milk together until combined.Let cool for about 15 minutes before drizzling with glaze.