Preheat the oven to 325°F. Spray a 9x13-inch pan with baking spray.
In a large bowl, stir together the flour, sugar, baking powder, and salt. Add the eggs, butter, sour cream, and milk. Stir until most of the large clumps disappear. The batter will be thick but spreadable.
Pour half of the batter into the prepare pan and smooth it out.
Stir together the brown sugar and cinnamon sugar in a bowl. Sprinkle evenly over the top of the cake batter. Cover with the remaining cake batter, spreading evenly.
Bake for 40 to 45 minutes, until golden and a toothpick inserted into the center comes out clean. Allow the cake to cook for 10 minutes.
Whisk together the powdered sugar, milk and vanilla. Pour over the warm cake and evenly smooth it out. Let it cool before cutting.