Peel and quarter the peaches. In a skillet over low heat, melt butter and stir in sugar, letting it dissolve. Add peach slices to the butter and sugar mixture and cook over low heat until sugar darkens and peaches are soft - about 30 minutes.
Meanwhile, in small bowl mix together flour, salt and baking powder. Set aside.
In mixer, cream butter and sugar. Add ginger and vanilla. Add eggs one at a time. Stir in sour cream. Gradually add flour mixture until just combined. Stir in half of one peach, finely chopped.
When peaches are soft, remove skillet from heat. Spoon cake batter over the peaches, leaving a slight border on edges. Bake for 30 minutes or until cake tests done. Cool 10 minutes, then invert onto a plate. Serve warm with ice cream.