Combine the first five ingredients in a bowl; cut in butter until mixture resembles coarse crumbs. Add milk, stirring only until moistened. Turn onto a lightly floured surface; knead 10 times.
Pat evenly into a greased 8-in. round baking pan. Bake at 425° for 20-25 minutes or until golden brown. Remove from pan to cool on a wire rack.
Just before serving, combine peaches, 4 tablespoons brown sugar and ginger. Whip cream with remaining brown sugar until stiff.
Split shortcake into two layers; place bottom layer on a serving platter. Spoon half of the peach mixture over cake; top with half of cream. Cover with second cake layer and remaining peach mixture. Garnish with remaining cream; sprinkle with pecans. Yield: 8-10 servings.