Preheat the oven to 350 degrees F (175 degrees C). Lightly grease a 9x9x2-inch baking pan. Place a pan of water into the bottom of the oven to help keep the pudding moist. Combine bread crumbs and walnuts in a bowl and pour into the bottom of the prepared baking pan. Beat condensed milk, pumpkin puree, eggs, nutmeg, cinnamon, and ginger in a mixing bowl. Stir in milk, melted butter, rum, 1 teaspoon vanilla extract, and rum extract. Pour mixture over bread crumbs. Bake in the preheated oven until a knife inserted in the center comes out clean, about 1 hour. Meanwhile, beat cream, sugar, and 1/2 teaspoon vanilla extract together in a chilled bowl until stiff and peaked. Fold in amaretto and refrigerate whipped cream until ready to serve with the bread pudding.