Combine pudding mix, sugar, and 1/4 cup of the milk in a 2-quart saucepan. Add slighly beaten egg yolks and blend well. Then add remaining milk; mix well. Cook over medium heat, stirring constantly, until mixture comes to a full boil, about 10 minutes. Cool. Beat egg whites until stiff peaks form; fold into cooled pudding mixture. Add vanilla and brandy; then chill several hours. Just before serving, whip the cream. Spoon onto eggnog and sprinkle with nutmeg.