Combine wine, lemon, orange, sugar, and rum in a large pitcher then use a wooden spoon to muddle the fruit. Refrigerate for at least several hours or overnight. When ready to serve, add club soda (or ginger ale, if using) to the pitcher then serve in glasses filled with ice and fresh fruit add ins. Note: If using ginger ale instead of club soda, cut back on sugar.