Pour olive oil into the bottom of a medium pot. Add garlic and set over low heat. Warm the oil though, taking care not to fry the garlic, for about 3 minutes. Add the orange peel, rosemary, and red pepper flakes and continue to warm until hot, about 3 more minutes. Remove from heat and mix in orange juice. Then add olives, mixing to coat and submerge in the oil. Let sit until cool. Serve immediately or pour mixture (rosemary sprigs, orange peel, and all) into sealable containers (I used mason jars with lids because #brooklyn) and store in the fridge. Will keep for a little over a week.