Add the wine, cloves, cinnamon sticks, allspice berries, and orange slices to a medium saucepan. (You can add the honey now or wait until serving to sweeten to taste.)
Place the saucepan over medium-low heat and cook for 20 minutes. Do not let the wine come to a simmer. You want it to heat up and stay just below bubbling.
Ladle the wine into mugs and garnish with orange slices, cinnamon sticks, cranberries, and/or a rosemary spring before serving.