First, mix the marinade. In a large bowl, add all of the olives (drained), olive oil, orange zest, lemon zest, lemon juice, thyme sprigs, fennel seeds, Aleppo pepper, salt, and pepper. Stir gently until everything is evenly combined.
Transfer the olive mixture to a medium saucepan. Place over low heat and warm gently for 8–10 minutes, stirring occasionally, until the olives are heated through and the oil is fragrant. Do not let the oil simmer or boil.
(Optional) If making ahead, allow the mixture to cool completely, then transfer to a deli container or airtight container. Cover and store in the refrigerator for up to 3 days.
When ready to serve, gently rewarm the olives over low heat until just warmed through. Transfer to a serving bowl, spoon some of the oil over the top, and serve with crusty bread.