In a small bowl, whisk together egg, sugar, milk, orange juice, and orange zest. Pour onto a rimmed plate.
Soak each slice of bread in the milk mixture for 30 seconds per side.
In a large skillet, melt 1 tablespoon butter over medium low heat. Once melted, fry the soaked slices until golden, about 2 minutes per side. Melt other tablespoon butter after frying 2 slices.
Serve warm, sprinkled with sugar or drizzled with syrup.