Grease a 9 x 13 inch baking dish with butter.
In a large bowl, whisk together the brown sugar, maple syrup, pumpkin puree, milk, cinnamon, ginger, cloves, allspice, vanilla, bourbon, and eggs, until well combined.
Dip each slice of challah or brioche into the custard and arrange in the baking dish. Repeat until all of the bread is used. Pour the remaining custard over the bread slices. Cover and refrigerate at least an hour, or overnight.
When ready to bake, preheat oven to 375 degrees.
In a small bowl, toss the brown sugar, white sugar, cinnamon, ginger, flour, and butter until combined. Sprinkle over the toast slices.
Cover the dish with aluminum foil and bake for 25 minutes, then uncover and bake 20 minutes longer until the tops of the bread slices are crisp and browned and the crunch topping bubbles around the bread slices.
Divide French toast among plates. Serve warm, garnished with whipped cream and maple syrup, as desired.