In a glass measuring jar or bowl, combine the milk with the Lucky Charms cereal. Let soak for 30 minutes, then strain out the solids and discard.
Preheat oven to 350 degrees. Grease 2 6-slot doughnut pans with oil or butter.
In a large bowl, beat together the butter and sugar until combined, about 2-3 minutes. Beat in the eggs, one at a time, then the vanilla and Greek yogurt. Scrape down the sides of the bowl. Add the flour, baking powder, baking soda, and salt. With the mixer running, stream in 1/4 cup of the cereal milk until just combined. Do not overmix. The batter will be very thick.
Scoop the batter into a piping bag or a large ziptop bag with a corner snipped off. Pipe the batter into the doughnut tins, filling 2/3 of the way full. Bake for 10-12 minutes until an inserted toothpick emerges with just a few moist crumbs. Cool 5 minutes in the pans, then gently run a knife around each doughnut to loosen, and move to a wire rack to cool completely.
To make the icing, whisk together the powdered sugar, vanilla, and 2 tablespoons of the cereal milk. If needed, add milk 1 tablespoon at a time to reach a thick but smooth consistency.
Dunk one side of each doughnut into the icing, letting excess drip off. Immediately press a few Lucky Charms marshmallows and sprinkle some nonpareils. Let sit 5 minutes to set the frosting, then enjoy!