Crush Oreos into fine crumbs and set aside.
Whisk pudding mix with cold milk 2 minutes. Let sit 5 minutes to thicken.
Beat softened cream cheese and butter until smooth, then add powdered sugar and beat until fluffy.
Fold thickened pudding into cream cheese mixture until combined.
Fold in whipped topping gently in two additions to keep it light.
Layer half the Oreo crumbs in dish, spread filling over top, then cover with remaining crumbs.
Refrigerate at least 2 hours, preferably overnight. Add gummy worms before serving.