Stir the cookie ingredients together until smooth. Scoop 1-inch balls onto a parchment-lined baking sheet.
For the classic shape, form each dough ball into a short log. Roll in sugar if desired, flatten with a fork in a criss-cross pattern, and pinch both sides inward.
Bake 12 minutes, then cool on the tray 1 to 2 minutes before transferring.
For the filling, beat butter and peanut butter until smooth. Slowly beat in powdered sugar, then add milk 1 tablespoon at a time until the desired consistency is reached.
Spread about 1 tablespoon filling on the bottom of one cookie and top with another to make a sandwich.