Preheat the oven to 450°F (230°C). Line a baking sheet with parchment paper.
Combine the herbs, garlic, chili pepper, capers, and anchovy paste in a small mixing bowl. Stir with a fork, smashing the anchovy paste with the back of the fork to make sure it gets incorporated. Cut the lemons in half. Slice one half and reserve. Juice the other 1½ and add the juice to the herb mixture. Whisk in ¾ cup olive oil. Taste and add salt and pepper, as needed. (This salsa verde can be made ahead and stored, covered, in the refrigerator for up to 1 week.)
Rinse the fish, inside and out, and pat dry. Cut slits into the skin with a sharp knife, about 1 inch apart. Season the fish, inside and out, with salt and pepper. Rub the outside of the fish with olive oil. Brush some of the salsa verde in the cavity of the fish. Place the lemon slices in the cavity and roast the fish for 20-25 minutes or until it just starts to flake easily.
Drizzle some of the salsa verde over the fish and serve the rest on the side. Serve with lemon wedges, if desired.