Heat the olive oil in a large skillet. Add the anchovies, garlic, oregano, and tomato paste. Stir for about a minute. Add the tomatoes with juice and bring to a boil. Stir in the mussels.
Cover and simmer until the mussels open, about 5 minutes. Discard any mussels that don’t open. Season with ground pepper to taste.
Divide the mussels and sauce over servings of linguine.