It is reasonably certain that Romans was written to the church in Rome in the context of Jews “re-entering” the congregations after having been exiled from them for a time under Claudius. Whether that is historically certain or not, it is clear from within the letter that Paul is consciously writing to a congregation composed of both ethnic Jews and ethnic Greeks. This concern about ethnicity is prominent throughout the book. For instance, when Paul says “all people” he follows up just to be clear, “both Jews and Greeks” (e.g. 3:9), so they don’t point fingers at the other group.
But while the ethnicities within the congregation is a prominent issue in Paul’s mind, at the same time he’s trying to transcend ethnic categories by constantly lumping them together, “both Jews and Greeks.” How many times does some form of this phrase occurs in Romans? Lots (I’m still working on a precise count…).
While Paul recognizes that ongoing ethnic distinctions are real, he is at the same time trying to deconstruct their understanding of what it means to be Jewish or to be Greek. For instance, the Greeks know more about the God of Israel than they are willing to admit (1:18-23), and they are more accountable to Israel’s God than they’d like to think (2:12-16). At the same time, Jews have less distinct privilege than they think (2:28-29), and they are living more like the Greeks than they’d like to admit (2:23-24).
These dynamics don’t eliminate ethnic distinctions, but they do highlight the fact that the theologically controlling categories are no longer ethnic, but rather spiritual. So Paul says, “For no one is a Jew who is merely one outwardly, nor is circumcision outward and physical,  but a Jew is one inwardly, and circumcision is a matter of the heart, by the Spirit, not by the letter” (2:28-29).
Still, because Paul is writing to an ethnically mixed congregation, we see him using ethnic language to address these groups and their concerns, but then transcending ethnic language to instruct these groups with new doctrinal categories.
For this reason we see the words “Jew” and “Israel” functioning in two different ways in the letter to the Romans, sometimes identifying Israel as an ethnic entity, and other times as a spiritual reality. This is what’s going on in 2:28, where he says, “No one is a Jew who is merely one outwardly.” Which is like saying, “No one is a Jew who is merely a Jew.” But in two different senses of the word. The spiritual sense followed by the ethnic sense. “No one is a Jew (truly and spiritually) who is merely a Jew (ethnically).”
Thus when reading through Romans, it is incumbent upon the interpreter to assess which meaning of “Jew” or “Israel” Paul is referring to—spiritual or ethnic? This will significantly impact one’s conclusions about Paul’s theology, especially in Romans 9-11.