In a series of essays, Said argues the impact of mainstream culture (mainly British writers of the 19th and early 20th century, like Jane Austen and Rudyard Kipling) on colonialism and imperialism, and conversely how imperialism, resistance to it, and decolonization influenced the English and French novel. In the introduction to the work, Said explains his focus on the novel: he "consider[s] it the aesthetic object whose connection to the expanding societies of Britain and France is particularly interesting to study. The prototypical modern realistic novel is Robinson Crusoe, and certainly not accidentally it is about a European who creates a fiefdom for himself on a distant, non-European island."

On the connection between culture and empire, Said observes that "The power to narrate, or to block other narratives from forming and emerging, is very important to culture and imperialism, and constitutes one of the main connections between them." Hence he analyzes cultural objects in large part to understand how empire works: "For the enterprise of empire depends upon the idea of having an empire... and all kinds of preparations are made for it within a culture; then in turn imperialism acquires a kind of coherence, a set of experiences, and a presence of ruler and ruled alike within the culture."

Said defines "imperialism" as "the practice, the theory, and the attitudes of a dominating metropolitan center ruling a distant territory." His definition of "culture" is more complex, but he strongly suggests that we ought not to forget imperialism when discussing it. Of his overall motive, Said states:

"The novels and other books I consider here I analyze because first of all I find them estimable and admirable works of art and learning, in which I and many other readers take pleasure and from which we derive profit. Second, the challenge is to connect them not only with that pleasure and profit but also with the imperial process of which they were manifestly and unconcealedly a part; rather than condemning or ignoring their participation in what was an unquestioned reality in their societies, I suggest that what we learn about this hitherto ignored aspect actually and truly enhances our reading and understanding of them."

The title is thought to be a reference to two older works, Culture and Anarchy (1867–68) by Matthew Arnold and Culture and Society (1958) by Raymond Williams.

Said argues that, although the "age of empire" largely ended after World War II, when most colonies gained independence, imperialism continues to exert considerable cultural influence in the present. To be aware of this fact, it is necessary, according to Said, to look at how colonialists and imperialists employed "culture" to control distant land and peoples.