Living on a budget

In a policy brief published by the Stanford Institute for Economic Policy Research (SIEPR), Matthew Harding, assistant professor of economics and a research fellow at SIEPR, describes how consumers make purchasing decisions, how they search for products and make trade-offs, and the implications of these choices.

Matthew Harding

Matthew Harding is an assistant professor of Economics at Stanford and a research fellow of SIEPR.


The grocery market in the United States accounts for more than 25 percent of all retail sales, totaling hundreds of billions of dollars. The last decade has seen important transformations in the market structure of this industry with non-traditional retailers such as Walmart and Costco accounting for more than 30 percent of total grocery sales. For any given product, the spread of consumer prices from different retailers is orders of magnitude larger than price fluctuations from changing world food market conditions. The average consumer faces prices that differ by more than 20 percent across stores in the same area.

Much standard economic research focuses on prices from retailers’ perspectives and is likely to miss the complexity of the effects of prices on consumer welfare. Consumers make many trade-offs: They substitute between similar products, search stores for lower prices, look out for sales and use store coupons. U.S. households spend very different shares of income on food and face different choice sets, both in terms of the stores and the products that are available. They have different habits in terms of the brands and quantities that they seek. Furthermore, consumers in different parts of the country face different prices and have different competitors and types of stores.

USDA compares food purchases by U.S. households of different income levels, showing that while all households engage in some sets of economizing practices, there are substantial differences across household types. Low-income households may shop more often in discount food stores, make imperfect nutritional choices or even pay higher prices because low-income households tend to be more concentrated in urban and rural areas. Low-income households appear to be more sensitive to discounted products, prefer generic labels and have an increased propensity to take advantage of volume discounts.

This policy brief summarizes my recent research into how consumers make purchasing decisions, how they search for products and make trade-offs, and the welfare implications of these choices.

Measuring shopping behavior

Recently, large-scale datasets on consumer purchases have become available, containing detailed information on all purchases made by a house-hold at the bar code level in addition to detailed household demographics. The massive amount of information presents
unique opportunities and challenges. On the one hand it allows us to extract detailed information about how consumers shop. At the same time we may need to worry that substantial measurement error in such large datasets may impact our analysis.

The sample used for the analysis in Harding and Lovenheim (2009a) comes from ACNielsen’s Homescan data and contains the price, quantity and the store of purchase for each transaction made by a household for the two-year period, January 1, 2004, through December 31, 2005. These data are recorded using an electronic home scanning unit. We restrict our attention to 28 product categories aggregated from individual UPCs, which cover approximately 1/3 of the transactions of a typical household in the sample.

Our final dataset contains 47,342 households in 52 metropolitan areas and a total of 13,162,440 household-by-week-by-product observations. A major advantage of these data is that they are linked to the demographics of the consumers. For each household, we have information on the number of household members, whether children under 18 are present, race, total household income, age, educational attainment and employment status. In general, the demographic characteristics of our sample line up closely with the national characteristics of the urban population in the United States, which suggests our sample is representative of urban consumers.

Price-convenience trade-off

A first question of interest is the extent to which consumers search for the best price and whether they chose a more expensive yet closer store. Burda, Harding and Hausman (2008) analyze consumer preferences over price and distance from a sub-sample of 675 households from the Nielsen data in the Houston area. Each household has a choice among 5 specific stores (H.E. Butt, Kroger, Randall’s, Walmart, PantryFoods) and an option of all other stores that fall under the standard grocery format.

Most consumers shop in at least two different stores in a given month, with the average number of trips to their first-choice store being approximately once a week. The analysis focuses on two key variables: price, which corresponds to the price of a basket of goods in a given store-month, and distance, which corresponds to the estimated driving distance for each household to the corresponding supermarket.

We find that most consumers appear to be insensitive to the cost of shopping for groceries both in terms of price and distance. Nevertheless some consumers are particularly sensitive to the cost of shopping for groceries and are willing to travel longer distances searching for a better deal while others value proximity to the store and are prepared to pay a higher price for the convenience.

Shopping frequency and expenditure

The simple trade-off between price and convenience however fails to take into account the extent to which households can vary the number of trips to a given store as part of their search strategies. Moreover, households have very different levels of expenditures, which are likely to be correlated with their demographics. In a more recent paper, Burda, Harding and Hausman (2009) analyze a sample of 1,210 households in the Houston area. The histogram in Figure 1 summarizes the frequency of each trip count for the households in the sample.

Figure 1 – Histogram shows the monthly total number of trips to a store for Houston households.

Figure 1 – Histogram shows the monthly total number of trips to a store for Houston households.

Consumers make important trade-offs between price, distance and expenditure consistent with an opportunity cost of time economic model. The shopping intensity of households, controlled for all other factors, declines for single-parent homes and for non-white households, while increasing in the presence of children in the household and for households whose head has reached retirement age or is unemployed.

When analyzing household purchasing behavior, it is extremely important to take into account that households often have strong store brand loyalty. Large stores like Walmart are typically shown to induce a significant degree of loyalty in their customers.

One simple measure of search intensity used by Harding and Lovenheim (2009) is the percentage difference between the minimum price in a consumer’s metro area in a given week for a product and the price paid by the consumer in that week for that good. If all consumers buy a product at the market minimum in each week, this measure will equal zero. But, if the average consumer price is twice that of the market minimum, this measure will equal one. An analysis by product category shows that most consumers pay considerably more for each product than the minimum prevailing market price. For example, the average consumer in our sample pays 83.5 percent more for shredded cheese than the minimum market price but pays 275.2 percent more for bottled water than the market minimum in that week. Furthermore, a more detailed examination shows there is considerable variation in this search intensity measure. Across categories, most of the percentage price margins are between 1.0 and 3.0, but fresh eggs and ice cream have considerably higher average margins, at 4.68 and 15.26, respectively.

Demographics and inequality

For each product group we can investigate the search intensity separately and relate it to underlying household demographics. Figure 2 shows that income appears to have a large effect on the economizing behavior of households, with richer households paying on average 20 percent more for a comparable unit of the same product. The variance in the degree of savings increases substantially with household income. At the same time, there is no effect of income on the quantity purchased. Our results support both an opportunity cost of time explanation where rich households face substantially higher costs of searching as well as a quality upgrading effect where rich households substitute toward more expensive products within the same product group.

Figure 2 – Box Plots of Percent Price Margin by Household Income Bracket: It shows the extent to which richer households spend more and search less.

Figure 2 – Box Plots of Percent Price Margin by Household Income Bracket: It shows the extent to which richer households spend more and search less.

We find limited effects of gender except for a large effect for females over 65 who appear to save more and consume less. We also find that unemployed males appear to buy more food without any significant changes in the extent to which they search for a better deal. Unemployed women also appear to purchase substantially higher quantities, but they engage in more search.

Large markets appear to increase geographic search costs. Consumers search more when they buy expensive products, just as economic search theory would predict. We also find a consistently large and negative effect of the purchased quantity on search intensity. The extent of economizing behavior is significantly larger for cookies or candy than for eggs or yoghurt, potentially reflecting the perishing of the products when bought in larger quantities. We do not find significant race or education effects.

Geography and sin taxes pass-through

A fascinating new area of inquiry has opened up recently with the availability of scanner data for cigarettes, beer, wine and liquor. Harding, Leibtag and Lovenheim (2009) explore the effect of border effects in the tax pass-through for these products. Consumers living close to state borders may drive to purchase these products if the bordering state has lower taxes. Hence geography limits the extent to which taxes on these goods can be passed on to the consumer, with important public welfare implications. The data show that excise taxes for beer and liquor have a very low pass-through while taxes on cigarettes have a pass-through of over 0.8. The interaction between tax differentials and distance to the nearest lower tax border plays an important role in determining the price paid by consumers.

In summary, it appears that consumers make complicated trade-offs in their shopping behavior subject to a variety of economic, institutional and demographic constraints, which lead to substantial inequality in outcomes with important policy implications. 


About the author

Matthew Harding is an Assistant Professor of Economics at Stanford University and a Research Fellow of SIEPR. He is an economist who conducts research on theoretical and applied econometrics focusing on the analysis of large panel datasets and the modeling of latent preferences and heterogeneity. This research has important implications for the estimation of financial risk, the estimation of social networks and the measurement of consumer preferences from scanner data. Recently he co-founded the Carbon Action Partnership, an interdisciplinary academic group focusing on the design and measurement of interventions in energy use and behavior. He  received a Ph.D. From MIT and an M.Phil from Oxford University.



Burda, M., M. Harding, and J. Hausman (2008): “A Bayesian Mixed Logit Probit Model for Multinomial Choice,” Journal of Econometrics, 147(2), 232–246.

Burda, M., M. Harding, and J. Hausman (2009): “Understanding Choice Intensity: A Poisson Mixture Model with Logit-Based Random Utility Selective Mixing,” mimeo.

Harding, M., E. Leibtag, and M. Lovenheim (2009): “The Spatial and Socio-Economic Distribution of the Incidence of Sin Taxes: Evidence from Homescan Data on Cigarette and Alcohol Purchases,” mimeo.

Harding, M., and M. Lovenheim (2009): “Living on a Budget: National Evidence on Price Search and Consumer Heterogeneity from Homescan Data,” mimeo.