Tag Archives: PPI

BLS Learns from Civic Digital Fellows

In the few months that I’ve had the pleasure of occupying the Commissioner’s seat at the Bureau of Labor Statistics, it’s been clear that I’m surrounded by a smart, dedicated, and innovative staff who collect and publish high-quality information while working to improve our products and services to meet the needs of customers today and tomorrow. And soon after I arrived, we added to that high-quality staff by welcoming a cadre of Civic Digital Fellows to join us for the summer.

In its third year, the Civic Digital Fellowship program was designed by college students for college students who wanted to put their data science skills to use helping federal agencies solve problems, introduce innovations, and modernize functions. This year, the program brought 55 fellows to DC and placed them in 6 agencies – Census Bureau, Citizenship and Immigration Service, General Services Administration, Health and Human Services, National Institutes of Health, and BLS. From their website:

Civic Digital Fellowship logo describing the program as "A first-of-its-kind technology, data science, and design internship program for innovative students to solve pressing problems in federal agencies."

BLS hosted 9 Civic Digital Fellows for summer 2019. Here are some of their activities.

  • Classification of data is a big job at BLS. Almost all of our statistics are grouped by some classification system, such as industry, occupation, product code, or type of workplace injury. Often the source data for this information is unstructured text, which must then be translated into codes. This can be a tedious, manual task, but not for Civic Digital Fellows. Andres worked on a machine learning project that took employer files and classified detailed product names (such as cereal, meat, and milk from a grocery store) into categories used in the Producer Price Index. Vinesh took employer payroll listings with very specific job titles and identified occupational classifications used in the Occupational Employment Statistics program. And Michell used machine learning to translate purchases recorded by households in the Consumer Expenditure Diary Survey into codes for specific goods and services.
  • We are always looking to improve the experience of customers who use BLS information, and the Civic Digital Fellows provided a leg up on some of those activities. Daniel used R and Python to create a dashboard that pulled together customer experience information, including phone calls and emails, internet page views, social media comments, and responses to satisfaction surveys. Olivia used natural language processing to develop a text generation application to automatically write text for BLS news releases. Her system expands on previous efforts by identifying and describing trends in data over time.
  • BLS staff spend a lot of time reviewing data before the information ends up being published. While such review is more automated than in the past, the Civic Digital Fellows showed us some techniques that can revolutionize the process. Avena used Random Forest techniques to help determine which individual prices collected for the Consumer Price Index may need additional review.
  • Finally, BLS is always on the lookout for additional sources of data, to provide new products and services, improve quality, or reduce burden on respondents (employers and households). Christina experimented with unit value data to determine the effect on export price movements in the International Price Program. Somya and Rebecca worked on separate projects that both used external data sources to improve and expand autocoding within the Occupational Requirements Survey. Somya looked at data from a private vendor to help classify jobs, while Rebecca looked at data from a government source to help classify work tasks.

The Civic Digital Fellows who worked at BLS in summer 2019

Our cadre of fellows has completed their work at BLS, with some entering grad school and the working world. But they left a lasting legacy. They’ve gotten some publicity for their efforts. Following their well-attended “demo day” in the lobby at BLS headquarters, some of their presentations and computer programs are available to the world on GitHub.

I think what most impressed me about this impressive bunch of fellows was the way they grasped the issues facing BLS and focused their work on making improvements. I will paraphrase one fellow who said “I don’t want to just do machine learning. I want to apply my skills to solve a problem.” Another heaped praise on BLS supervisors for “letting her run” with a project with few constraints. We are following up on all of the summer projects and have plans for further research and implementation.

We ended the summer by providing the fellows with some information about federal job opportunities. I have no doubt that these bright young minds will have many opportunities, but I also saw an interest in putting their skills to work on real issues facing government agencies like BLS. I look forward to seeing them shine, whether at BLS or wherever they end up. I know they will be successful.

And, we are already making plans to host another group of Civic Digital Fellows next summer.

What Happened to Natural Gas Prices at the End of 2018?

Natural gas prices mirrored a rollercoaster during the last few months — lots of ups and downs. Let’s explore why. The U.S. Bureau of Labor Statistics publishes many indexes that measure changes in prices for natural gas. Producer Price Indexes and Import Price Indexes tell us about these and other price movements faced by U.S. businesses. Natural gas is critical to the U.S. economy, and changes in natural gas prices can have a large impact on our daily lives. You may use natural gas for cooking or heating your home, but did you know these facts about natural gas?

  • Natural gas has surpassed coal as the largest source of electricity generation in the United States.
  • The United States has become the world’s largest natural gas producer and consumer.
  • Natural gas consumption in the United States reached historic highs in 2018.

According to the U.S. Energy Information Administration, natural gas production reached record levels in 2018, driven by improved drilling techniques (such as hydraulic fracturing or “fracking”), more wells, and increased crude oil production. Natural gas can be produced either through direct extraction or as a byproduct of crude oil drilling. With more production, we might expect prices to fall because more natural gas is available on the market. Natural gas prices increased sharply in the fourth quarter of 2018, however.

Let’s look at that rollercoaster of natural gas prices. In the early part of 2018, normal seasonal fluctuations and increased production pushed natural gas prices down. However, higher U.S. demand limited the decline in prices. Prices leveled off somewhat during the spring and summer months. Over the final 3 months of 2018, U.S. import prices for natural gas increased by 138.8 percent, the largest quarterly increase since the index began in 1982. Likewise, producer prices for natural gas increased by 90.2 percent over the same period.

U.S. import and domestic producer price indexes for natural gas, December 2017 to January 2019

Editor’s note: Data for this chart are available in the table below.

There were many reasons prices rose sharply in the last 3 months of 2018. The industrial and electric power sectors are the biggest users of natural gas. The record-high natural gas consumption in the United States in 2018 was driven by record-high demand from the electric power sector. Despite record production in 2018, natural gas storage stocks hit a 16-year low by the end of the year. More domestic consumption and increased exports cut into the natural gas inventory. That pushed prices up starting in October, but there were other reasons prices rose.

An explosion of a major Canadian natural gas pipeline disrupted supply in mid-October. This explosion sharply limited U.S. imports of natural gas from Western Canada and reduced natural gas supply in the Northwest United States. Natural gas distributors asked customers to restrain usage, but import prices still rose. In November, imports from the pipeline grew to about half the amount before the explosion. Even though the natural gas supply from the pipeline moved toward full capacity in December, the demand for natural gas kept upward pressure on prices.

California also experienced natural gas pipeline capacity and storage issues later in the year. Those issues pushed prices up for the entire West Coast region during the fourth quarter of 2018.

Domestically, the increased demand for natural gas over the year resulted in the fourth lowest volume added into working stocks during the refill season since 2005. The refill season, typically April through October, is when natural gas supply typically outpaces demand, allowing working stocks to grow for the upcoming winter season. By November’s end, working gas stocks in the lower 48 states were below 3 trillion cubic feet for the first time since 2002 because of the depletion over the year. This lower supply, coupled with reduced imports, pushed domestic prices up as demand grew from electricity generation and heating needs from severe cold weather in most regions of the country.

Prices then fell in January 2019. Import prices for natural gas decreased by 44.2 percent in January, and producer prices fell 32.2 percent. Imported natural gas from Western Canada returned to more normal levels in the first half of January. As a result, natural gas supply caught up to demand, pushing prices lower. In addition, warmer-than-average temperatures during the beginning of January limited demand for natural gas, also placing downward pressure on prices.

Want to learn more?

U.S. import and domestic producer price indexes for natural gas
Month Domestic producer prices U.S. import prices
Dec 2017 100.0 100.0
Jan 2018 96.5 113.4
Feb 2018 107.5 95.8
Mar 2018 80.6 85.8
Apr 2018 82.3 84.2
May 2018 85.8 63.4
Jun 2018 88.3 60.0
Jul 2018 90.3 72.9
Aug 2018 89.1 69.3
Sep 2018 89.9 68.1
Oct 2018 90.5 80.3
Nov 2018 104.0 128.6
Dec 2018 171.0 162.7
Jan 2019 115.9 90.7

Why This Counts: What is the Producer Price Index and How Does It Impact Me?

The Producer Price Index (PPI) – sounds familiar, but what is it exactly? Didn’t it used to be called the Wholesale Price Index? It is related to the Consumer Price Index, but how? How does the PPI impact me?

Lots of questions! In this short primer we will provide brief answers and links for more information. Note, if you are an economist, this blog is NOT for you. It’s an introduction for everyone else!

Video: Introduction to the Producer Price Index

Before we go any further – what is an index? (You said this was a primer!)

An index is like a ruler. It is a way of measuring the change of just about anything. Producer price indexes measure the average change in prices for goods, services, or construction products sold as they leave the producer.

Here is an example of how an index works:

  • Suppose we created an index to track the price of a gallon of gasoline.
  • When we start tracking, gasoline costs $2.00 a gallon.
  • The starting index value is 100.0.
  • When gasoline rises to $2.50, our index goes to 125.0, which reflects a 25-percent increase in the price of gasoline.
  • If gasoline then drops to $2.25, the index goes to 112.5. The $0.25 decline in price reflects a 10-percent decrease in the price of gasoline from when the price was $2.50.

If you are a gasoline dealer, you might find a gasoline index useful. Instead of driving around every day to write down the prices of each competitor’s gasoline and averaging them together, the index can provide the data for you. (Question #5 in the PPI Frequently Asked Questions explains how to interpret an index.)

PPI is called a “family” of indexes. There are more than 10,000 indexes for individual products we release each month in over 500 industries. That is one big family!

OK, so PPI has lots of data – but what kind of data?

PPI produces three main types of price indexes: industry indexes, commodity indexes, and final demand-intermediate demand (FD-ID) indexes.

An industry refers to groups of companies that are related based on their primary business activities, such as the auto industry. The PPI measures the changes in prices received for the industry’s output sold outside the industry.

  • PPI publishes about 535 industry price indexes and another 500 indexes for groupings of industries.
  • By using the North American Industry Classification System (NAICS) index codes, data users can compare PPI industry-based information with other economic programs, including productivity, production, employment, wages, and earnings.

The commodity classification of the PPI organizes products by type of product, regardless of the industry of production. For example, the commodity index for steel wire uses pricing information from the industries for iron and steel mills and for steel wire drawing.

  • PPI publishes more than 3,700 commodity price indexes for goods and about 800 for services.
  • This classification system is unique to the PPI and does not match any other standard coding structure.

We also have more information on the differences between the industry and commodity classification systems.

The FD-ID classification of the PPI organizes groupings of commodities by the type of buyer. For example, the PPI for final demand measures price change in all goods, services, and construction products sold as personal consumption, capital investment, export, or to government. As a second example, the PPI for services for intermediate demand measures price change for services sold to business as inputs to production.

  • PPI publishes more than 300 FD-ID indexes.
  • This FD-ID classification system is unique to the PPI and does not match any other standard coding structure.

Now let’s go back to the beginning

  • 1902: Wholesale Price Index program begins, which makes it one of the oldest continuous set of federal statistics. The Wholesale Price Index captures the prices producers receive for their output. In contrast, the Consumer Price Index captures the prices consumers pay for their purchases.
  • 1978: BLS renames the program as the Producer Price Index to more accurately reflect that prices are collected from producers, rather than wholesalers.
  • PPI also shifts emphasis from a commodity index framework to a stage of processing index framework. This minimized the multiple counting that can occur when the price for a specific commodity and the inputs to produce that commodity are included in the same total index. For example, think of gasoline and crude petroleum both included in an all-commodities index.
  • 1985: PPI starts expanding its coverage of the economy to include services and nonresidential construction. As of January 2018, about 71 percent of services and 31 percent of construction are covered.
  • 2014: PPI introduces the Final Demand-Intermediate Demand system.
  • The “headline” number for PPI is called the PPI for Final Demand. It measures price changes for goods, services, and construction sold for personal consumption, capital investment, government purchases, and exports. We also produce a series of PPIs for Intermediate Demand, which measure price change for business purchases, excluding capital investment.
  • Let me give you an example: Within the PPI category for loan services, we have separate indexes for consumer loans and business loans. The commodity index for consumer loans is included in the final demand index and the commodity index for business loans is mostly in an intermediate demand index.
  • The Frequently Asked Question on the PPI for Final Demand provides even more information on this new way of measuring the PPI. The blog, Understanding What the PPI Measures, may also be helpful.
  • We also have an article that explains how the PPI for final demand compares with other government prices indexes, such as the CPI.

Why is the PPI important?

To me?

  • Inflation is the higher costs of goods and services. Low inflation may be good for the economy as it increases consumer spending while boosting corporate profits and stocks.
  • A change in producer prices may be a leading indicator of consumers paying more or less. Higher producer prices may mean consumers will pay more when they buy, whereas lower producer prices may mean consumers will pay less to retailers. For example, if the PPI gasoline index increases, you may see an increase soon at the pump!

To others (which may impact me!)?

  • Policymakers, such as the Federal Reserve, Congress, and federal agencies regularly watch the PPI when making fiscal and monetary policies, such as setting interest rates for consumers and businesses.
  • Business people use the PPI in deciding price strategies, as they measure price changes in inputs for their goods and services. For example, a company considering a price increase can use PPI data to compare the growth rate of their own prices with those in their industry.
  • Business people adjust purchase and sales contracts worth trillions of dollars by using the PPI family of indexes. These contracts typically specify dollar amounts to be paid at some point in the future. For example, a long-term contract for bread may be escalated for changes in wheat prices by applying the percent change in the PPI for wheat to the contracted price for bread.

Video: How the Producer Price Index is Used for Contract Adjustment

PPI is a voluntary survey completed by thousands of businesses nationwide every month. BLS carefully constructs survey samples to keep the number of contacts to a minimum, making every business, large and small, critical to the accuracy of the data. We thank you, our faithful respondents! Without you, BLS could not produce gold-standard PPI data.

Finally, check out the most recent monthly PPI release to get all the latest numbers. Head to the PPI Frequently Asked Questions to learn more. Or contact the PPI information folks at (202) 691-7705 or ppi-info@bls.gov.

Want to learn more about BLS price programs? See these blogs:


Data Privacy Day is Every Day at BLS

There are many commemorative days, weeks, and months, but Data Privacy Day on January 28 is one that we here at BLS live every day of the year.

If this is the first time you’re hearing about it, Data Privacy Day is an international effort to “create awareness about the importance of:

  • respecting privacy,
  • safeguarding data and
  • enabling trust.”

These three phrases are central to everything we do at BLS—but don’t take my word for it! Instead, let’s hear from some of our staff members about what data privacy means in their day-to-day work lives.

I chatted with staff members from three key areas at the Bureau:

  • Collection — our field economists collect data from respondents.
  • Systems — our computer specialists protect the IT infrastructure where we keep the data.
  • Analysis — our economists analyze the data, prepare products, and explain the data to our customers.

Now, let’s meet the staff.

Richard Regotti

Richard Regotti

My name is Richard Regotti, Field Economist in the BLS Chicago Regional Office, Cleveland Area Office. I have proudly served the public in this position for 12 years. As a Field Economist I am responsible for collecting data and developing positive relationships and securing cooperation from survey respondents for the Producer Price Index and the International Price Indexes.







Jess Mitchell

Jess Mitchell

My name is Jess Mitchell and I have been an Information Security Specialist in the Bureau’s national office since 2013. I started with BLS in 1999. Currently, I am the Computer Security Incident Response Team Lead, so I, along with my team members, investigate, analyze and report on computer security incidents as well as the impact or potential impact of cyber threats and vulnerabilities to BLS systems and data.






Karen Kosanovich

Karen Kosanovich

My name is Karen Kosanovich, Economist, and I have spent the past 19 years working with unemployment data from the Current Population Survey, and 25 years total at BLS. I develop analyses, such as The Employment Situation, and talk to our customers about the data.







Question 1. One of our core BLS values is the confidentiality of data: All respondent data are completely confidential and used for statistical purposes only. How does this impact you in your daily work?

Richard: On a daily basis I am asking producers and service providers to voluntarily provide very sensitive company information. Even after identifying myself as a representative of our Federal Government, some respondents are not comfortable with agreeing to provide us their confidential information for use in our statistical output. By focusing on the mission of the BLS and the legal protections that are in place to safeguard survey data, I am able to function on the front line as a data collector.

Jess: This core value of data confidentiality helps me to focus on the importance of protecting the confidentiality of BLS data when my team members and I are investigating threats. The importance of BLS data underscores the importance of our daily work to keep BLS data and data respondent information confidential.

Karen: I don’t have access to information about specific people who respond to our survey. All personally identifying information is stripped away before the statistical information is given to an economist like me to analyze. For my colleagues and me, confidentiality means protecting our estimates from being distributed in advance of the official release of the unemployment rate at 8:30 a.m. on the day we publish our data.

Question 2. Does adherence to this core value create any challenges for you in your work? How have you overcome those challenges?

Richard: Adherence to complete confidentiality, supported by the fact that the data are used for statistical purposes only, presents no challenge to me; this core value is a selling point and something I make sure all potential survey participants are aware of prior to providing any data to the BLS.

Jess: Adherence to the core BLS value of data confidentiality does create a challenge when we need to engage our office in an incident or threat investigation; we must be very diligent not to share Confidential Information Protection and Statistical Efficiency Act (CIPSEA) information.

Karen: Our procedures for working with embargoed (prerelease) information are so ingrained in my work routine that I don’t notice any challenges from them. The people I work with all have the same responsibility and a strong commitment to public service, so it is easy for us to keep vigilant.

Question 3. If you could make a statement to the American people about why they should trust BLS with their information, what would that be?

Richard: BLS is not a compliance or regulatory agency in any way. We are only concerned with providing accurate, timely, relevant, and unbiased data that reports on the health and well-being of our economy. Your information contributes to the validity of BLS data.

Jess: The confidentiality of BLS data is always at the root of my office’s work, and I see the same focus on data privacy and confidentiality and diligence toward the safeguarding of CIPSEA data throughout the entire culture of BLS.

Karen: Although I don’t have names and personal details of specific unemployed people who respond to our survey, my colleagues and I are very mindful of the importance of representing the experience of all Americans when we produce our estimates. The data we publish are not just numbers, but tell the story of real people. It can be very stressful to be unemployed, and those who have been looking for work for a very long time face significant challenges in the labor market. We take our jobs, and our mission, very seriously.

And now the rules:

Of course, we don’t work in a vacuum. Like any other organization, we have rules that we live under.

BLS makes a pledge of confidentiality to its respondents that data collected are used for statistical purposes only. The pledge is covered by CIPSEA, which makes it a felony to disclose or release the information for either nonstatistical purposes (for example, regulatory or law-enforcement purposes) or to unauthorized persons. In addition, the Office of Management and Budget has Statistical Policy Directives (3 and 4) that govern BLS news releases to ensure they meet specific accuracy, timeliness, and accountability standards.

On January 28, and every day, we hope you will take steps to protect your own privacy and the privacy of others. Here at BLS we will continue to educate and raise awareness about respecting privacy and safeguarding data. It is core to our mission and central to our staff values. Without the trust these actions produce among the American people, we could not do our work in providing gold-standard data for and about America’s workers.

Thank you for your trust and happy Data Privacy Day!

Measuring Uncertainty in the Producer Price Index

Our mission at the U.S. Bureau of Labor Statistics is to publish information about the labor market and economy. We always seek to improve our methods and provide the most accurate data in a cost-effective manner. All statistics, however, come with some uncertainty. Last year I wrote about how we deal with uncertainty in our measures. Today let’s talk about how we recently have improved our uncertainty measures in the Producer Price Index.

You may think it’s odd that an agency that tells the public what we know also works hard to explain what we don’t know. It may seem like we’re airing our dirty laundry, but that’s not how we see it. At BLS, one of our core values is to be transparent about our methods. Not only don’t we consider the laundry dirty, but we believe that airing it—that is, giving you more information about the strengths and the limitations of our data—is central to our mission. It’s part of our responsibility to give you information you can use to make better decisions.

The Producer Price Index (PPI) program measures the average change over time in the prices U.S. businesses receive for the goods they produce and the services they provide. BLS started publishing the PPI 126 years ago, making it one of our oldest measures. In 2014, the PPI expanded its coverage to provide a broader view of price change for goods, services, and construction. The PPI for final demand measures price change for goods and services sold for personal consumption, capital investment, government, and export. The PPI for intermediate demand tracks price change for goods, services, and construction products sold to businesses.

The PPI for final demand was unchanged in October 2016 and was up 0.8 percent over the last 12 months. But these figures are subject to sampling error. What’s that? It’s the uncertainty that results when we collect data from a sample of prices, rather than gathering prices from each of the millions of transactions that occur every day. For the PPI, we collect about 93,500 prices every month. A different sample of prices might give us different estimates of price change. Fortunately, we have tools to measure this sampling error. Most BLS programs collect data from sample surveys because it is far too expensive and would overburden businesses and workers to send all our surveys to everyone. Instead, we select samples carefully using scientific methods. These sampling methods work well, but they can’t avoid the possibility that the characteristics of a sample may differ from those of the population. We provide estimates of this sampling error by publishing variance estimates with the data. We recently released the first-ever variance estimates for the PPI.

If you aren’t into math, skip the next paragraph.

The measure of variance we use for the PPI is called a standard error. We use the standard error to calculate what statisticians call a confidence interval around the estimate. For example, the 1-month median absolute percent change in the PPI for final demand in 2015 was 0.30 percent. The standard error of that median was 0.11 percent. We can use these two numbers to calculate a confidence interval. In this example, we will use what we call a 95-percent confidence interval. To calculate that confidence interval, we take the estimated median price change of 0.30 percent, plus and minus two times the standard error of 0.11 percent. This gives us a confidence interval between 0.08 percent and 0.52 percent. We call this a 95-percent confidence interval because, if we were to choose 100 different samples of producer prices, the median price change would be between 0.08 percent and 0.52 percent in 95 of those samples.

Chart showing median 1-month changes in Producer Price Indexes in 2015 and the 95-percent confidence intervals around those changes.

OK, if you don’t like math, you can come back now. The chart above shows estimates of 1-month PPI changes (the red dots) each surrounded by its sampling uncertainty (the blue bars). If the blue bar crosses the 0.0 percent line, it means the change is not significantly different from zero.

Variance estimates are just one way BLS evaluates and explains the quality of our data and our methods. We have published information about our methods almost since our beginnings in 1884. Carroll Wright, the first BLS Commissioner, insisted on the “fearless publication of the facts.” We believe the fearless publication of the facts means not just explaining our measures and methods in highly technical terms. We want our measures and the uncertainty around them to be understood by a wide range of people, not just those who have advanced degrees in economics or statistics. We continue to seek clearer ways to explain uncertainty. One way is a new chart we are publishing on the monthly changes in nonfarm employment. In the future, we hope to publish more charts like this and simpler explanations of our methods. If you have ideas on how we can explain our data and methods more clearly, please share them with us below.

BLS data are the gold standard of economic statistics. But even gold bars have marks to indicate their impurities. Similarly, we at BLS don’t hide our impurities. We want you to understand the strengths and limitations of our data so you can use that knowledge to make good decisions.