Tag Archives: Methodology

What is “Benchmarking” of Bureau of Labor Statistics Employment Data?

BLS has released the “preliminary benchmark” information for the Current Employment Statistics (CES) survey, the source of monthly information on jobs.

You know what a bench is

Image of a park bench

and you know what a mark is,

Image of a checkmark

but what pray tell is a benchmark? And what does this preliminary benchmark tell us?

So as not to bury the lead, I’ll let you know that this year’s preliminary estimate of the benchmark revision is a bit bigger than it has been in the last few years. Our preliminary estimate indicates a downward adjustment to March 2019 total nonfarm employment of 501,000. Still, that estimated revision is only -0.3 percent of nonfarm employment. In most years our monthly employment survey has done a good job at estimating the total number of payroll jobs. More details on that below. This year our survey estimates are off more than we would like. Our goal is to provide estimates that are excellent and not just good or pretty good, and that’s why we benchmark the survey data each year.

What is benchmarking and why do we do it?

The CES is a monthly survey of approximately 142,000 businesses and government agencies composed of approximately 689,000 individual worksites. As with all sample-based surveys, CES estimates are subject to sampling error. This means that while we work hard to ensure those 689,000 worksites represent all 10 million worksites in the country, sometimes our sample may not perfectly reflect all worksites. So the monthly CES estimates aren’t exactly the same as if we had counted employment from all 10 million worksites each month. To fix this problem, we “benchmark” the CES data to an actual count of all employees, information that’s only available several months after the initial CES data are published.

In essence, we produce employment information really quickly from a sample of employers, then anchor that information to a complete count of employment once a year.

The primary source of the CES sample is the BLS Quarterly Census of Employment and Wages (QCEW) program, which collects employment and wage data from states’ unemployment insurance tax systems. This is also the main source of the complete count of employment used in the benchmark process. QCEW data are typically available about 5 months after the end of each quarter.

Each year, we re-anchor the sample-based employment estimates to these full population counts for March of the prior year. This process—which we call benchmarking—improves the accuracy of the CES data. That’s because the population counts are not subject to the sampling and modeling errors that may occur with the CES monthly estimates. Since the CES data are re-anchored to March of the last year, CES estimates are typically revised from April of the year prior up to the March benchmark. Then estimates from the benchmark forward to December are revised to reflect the new March employment level.

We will publish the final benchmark revision in February 2020 and will incorporate revisions to data from April 2018 to December 2019. (Thus, we’re not showing a 2019 number in graph and table below). On August 21, BLS released a first look at what this revision will be—what we call the “preliminary benchmark.” This preliminary benchmark gives us an idea of what the revised nonfarm employment estimates for March 2019 will be.

The size of the national benchmark revision is a measure of the accuracy of the CES estimates, and we take pride that these revisions are typically small.

Chart showing differences in nonfarm employment after benchmarking, 2009–18

For total employment nationwide, the absolute annual benchmark revision has averaged about 0.2 percent over the past decade, with a range from −0.7 percent to +0.3 percent.

The following table shows the total payroll employment estimated from the CES before and after the benchmark over the past 10 years. For example, pre-benchmark employment for 2018 was 147.4 million; post-benchmark employment was also 147.4 million.

Nonfarm employment estimates before and after benchmarking, March 2009–March 2018
Year Level before benchmark Level after benchmark Difference Percent difference
2009 132,077,000 131,175,000 -902,000 –0.7
2010 128,958,000 128,584,000 -374,000 –0.3
2011 129,899,000 130,061,000 162,000 0.1
2012 132,081,000 132,505,000 424,000 0.3
2013 134,570,000 134,917,000 347,000 0.3
2014 137,147,000 137,214,000 67,000 <0.05
2015 140,298,000 140,099,000 -199,000 –0.1
2016 142,895,000 142,814,000 -81,000 –0.1
2017 144,940,000 145,078,000 138,000 0.1
2018 147,384,000 147,368,000 -16,000 <-0.05

The 2019 preliminary benchmark revision is following the same pattern, with an estimated difference of -0.3 percent. We provide this first look at the benchmark revision to give data users a sense of what we are seeing in the data. The final benchmark may be a little different—could be higher, could be lower. But based on recent experience, we are confident the benchmark released next February will show only a moderate difference from what we’ve been publishing each month and will validate the accuracy of our monthly CES estimates.

Want to know more? See our Current Employment Statistics webpage, send us an email, or call (202) 691-6555.

Why This Counts: Measuring Occupational Requirements

You probably know that BLS publishes data and analysis about employment, unemployment, job openings, earnings, productivity, occupational safety and health, and more. But did you know we also publish information about how often workers have to lift objects; the maximum weight they lift or carry; whether they work in extreme heat or cold; and how much training and experience they need for a job? We call these characteristics “occupational requirements.”

What are occupational requirements?

The Occupational Requirements Survey provides information about the requirements of jobs:

  • Physical demands of work, such as keyboarding, reaching overhead, lifting or carrying
  • Environmental conditions, such as extreme heat, exposure to outdoors, proximity to moving parts
  • Education, training, and experience requirements, such as prior work experience, on-the-job training, and license requirements
  • Cognitive and mental requirements, such as interaction with other people, independence of work, and the amount of review

How did BLS get into doing this survey?

This survey is one of our newest statistical programs; we first published data on December 1, 2016.

The Social Security Administration asked us to help them obtain accurate and current data to use in their disability programs. They are developing an Occupational Information System, which will use data from the Occupational Requirements Survey. That means the survey is crucial for Social Security to manage their disability programs fairly and efficiently.

How can I use occupational requirements information?

Users of Occupational Requirements Survey data include:

  • Researchers exploring occupational change
  • Jobseekers and students
  • Government agencies evaluating skill gaps
  • People with disabilities and their advocates

Let’s discuss a couple of examples to show you what I mean.

Educational requirements

You may want to know the minimum formal education requirements for jobs. The survey has a stat for that! In 2018, a high school diploma was required for jobs covering 40.7 percent of workers, while 17.9 percent had a bachelor’s degree requirement. The chart below shows the percent of jobs by minimum education requirement.

Percent of jobs with a minimum education requirement, 2018

Editor’s note: Data for this chart are available in the table below.

We have more information on education, training, and experience. The 2018 news release showed that on-the-job training was required for about 77 percent of workers, and the average duration was 34 days.

We also have information on preparation time, which includes minimum formal education, training, and work experience a typical worker needs to perform a job. Preparation time between 4 hours and 1 month was required for 31.5 percent of workers.

Environmental Conditions

Is the noise level at your workplace closer to a library (quiet) or a rock concert (very loud)? For some jobseekers, understanding the noise level and other environmental conditions might be extremely important as they evaluate job options. The chart below provides examples of the noise intensity in different occupations.

Percent of jobs with noise intensity level requirements, selected occupations, 2018

Editor’s note: Data for this chart are available in the table below.

Examples of work environments with different noise intensity levels include:

  • Quiet: private office, a golf course, or art museum
  • Moderate: department stores, business office, or fast food restaurant
  • Loud: manufacturing plant, atop large earth moving equipment, or jobs next to the highway
  • Very loud: rock concert venues, working with jack hammers, or rocket testing areas

How do we collect job requirement data?

To collect job requirement data, our field economists ask business owners, human resource professionals, worker safety officers, and supervisors to collect requirements of work. Field economists do not use paper or online questionnaires to collect these data; instead, they rely on a conversational interviews and descriptive documents, such as task lists, to collect information on occupational requirements.

How are we improving the survey?

Survey scope: Since it began, we have continued to refine the survey to improve its accuracy. In the third year of collection, we redefined the survey scope to focus on critical job functions—that is, the reason the job exists.

Survey content: Beginning with the current sample in collection, we added questions about cognitive and mental requirements. The Social Security Administration asked for this change so we can provide information on the requirements for workers to adapt to changes in the pace of work, solve problems, and interact with others.

Sample: The survey sample is collected over a 5-year period. That will provide the large amount of data necessary to publish information about detailed occupations. We have revised the sampling process to ensure we collect information about less common occupations.

Website: We recently improved the web layout to make it easier for users to find the data they want.

Where is more information?

We have data for occupational groups and occupations through the Occupational Profiles. All data are available through the public data tools. For concepts, methods, and history of the survey see the Handbook of Methods or visit our homepage.

Let us know if you have questions or comments about occupational requirements:

  • Email
  • Phone: (202) 691-6199

Use these gold-standard data to learn more about your job requirements or to find out about new ones. Whatever your occupational requirements question, “We have a stat for that!”

Percent of jobs with a minimum education requirement, 2018
Education requirement Percent
No minimum education requirement 31.5%
High school diploma 40.7
Associate’s degree 3.8
Associate’s vocational degree 2.1
Bachelor’s degree 17.9
Master’s degree 2.3
Professional degree 0.9
Doctorate degree 0.5
Percent of jobs with noise intensity level requirements, selected occupations, 2018
Occupation Quiet Moderate Loud
Bus and truck mechanics and diesel engine specialists 49.0% 51.0%
Computer programmers 60.1
Construction laborers 48.6 51.4
Electricians 49.0 51.0
Highway maintenance workers 46.2 53.8
Home health aides 54.1 45.9
Library technicians 56.0
Medical transcriptionists 68.7
Paralegals and legal assistants 66.5 33.5
Welders, cutters, and welder fitters 48.2 50.9

Making It Easier to Find Data on Pay and Benefits

We love data at the U.S. Bureau of Labor Statistics. We have lots of data about the labor market and economy, but we sometimes wish we had more. For example, we believe workers, businesses, and public policymakers would benefit if we had up-to-date information on employer-provided training. I recently wrote about the challenges of collecting good data on electronically mediated work, or what many people call “gig” work. I know many of you could make your own list of data you wish BLS had. One topic for which we have no shortage of data is pay and benefits. In fact, we have a dozen surveys or programs that provide information on compensation. We have so much data on compensation that it can be hard to decide which source is best for a particular purpose.

Where can you get pay data on the age, sex, or race of workers? Where should you go if you want pay data for teachers, nurses, accountants, or other occupations? What about if you want occupational pay data for a specific metro area? Or if you want occupational pay data for women and men separately? What if you want information on workers who receive medical insurance from their employers? Where can you find information on employers’ costs for employee benefits? Here’s a short video to get you started.

But wait, there’s more! To make it easier to figure out which source is right for your needs, we now have an interactive guide to all BLS data on pay, benefits, wages, earnings, and all the other terms we use to describe compensation. Let me explain what I mean by “interactive.” The guide lists 12 sources of compensation data and 32 key details about those data sources. 12 x 32 = a LOT of information! Having so much information in one place can feel overwhelming, so we created some features to let you choose what you want to see.

For example, the guide limits the display to three data sources at a time, rather than all 12. You can choose which sources you want to learn about from the menus at the top of the guide.Snippet of interactive guide on BLS compensation data.

If you want to learn about one of the 32 key details across all 12 data sources, just press or click that characteristic in the left column. For example, if you choose “Measures available by occupation?” a new window will open on your screen to describe the pay data available from each source on workers’ occupations.

There are links near the bottom of the guide to help you find where to go if you want even more information about each data source.

Check out our overview of statistics on pay and benefits. The first paragraph on that page has a link to the interactive guide. We often like to say, “We’ve got a stat for that!” When it comes to pay and benefits, we have lots of stats for that. Let us know how you like this new interactive guide.

Why This Counts: Breaking Down Multifactor Productivity

Productivity measures tell us how much better we are at using available resources today compared to years past. All of us probably think about our own productivity levels every day, either in the workplace or at home. I find my own productivity is best in the morning, right after that first cup of coffee!

On a larger scale, here at the U.S. Bureau of Labor Statistics, we produce two types of productivity measures: labor productivity and multifactor productivity, which we will call “MFP” for short. An earlier Why This Counts blog post focused on labor productivity and its impact on our lives. In this blog we will focus on why MFP measures matter to you.

Why do we need two types of productivity measures?

Labor productivity compares the amount of goods and services produced—what we call output—to the number of labor hours used to produce those goods and services.

Multifactor productivity differs from labor productivity by comparing output not just to hours worked, but to a combination of inputs.

What are these combined inputs?

For any given industry, the combined inputs include labor, capital, energy, materials, and purchased services. MFP tells us how much more output can be produced without increasing any of these inputs. The more efficiently an industry uses its combination of inputs to create output, the faster MFP will grow. MFP gives us a broader understanding of how we are all able to do more with less.

Does MFP tell us anything about the impact of technology?

It does. But we cannot untangle the impact of technology from other factors. MFP describes the growth in output that is not a result of using more of the inputs that we can measure. In other words, MFP represents what is left, the sources of growth that we cannot measure. These include not just technology improvements but also changes in factors such as management practices and the scale or organization of production. Put simply, MFP uses what we do know to learn more about what we want to know.

What can MFP tell us about labor productivity?

Labor productivity goes up when output grows faster than hours. But what exactly causes output to grow faster than hours? Labor productivity can grow because workers have more capital or other inputs or their job skills have improved. Labor productivity also may grow because technology has advanced, management practices have improved, or there have been returns to scale or other unmeasured influences on production. MFP statistics help us capture these influences and measure their impact on labor productivity growth.

How are MFP statistics used?

We can identify the sources of economic growth by comparing MFP with the inputs of production. This is true for individual industries and the nation as a whole.

For example, a lot has been written about the decline of manufacturing in the United States. MFP increased between 1992 and 2004 by an average of 2.0 percent per year. In contrast, MFP declined from 2004 through 2016 by an average of 0.3 percent per year. A recently published article uses detailed industry data to analyze sources of this productivity slowdown.

MFP is a valuable tool for exploring historical growth patterns, setting policies, and charting the potential for future economic growth. Businesses, industry analysts, and government policymakers use MFP statistics to make better decisions.

Where can I go to learn more?

Check out the most recent annual news release to see the data firsthand!

If you have a specific question, you might find it answered in our Frequently Asked Questions. Or you can always contact MFP staff through email or call (202) 691-5606.

Just like your own productivity at work and at home, the productivity growth of our nation can lead to improvements in the standard of living and the economic well-being of the country. Productivity is an important economic indicator that is often overlooked. We hope this blog has helped you to learn more about the value of the MFP!

BLS Measures Electronically Mediated Work

Are you a ride-share driver using a mobile app (like Uber or Lyft) to find customers? Maybe you do household chores or yardwork for others by finding short-term jobs through a website (such as TaskRabbit or Handy) that arranges the payment for your work. Or perhaps you perform online tasks, like taking surveys or adding descriptive keywords to photos or documents through a platform (like Amazon Mechanical Turk or Clickworker). If so, you are an electronically mediated worker. That’s a term BLS uses to identify people who do short jobs or tasks they find through websites or mobile apps that connect them with customers and arrange payment for the tasks. Have you ever wondered how many people do this kind of work?

BLS decided to find out. In the May 2017 Contingent Worker Supplement to the Current Population Survey, we asked people four new questions designed to measure electronically mediated employment.

Measuring electronically mediated work is difficult

After studying respondents’ answers to the new questions and other information we collected about them, we realized the new questions didn’t work as intended. Most people who responded “yes” to the questions clearly had not found their work through a website or app. For example, a vice president of a major bank, a local police officer, and a surgeon at a large hospital all said they had done electronically mediated work on their main job. Many people seemed to think we were asking whether they used a computer or mobile app on their job. That could apply to many jobs that aren’t electronically mediated.

But it wasn’t all for naught. After extensive evaluation, we concluded we could use the other information in the survey about respondents’ jobs to identify and recode erroneous answers. That allowed us to produce meaningful estimates of electronically mediated employment.

So, who does electronically mediated work?

Based on our recoded data, we found that 1.6 million people did electronically mediated work in May 2017. These workers accounted for 1.0 percent of total employment. Compared with workers overall, electronically mediated workers were more likely to be ages 25 to 54 and less likely to be age 55 or older. Electronically mediated workers also were slightly more likely to be Black, and slightly less likely to be White, than workers in general. In addition, electronically mediated workers were more likely than workers overall to work part time (28 percent versus 18 percent).

Workers in the transportation and utilities industry were the most likely to have done electronically mediated work, with 5 percent of workers in this industry having done such work. Self-employed workers were more likely than wage and salary workers to do electronically mediated work (4 percent versus 1 percent).

What’s next?

We currently don’t have plans to collect information on electronically mediated work again. And even if we did, we wouldn’t want to use the same four questions. At the least, we would need to substantially revise the questions so they are easier for people to understand and answer correctly.

Taking a broader look, we are working with the Committee on National Statistics to learn more about what we should measure if we field the survey again. The committee is a federally supported independent organization whose mission is to improve the statistical methods and information on which public policies are based.

How can I get more information?

The data are available on our website, along with an article that details how we developed the questions, evaluated the responses, recoded erroneous answers, and analyzed the final estimates.

If you have a specific question, you might find it in our Frequently Asked Questions. Or you can contact our staff.