Tag Archives: Interviews

How We Collect Data When People Don’t Answer the Phone

I was asked recently how the U.S. Bureau of Labor Statistics can collect data these days when no one answers the telephone. A legitimate question and one we grapple with all the time. I had two answers – one related to data collection methods and one related to sources of data. I will elaborate here about both.

Beige wall phone with rotary dial

But first, do you remember the days before caller ID, when everyone answered the phone? If you were at home, the rotary phone, permanently attached to the kitchen wall, always rang during dinner.

If you were in the office, the phone probably had a row of clear plastic buttons at the bottom that would light up and flash. In either case, who was on the other end of the phone was a mystery until you answered. In those days, your friendly BLS caller could easily get through to you and ask for information.

Vintage office phone with rows of buttons

Fast forward to today’s world of smart phones and other mobile devices. Nobody talks on the phone anymore. Many phone calls are nuisances. A call from BLS might show up as Unknown Number, U.S. Government, or U.S. Department of Labor on your caller ID, or identified as potential spam. With the spread of “spoofing,” many people do not answer calls from numbers they don’t recognize. How do we get around these issues?

Data Collection

At BLS, we consider data collection as much an art as a science. Sure, our staff needs to be well-versed in the information they are collecting. But they also need to be salespersons, able to convince busy people to spend a few minutes answering key questions. Part of that art is making a connection. There are old-fashioned ways that still work, such as sending a letter or showing up at the door. And there are more modern techniques, such as email and text. We are nothing if not persistent.

Our data-collection techniques have been called “High Touch, High Tech.” We start by building a relationship—the High Touch step. BLS has a wide range of information that people and businesses can use to help make informed decisions. We can help you access that information, and we love to see survey respondents use BLS data they helped us produce. In return, we ask for some information from you. There’s where High Tech comes in. We continue to add flexibility to our data-collection toolkit. You can provide information in person, on paper, or on the phone. You also can email information or an encrypted file. Or you can access our online portal anytime and anywhere to provide information or upload a data file. We need your information, and we want to make providing that information as easy as possible.

For example, this chart shows the number of employer self-reports that we’ve received through our online portal over the past several years. Internet data collection has really taken off.

Editor’s note: Data for this chart are available in the table below.

Another data-collection strategy we use is asking businesses to allow us to get the information we need from their website. This might involve web scraping data or using an Application Programming Interface (API). We have had success showing businesses that we can get what we need from their website, often eliminating the need for them to compile data.

Alternative Data

Beyond these data-collection strategies, we are expanding efforts to get information from alternative sources, lessening our need to contact businesses and households. Some BLS programs, such as Local Area Unemployment Statistics, the Quarterly Census of Employment and Wages, and Productivity Studies, rely heavily on administrative data and information from other surveys. In these cases, there is little need to contact businesses or people directly.

Other BLS programs, such as the Consumer Price Index (CPI) and the Employment Cost Index (ECI), need to capture timely information. But there are alternatives that can complement direct data collection. The CPI, for example, has produced an experimental price index for new vehicles based on a file of vehicle purchase transactions provided by J.D. Power. Using information from sources like that may eventually lessen the need to have BLS employees contact vehicle dealerships. The ECI found that it was easier to capture employer premiums for unemployment insurance from state tax records than to ask employers.

Alternative data come in many forms, from government records, data aggregators, scanners, crowdsourcing, corporate data files, and many more. BLS is investing heavily in alternative data-collection techniques and alternative data sources. The High Touch and High Tech approach we use every day in our data-collection operations helps us to maximize data quality and minimize respondent burden and cost.

The telephone may go the way of the dinosaur, but that’s not stopping us from using every tool at our disposal to continue to produce gold standard data to inform your decisions.

Number of transactions with BLS internet data collection
YearNumber of transactions

2004

105,145

2005

148,754

2006

219,923

2007

534,555

2008

972,605

2009

1,544,795

2010

1,909,410

2011

2,322,540

2012

2,769,694

2013

3,236,376

2014

3,288,665

2015

3,554,639

2016

4,013,415

2017

4,513,297

2018

4,685,414

2019

4,868,939

Tracking the Changing Nature of Work: the Process Continues

The days of working the same 9-to-5 job for 40 years are a fading memory. Work today may involve multiple part-time jobs, working from home, obtaining work through a mobile device, and changing jobs frequently. The so-called “changing nature of work” is already here, and at the U.S. Bureau of Labor Statistics we are trying to keep up with this new world.

One of our primary sources of information on Americans’ labor market activity is the Current Population Survey (CPS), a monthly survey of households that provides a real-time snapshot of the share of the population who are employed and unemployed. These data are complemented by other BLS programs that focus on labor turnover, how Americans spend their time, details about local labor markets, and other topics.

But how well do these programs track nontraditional forms of employment, including short-term assignments, platform work, temporary help, and jobs so new and different we haven’t even named them yet? BLS has been working on these issues for many years. Let’s consider a few timely questions and see how BLS has responded.

Not all jobs are permanent. What do we know about jobs that are not expected to last?

Throughout its history, BLS has been exploring perceived changes in the nature of work. For example, an article in the October 1996 Monthly Labor Review described “…reports of corporate downsizing, production streamlining, and increasing use of temporary workers…” as raising questions about “…employers’ commitment to long term, stable employment relationships.” This article, and many others in the same issue, went on to introduce the first “Contingent Worker Supplement” (CWS) to the CPS. Supplements such as this are additional questions on specific topics generally asked once (as opposed to every month) of CPS households.

The CWS asks about jobs that are not expected to last, as well as alternative work arrangements, such as working as an independent contractor or through a temporary help agency. While not an ongoing BLS program, we received funding to conduct the supplement in 1995, 1997, 1999, 2001, 2005, and 2017. This allows us to track contingent work over time. In May 2017, there were 5.9 million contingent workers – those who did not expect their job to last. This represented 3.8 percent of the total employed. Twelve years earlier, a slightly higher percentage, 4.1 percent, did not expect their job to last.

Percent of employed in contingent jobs

Editor’s note: Data for this chart are available in the table below.

How many people are in different types of jobs, such as independent contractors?

The CWS also included questions to identify people who were in four types of alternative work arrangements:

  • Independent contractors
  • On-call workers
  • Temporary help agency workers
  • Workers provided by contract firms

The most prevalent of these arrangements was independent contractors. The 10.6 million independent contractors identified in May 2017 represented 6.9 percent of the total employed.

Percent of employed in alternative arrangements

Editor’s note: Data for this chart are available in the table below.

Does BLS have a measure of the “gig” economy?

BLS does not have a definition of the gig economy or gig workers. In fact, researchers use many different definitions when they talk about the gig economy. You may think of a gig as something your high school band played on a Saturday night. Or today you might consider your ride-share driver as performing a gig. Classifying workers as gig could get very confusing. For example:

  • A plumber or electrician may be on the payroll of a contracting company on the weekdays and obtain individual jobs through an app on the weekend. Gig worker?
  • A substitute teacher in one school district may obtain assignments and pay through traditional means, while the neighboring district assigns and pays workers through an app. Is one a gig worker?

Confused? So am I. To repeat, BLS does not have a definition of gig. Definitions developed by others may overlap with contingent workers and some of those in alternative employment arrangements in the CWS. Rather than try to develop such a definition, BLS chose to focus new questions narrowly, as you will see in the next section.

What about work obtained through an app?

In preparing for the 2017 CWS, and knowing the interest in work obtained through an app on a phone or other mobile device, BLS added four questions about short jobs or tasks that workers find through an app or website that both links them with customers and arranges payment. Separate questions asked about in-person work (such as driving for a ride-sharing company or providing dog-walking services) and online-only work (such as coding medical records). At BLS, we call these jobs “electronically mediated employment.”

While BLS conducted some testing of the questions on electronically mediated employment and vetted them with a variety of stakeholders, the results made it clear that people had difficulty understanding the questions. This effort resulted in many false-positive answers, such as a surgeon who said all of his work was obtained through an app. BLS used companion information, where available, to recode responses. To be completely transparent, BLS published both the original and recoded data, but we encourage data users to focus on the recoded information. These results indicate that 1 percent of the employed in May 2017 – about 1.6 million people – held electronically mediated jobs. A slightly higher number of workers (990,000) held in-person jobs than online-only jobs (701,000). Note that some workers indicated they had both types of jobs.

Compared with workers overall, electronically mediated workers were more likely to be ages 25 to 54 and less likely to be age 55 and older.

Percent distribution of workers by age, May 2017

Editor’s note: Data for this chart are available in the table below.

Maybe these “app” jobs are a second job. Do we know how many people hold more than one job?

We get information from the CPS each month on the number of workers who hold more than one job. In 2018, there were 7.8 million multiple jobholders – about 5.0 percent of total employment in 2018. That’s around the same share of employment it has been since 2010, but it was below the rates recorded during the mid-1990s, which were above 6.0 percent.

With all these new types of work, is the BLS monthly employment information missing anyone?

As noted, the CPS is an authoritative source of labor market information and has provided consistent data for over three-quarters of a century. But BLS is always looking to improve its measures, and there are other data sources that can supplement the CPS. For example, the American Time Use Survey obtains information about an individual’s activities during a 24-hour period. Among the categories that may be identified are “income-generating activities,” such as making pottery for pay, playing in a band for pay, and mowing lawns for pay.

Recently, BLS looked at people who were not counted as “employed” but who participate in income-generating activities. The research suggested that between 657,000 and 4.6 million people participated in income-generating activities but were not otherwise counted as employed in the survey. Given that total employment is around 155 million Americans, this undercount ranges from 0.4 to 3.0 percent of the total.

The study also examined the extent that employed people who did informal work in addition to a regular job might not be correctly classified as multiple jobholders. The research found that reclassifying workers misclassified as single jobholders would increase the number of multiple jobholders somewhere between 3.0 percent and 20.7 percent.

What more is BLS doing to improve labor market measures?

So, yes, BLS is doing a lot to improve our labor market measures, and the work continues. We know there is likely a small number of people who are not counted as employed yet perform income-generating activities. We know that definitions and concepts may need to be updated from time to time. We know that some terms, like “gig,” are not well defined and mean different things to different people. And we know it is not easy to define or identify electronically mediated employment.

Given all this, we continue to move forward. BLS has contracted with the Committee on National Statistics, part of the National Academies of Sciences, Engineering, and Medicine, to convene an expert panel to address these issues and provide recommendations to BLS. This work began in late 2018 with a report due in early 2020. BLS will review the recommendations and, resources permitting, develop plans to test any new concepts or questions.

There’s been interest in emerging types of work for many years. It’s also a moving target, as the “changing nature of work” keeps changing. BLS has provided gold-standard data on America’s labor force for many years and will continue to research and refine and improve.

Percent of employed in contingent jobs
Year Percent of employed
February 1995 4.9%
February 1997 4.4
February 1999 4.3
February 2001 4.0
February 2005 4.1
May 2017 3.8
Percent of employed in alternative arrangements
Alternative arrangement May 2017 February 2005 February 2001 February 1999 February 1997 February 1995
Independent contractors 6.9% 7.4% 6.4% 6.3% 6.7% 6.7%
On-call workers 1.7 1.8 1.6 1.5 1.6 1.7
Temporary help agency workers 0.9 0.9 0.9 0.9 1.0 1.0
Workers provided by contract firms 0.6 0.6 0.5 0.6 0.6 0.5
Percent distribution of workers by age, May 2017
Workers 16 to 24 years 25 to 54 years 55 years and older
Total employed 12.4% 64.4% 23.1%
Workers with electronically mediated jobs 10.3 71.2 18.5
Electronically mediated jobs, in-person work 7.4 72.5 20.1
Electronically mediated jobs, online work 15.7 69.6 14.8

Why This Counts: Tracking Workers over Time

In many ways, BLS is very much about the now. For example, two of our major statistical programs are the Current Employment Statistics and the Current Population Survey. But to understand the U.S. labor market, we also need a longer-term focus.

The National Longitudinal Surveys (NLS) program provides information about the long-term workings of the economy.

What is a “longitudinal survey”?

A longitudinal survey interviews the same sample of people over time. At each interview, the surveys ask people about their lives and changes since their prior interview. With this information we create histories that allow researchers to answer questions about long-term labor market outcomes. For example, how many jobs do people hold over their lifetimes? How do earnings grow at different stages of workers’ careers? How do events that happened when a person was in high school affect labor market success as an adult?

How does the NLS work?

The NLS program is more than 50 years old, and today we have two active cohorts, or nationally representative samples of people, whom we interview every year or two:

  • The National Longitudinal Survey of Youth 1979 (NLSY79) consists of people born from 1957 to 1964, who were ages 14 to 22 when first interviewed in 1979.
    • The NLSY79 cohort has been interviewed 27 times since the late 1970s.
    • The children of the women in this sample (captured in the NLSY79 Children and Young Adults survey) have been assessed and interviewed 16 times since 1986.
  • The National Longitudinal Survey of Youth 1997 (NLSY97) consists of people born in the years 1980 to 1984, who were ages 12 to 17 when first interviewed in 1997.
    • The NLSY97 cohort has been interviewed 17 times.

These surveys are voluntary, and what a commitment our participants have shown! A big “thank you” to our respondents for their help!

What information is available from NLS?

By gathering detailed labor market information over time, researchers can create measures that are not available in other surveys.

One measure is the number of jobs held across various ages. The chart that follows is from the most recent NLSY79 news release.

  • The chart shows the cumulative number of jobs held from ages 18 to 50.
  • People born from 1957 to 1964 held an average of 11.9 jobs from ages 18 to 50. From ages 18 to 24 these baby boomers held an average of 5.5 jobs. The number steadily fell over time until these baby boomers held an average of just 0.8 job from ages 45 to 50.
  • The decline in the slope of the curves shows that workers change jobs more often when they are younger.

Cumulative number of jobs held from ages 18 to 50, by sex and age

Editor’s note: Data for this chart are available in the table below.

The decline in the number of jobs held over time is also true for the NLSY97 cohort.

A second measure available from the surveys is the percentage of weeks worked over various ages. Let’s look at data from the most recent NLSY97 news release.

  • The chart below shows the percent of weeks worked from ages 18 to 30, by educational attainment and sex.
  • Women with less than a high school diploma were employed an average of 40 percent of weeks from ages 18 to 30. Men with less than a high school diploma were employed 64 percent of weeks.
  • Among people with a bachelor’s degree and higher, women were employed an average of 80 percent of weeks, while men were employed 78 percent of weeks.

Percent of weeks employed from ages 18 to 30, by educational attainment and sex

Editor’s note: Data for this chart are available in the table below.

Who uses the NLS?

The main users of these data are researchers in academia, think tanks, and government. They use the surveys to examine how life experiences are connected. For example, how do early life events (schooling, employment during one’s teens, parental divorce) affect adult outcomes (employment, income, family stability)?

“Studies using the NLS cover a staggeringly broad array of topics. Looking through them, I was startled to realize how much of what we know about the labor market is only knowable because of the NLS.” — Janet Currie, Henry Putnam Professor of Economics and Public Affairs, Princeton University

Researchers value the surveys’ combination of large samples, long histories, and range of topics. These features allow researchers to study our economy and society from a rare and complex perspective.

Researchers have used the data in thousands of journal articles, working papers, Ph.D. dissertations, and books that shape theory and knowledge in economics, sociology, education, psychology, health sciences, and other fields.

You can find information about more than 8,000 studies in the NLS Bibliography. Looking at journal articles published in 2018, I found these studies using NLS data:

  • Racial and Ethnic Variation in the Relationship between Student Loan Debt and the Transition to First Birth
  • The Impact of Childhood Neighborhood Disadvantage on Adult Joblessness and Income
  • The Effect of an Early Career Recession on Schooling and Lifetime Welfare
  • The Early Origins of Birth Order Differences in Children’s Outcomes and Parental Behavior
  • Earnings Dynamics: The Role of Education Throughout a Worker’s Career

“[From the NLS] I learned that we cannot understand why adults have such diverse employment and earnings trajectories without going back to their youth to understand the skill and background differences that shaped how they transitioned into adulthood.” — Derek Neal, Professor of Economics, University of Chicago

How can I get more information?

The data are free to the public and provided online with search and extraction tools and detailed documentation.

If you have a specific question, you might find it answered in our Frequently Asked Questions. Or you can always contact the staff by email or phone at 202-691-7410.

If you care about the long view—how peoples’ careers evolve over time, how people fare after job loss, how childbirth affects women’s careers, and so on—the National Longitudinal Surveys may be just what you need! Check out these gold-standard data!

Cumulative number of jobs held from ages 18 to 50, by sex and age
Age Men Women
18 1.6 1.5
19 2.4 2.3
20 3.1 2.9
21 3.8 3.5
22 4.5 4.2
23 5.1 4.7
24 5.7 5.3
25 6.2 5.7
26 6.7 6.2
27 7.2 6.6
28 7.6 7.0
29 8.0 7.3
30 8.3 7.6
31 8.6 7.9
32 8.9 8.2
33 9.2 8.5
34 9.5 8.8
35 9.7 9.0
36 10.0 9.3
37 10.2 9.5
38 10.4 9.8
39 10.5 10.0
40 10.7 10.1
41 10.9 10.3
42 11.0 10.5
43 11.2 10.6
44 11.4 10.8
45 11.5 11.0
46 11.6 11.1
47 11.7 11.3
48 11.9 11.4
49 12.0 11.5
50 12.1 11.6
Percent of weeks employed from ages 18 to 30, by educational attainment and sex
Education Men Women
Less than a high school diploma 63.5% 40.3%
High school graduates, no college 75.5 64.4
Some college or associate degree 79.4 72.0
Bachelor’s degree and higher 78.4 80.1

Ensuring Gold-Standard Data in the Eye of a Storm

“Hurricanes Harvey, Irma and Maria were the most notable storms of 2017, leaving paths of death and destruction in their wake.”
Colorado State University’s Tropical Meteorology Project 2017 summary report

Colorado State University’s Tropical Meteorology Project is forecasting the 2018 hurricane season activity (as of May 31) to be average, with 13 named storms, 6 hurricanes, and 2 major hurricanes expected. Is BLS ready?

How does BLS deal with hurricanes?

Since June starts hurricane season, we want to share with you one example of how last year’s storms affected our data. We present a case study using our national employment survey, the Current Employment Statistics program. This program provides monthly estimates we publish in The Employment Situation—sometimes called the “jobs report.”

We have procedures to address natural disasters. We highlight some of our challenges and how we address them. We do everything possible to provide you with gold-standard data to help you make smart decisions!

2017 Hurricane Destruction

Two major hurricanes—Harvey and Irma—blasted the U.S. mainland in August and September 2017. Hurricane Maria devastated Puerto Rico and the U.S. Virgin Islands later in September.

  • Harvey first made landfall in Texas on August 25. The Federal Emergency Management Agency (FEMA) declared 39 Texas counties eligible for federal disaster assistance after Harvey. Harvey also caused heavy damage in Louisiana.
  • Irma hit the Florida Keys on September 10 and then later hit Florida’s southern coast. FEMA declared 48 Florida counties eligible for federal disaster assistance. Before Irma hit the lower Florida Keys, the hurricane already had caused severe damage in St. Thomas and St. John in the U.S. Virgin Islands and in Puerto Rico.
  • Hurricane Maria made landfall in St. Croix in the U.S. Virgin Islands and in Puerto Rico on Wednesday, September 20, causing catastrophic damage. These areas already had suffered damage from Hurricane Irma earlier in the month.

Some things to know about the monthly employment survey

The monthly employment survey is a sample of nonfarm businesses and government agencies. The reference period is the pay period that includes the 12th of the month. The sample has just over 23,000 active reporting units in the disaster areas, representing about 6 percent of the entire active sample.

What does it mean to be employed? If the employer pays someone for any part of the reference pay period, that person is counted as employed.

How did BLS collect data in these disaster areas?

Our biggest challenge is to collect representative sample data so we publish high-quality estimates. In the “old days,” the survey was a mail survey (yes, I mean snail mail), but no more! Now we collect data electronically by several different methods. These are the most common:

  • About half the total sample uses electronic data interchange. That’s a centralized electronic data reporting system for multi-establishment firms. The firm provides an electronic file directly from their payroll system to BLS for all establishments included in the report. Most of the firms reporting are outside of the hurricane-affected areas, although they may report on establishments within the affected areas.
  • About 23 percent of establishments use computer-assisted telephone interviews.
  • Another 16 percent report using our Internet Data Collection Facility.

Using these methods, we were able to collect data from most sampled businesses in these areas using normal procedures.

What about the emergency workers working in the disaster areas? How are they counted?

  • We count emergency workers where their employer is located, not where they are working.
  • We don’t count volunteers as employed because they are not paid.
  • Activated National Guard troops are considered active duty military and are outside the scope of the survey.

Did the estimation procedures change?

Once we collect the data from businesses in the affected areas, we consider whether we need to change our estimation procedures to adjust for missing data. The survey staff determined that we didn’t need to change our methods because the collection rates in the affected areas were within normal ranges.

How did the hurricanes affect national employment data for September 2017?

Hurricanes Harvey and Irma reduced the estimate of national payroll employment for September 2017. We can’t measure the effects precisely because the survey is not designed to isolate the effects of catastrophic events. National nonfarm employment changed little (+14,000) in September 2017, after increasing by an average of 189,000 per month over the prior 12 months. A steep employment decline in food services and drinking places and below-trend growth in some industries likely reflected the impact of Hurricanes Harvey and Irma.

What about Puerto Rico and the U.S. Virgin Islands?

National nonfarm employment estimates do not include Puerto Rico or the U.S. Virgin Islands.

Because of the devastation caused by Hurricanes Irma and Maria, Puerto Rico and the U.S. Virgin Islands could not conduct normal data collection for their establishment surveys. The September estimates for Puerto Rico and the Virgin Islands were delayed. The October and November estimates for the Virgin Islands also were delayed. Puerto Rico and the Virgin Islands eventually were able to produce estimates for September, October, and November 2017.

Want more information?

For more information on the impact of Harvey, Irma, and Maria, check out these pages:

What else does BLS know about hurricanes?

The Quarterly Census of Employment and Wages produces maps of businesses and employment in flood zones for states on the Atlantic and Gulf Coasts that are vulnerable to hurricanes and tropical storm. You can read more about those maps in another recent blog.

We hope the 2018 hurricane season won’t bring the loss of life and destruction of property that we saw in 2017. Regardless of what the season brings, BLS will be ready to continue providing gold-standard data about the labor market and economy.

Reaching out to Stakeholders—and Steakholders—in Philadelphia

The U.S. Bureau of Labor Statistics has staff around the country who serve several critical roles:

  • Contacting employers and households to collect the vital economic information published by BLS
  • Working with partners in the states who also collect and review economic data
  • Analyzing and publishing regional, state, and local data and providing information to a wide variety of stakeholders

To expand the network of local stakeholders who are familiar with and use BLS data to help make good decisions, the BLS regional offices sponsor periodic Data User Conferences. The BLS office in Philadelphia recently held such an event, hosted by the Federal Reserve Bank of Philadelphia.

These Data User Conferences typically bring together experts from several broad topic areas. In Philadelphia, participants heard about trends in productivity measures; a mash-up of information on a single occupation—truck drivers—that shows the range of data available (pay and benefits, occupational requirements, and workplace safety); and an analysis of declines in labor force participation.

Typically, these events provide a mix of national and local data and try to include some timely local information. The Philadelphia conference included references to the recent Super Bowl victory by the Philadelphia Eagles and showed how to use the Consumer Price Index inflation calculator to compare buying power between 1960 (the last time the Eagles won the NFL Championship) and today.

We also tried to develop a cheesesteak index, a Philadelphia staple. Using data from the February 2018 Consumer Price Index, we can find the change in the price of cheesesteak ingredients over the past year.

Ingredient Change in Consumer Price Index, February 2017 to February 2018
White bread 2.5 percent decrease
Beef and veal 2.1 percent increase
Fresh vegetables 2.1 percent increase
Cheese and related products 0.8 percent decrease

Image of a Philadelphia cheesesteak

These data are for the nation as a whole and are available monthly. Consumer price data are also available for many metropolitan areas, including Philadelphia. These local data are typically available every other month and do not provide as much detail as the national data.

While the Data User Conferences focus on providing information, we also remind attendees the information is only available thanks to the voluntary cooperation of employers and households. The people who attend the conferences can help us produce gold standard data by cooperating with our data-collection efforts. In return we remind them we always have “live” economists available in their local BLS information office to answer questions by phone or email or help them find data quickly.

Although yet another Nor’easter storm was approaching, the recent Philadelphia Data User Conference included an enthusiastic audience who asked good questions and left with a greater understanding of BLS statistics. The next stop on the Data User Conference tour is Atlanta, later this year. Keep an eye on the BLS Southeast Regional Office webpage for more information.