By Lydia DePillis, CNN Business

(CNN) -- It was one of the most striking findings of 2016: The share of workers in "alternative" work arrangements, such as Uber drivers and other independent contractors, had risen dramatically over the previous decade.

The gig economy was exploding. The study from two well-known economists seemed to confirm that the traditional employment relationship was on the way out, to be replaced by micro-tasks delivered on demand.

Now the authors of that study — Alan Krueger of Princeton University and Larry Katz of Harvard — have walked back its conclusions. Their new study, released Monday, shows the challenge of figuring out how much Uber-type jobs have changed the landscape of American work.

Katz and Krueger's 2016 paper, based on data collected with the RAND Corporation, attempted to replicate the US Bureau of Labor Statistics' 2005 contingent worker supplement. At the time the Labor Department, the parent agency of BLS, didn't have the money to field a new survey itself.

The finding by Katz and Krueger was a blockbuster: They found that 15.8% of workers were in some kind of alternative work situation — from temporary help companies to the self-employed — up from 10.7% in 2005.

Soon after, fueled by questions about how much rideshare apps and other on-demand services were changing work, Congress found the money for an update to the 2005 survey. The new government survey found that the share of workers in alternative work arrangement had actually decreased, from 10.7% to 10.1% in 2017.

So how did Katz and Krueger go so wrong?

Their comprehensive re-evaluation chalks up the gap to differences in survey design that seemed trivial but turned out to matter a lot. For example, the sample of the BLS survey was much larger and conducted over the phone, rather than online, as the RAND survey was. Also, BLS allowed survey respondents to answer for other people in their own households, and they may not have realized the full extent of the side jobs held by their siblings, children or spouses.

After adjusting for all those differences, the economists concluded that gig work had increased by only one or two percentage points since 2005, rather than the more than 5 point increase they reported earlier.

The elusive definition of a gig job

But that's hardly the last word on this subject. Since Katz and Krueger began their re-evaluation, a steady stream of research has looked at various measures of gig employment.

Another reason it's hard to pin down the number of gig economy jobs: There's little agreement on what it actually means.

JPMorgan Chase, for example, used its own customers' data to show that the share of people making money from online platforms in any given month has been steadily rising, and now stands at 4.5%. The bank's methodology was able to capture jobs that aren't a person's primary source of income — which is how the original BLS survey looked at.

Another measure, the Federal Reserve's 2017 Survey of Household Economics and Decisionmaking, found that 31% of adults engaged in some form of gig work in the previous month, including offline and possibly off-the-books activities like babysitting or house cleaning.

Meanwhile, a team of economists from the University of Maryland and the US Census Bureau found a marked increase in the number of self-employed taxi cab drivers — but not other occupations — by looking at data from the IRS. Administrative records can be more precise than household surveys, because many people don't recognize the difference between a traditional job and independent contracting. To the worker answering a survey question, a job is a job.

Why this research matters

One thing is clear: With the possible exception of passenger transportation, the revolution that allows consumers to order anything on a smartphone hasn't translated yet into a dramatic shift in how most people make a living.

But even if the gig economy is really only an Uber phenomenon at the moment, the shift could spread quickly. That's why figuring this economic research is so important.

"The rapid and dramatic changes in the passenger transportation sector nonetheless highlight the potential for change," the authors from Census and University of Maryland wrote. "Adoption of new technologies often involves long and variable lags. An important objective for economic measurement will be to capture changes in the nature of work as they occur in the years to come."