Startups are hiring low-paid gig workers to train robots on manual labour

Earlier this year, a woman in Nigeria named Omolara was commiserating with a friend about finding a job. At 25, she held a university degree and lived in a region with a growing manufacturing base, but steady work was elusive.

Her friend told her about an app called Atlas Capture. “Earn cash completing quick video tasks from your phone,” read the description in the app store. All she had to do was mount an iPhone to her head and record herself doing things around the house. She was skeptical. Why would anyone pay her for that?

But she signed up, purchased a head strap and worked in four-hour stretches recording whatever tasks Atlas Capture required. She mopped the floor, cleaned the bathtub and picked weeds. She found it stressful – she could not sit down when recording and had to ensure both hands were visible – and some jobs required more footage than seemed practical. You cannot clean a toilet for 20 minutes, she told me.

The best strategy, she found, was to record in the morning and upload videos overnight since her internet connection was so slow. By the end of the week, she had recorded 17 hours of video, for which Atlas Capture paid her US$50, about US$2.94 per hour, issued in cryptocurrency pegged to the U.S. dollar. It was not a lot of money, she said, but at least it was something. And besides, she didn’t have many other options.

Omolara has been relying on Atlas Capture and similar apps since then to earn money while looking for a job. (Omolara is a pseudonym. The Globe and Mail is not identifying her or other users by name because they are afraid to risk getting cut off from these apps.)

On its website, Atlas Capture says it has a million users, but there’s no information about who runs the company or where it’s based. Corporate records indicate it was registered in Wyoming late last year, while the website says complaints can be directed to the Privacy Commissioner of Canada.

There was a clue on Apple’s app store, however. Until very recently, the copyright holder was listed as Carbon Based Technology Corp., the registered name of a Canadian company doing business as Mecka AI. Founded by a group of guys not long out of university, Mecka has raised more than US$120-million, including a US$60-million round announced earlier this week, led by prominent U.S. venture firm Sequoia Capital. Nvidia is a backer, too. The company is projecting a US$300-million revenue run rate by the end of year, despite only incorporating in 2025.

Mecka is part of a wave of companies providing data to perfect general-purpose robots to work in factories and warehouses, and vacuum floors and load dishwashers in homes. In the past few years, researchers have found that first-person or egocentric video is a crucial ingredient for the AI models that serve as brains for these robots. Just as the large language models behind chatbots are trying to ingest the sum total of humanity’s written and visual output, AI models for robots need to understand how we move and interact with the world.

“What we truly want to do is learn from the physical intelligence of humans,” said Danfei Xu, an assistant professor at Georgia Tech and an Nvidia researcher who has collaborated with Mecka.

The bottleneck is data, and that’s where Mecka comes in. The company, which declined interviews, amasses huge amounts of first-person video for customers such as AI and robotics companies, and has also developed its own sensor hardware. It’s far from the only one. There is Contracted AI, Claru and the aptly named Human Motion Data. Scale AI, in which Meta Platforms Inc. took a 49-per-cent stake last year, collects data for robotics, while humanoid robot company Figure AI unveiled its own video collection pipeline in August. Some outfits, including Atlas Capture, partner with businesses such as restaurants, cleaning firms and skilled trades to capture the movements of workers on the job, too.

These efforts are a sign that humanoid robots are no longer relegated to science fiction. Auto manufacturers such as BMW Group are experimenting with bipedal, five-fingered mechanoids on the factory floor, while Tesla Inc. is preparing to ramp up production of its Optimus humanoid. 1X Technologies, a California-based company and Mecka customer, plans to send its US$20,000 domestic androids into customers’ homes by the end of the year, though human operators will remotely take control when necessary.

The demand for AI-powered robots is potentially massive. RBC Capital Markets estimates the global humanoid robot market could hit US$9-trillion by 2050, with early adoption starting in warehouses and factories to perform jobs that don’t require much dexterity. Household robots could take more than 20 years to hit the mainstream, according to the report, in part because the ability to complete so many tasks autonomously in different environments will take more time to perfect.

Even if this estimate proves wildly optimistic, we are getting closer to a time when robots start to free us from domestic drudgery, work alongside us and, perhaps, put some of us out of work. Should this future arrive, it will not only be due to the bright minds in robotics labs but to a large, unseen population of contributors from low-income countries – including, from what I can tell, a lot of moms in the Philippines.

The idea that robots can learn from observing us is an old one, but it hasn’t been easy to pull off. One method of teaching robots is teleoperation. A human operator controls a robotic arm rig to perform any number of jobs, like folding a shirt, which generates high-quality training data.

But teleoperation quickly becomes expensive and impractical. “You need to train people to do it. You need to buy a bunch of robots. You need to find space for these robots. It’s actually quite difficult to scale,” said Philipp Wu, the chief executive officer of XDOF, a robotics data startup in California.

Learning from video can complement teleoperation, albeit with at least one major problem: You need lots – and lots – of first-person footage. Years ago, companies and labs started assembling video datasets for AI research. Meta, for one, partnered with more than a dozen universities to build Ego4D in 2021, which consists of 3,670 hours of egocentric footage from 931 contributors. People filmed with head-mounted cameras as they walked their dogs, chatted with friends, played Connect 4 and read books. “The data will allow AI to learn from daily life experiences around the world – seeing what we see and hearing what we hear,” the authors wrote.

The following year, researchers from Stanford University and Meta used Ego4D as the basis for an AI system that powers a robotic arm, which proved to be more successful at tasks such as folding towels and closing drawers than other mechanical rigs. Additional studies have only cemented the importance of egocentric video. Earlier this year, Nvidia trained an AI model with 20,854 hours of video and found a clear relationship between quantity and performance.

In short, the more egocentric video used to train an AI model, the better the performance of the robot. This scaling law, as it’s called, gives researchers a better idea of the costs and amount of data necessary to improve performance, too.

When outlining what goes into these AI models, Mr. Wu refers to a pyramid. At the bottom is the vast array of egocentric video recorded with head-mounted cameras, which is cheap and abundant, but low-fidelity. Above that is a smaller amount of data for what’s called a “mid-training” step. That can include footage from lab settings where both humans and robots are recorded performing the same tasks, while the human can wear wrist cameras, motion sensors and hand-tracking gloves. Finally, companies can fine-tune with teleoperations data for specific tasks.

XDOF, which raised US$70-million this year, collects all manner of data, including from teleoperations centres, or “arm farms,” located in Southeast Asia. “You can probably think of it as a modern-day call centre,” Mr. Wu says. “As opposed to people sitting in a booth to call, you’re going into a booth and you’re controlling a robot.”

The amount of data companies require is growing fast. In 2024, San Francisco-based startup Physical Intelligence created an AI model trained on more than 10,000 hours of data. In August, California-based Dyna Robotics unveiled a model trained on more than one million hours of egocentric video. Companies are now targeting 10 million hours of video, Mr. Wu said. Played back, that much footage would run for 1,141 years.

The end goal is to build a model that can generalize, meaning it can learn new feats of prehension, manipulation and dexterity without massive amounts of fresh training material. “Once we have built the perfect general-purpose robot that has an innate learning algorithm similar to what humans can learn, we may not need companies collecting human data,” said Igor Gilitschenski, an assistant professor at the University of Toronto who studies AI and robotics. “It can easily be 20 more years, but who knows?”

For now, data is needed, and the proliferation of smartphones allows companies such as Mecka to collect it at an industrial scale. The company’s founders have an eclectic mix of backgrounds. CEO Josh Gao studied at the Ivey School of Business, and a 2021 article on the university’s website describes him as a serial entrepreneur who had started about a dozen ventures, including a clothing brand. At the time, he’d launched a new company providing digital solutions for restaurants, alongside fellow Mecka co-founder Mogen Cheng. University of British Columbia grad Jason Chong started a crypto payment company that was sold to Coinbase, while Duy Nguyen was a sneakerhead who made millions flipping shoes, according to Fortune.

In a video posted in June, Mr. Gao explained the company’s name comes from the term “mecha,” a sci-fi concept wherein a human pilots a giant robot. “The idea being technology as an extension of the human, making people far more capable,” he said.

We humans are the ones making the robots capable for now, and I wanted to do my part.

Earlier this summer, I downloaded Atlas Capture. I purchased a headband and poked around online while waiting for it to be delivered. I soon came across a video that someone had recorded for the platform. The individual had filmed themselves inside of a grocery store gingerly removing boxes of tea from a shelf and lining them up in a shopping cart. Their arms protruded straight out, and they moved stiffly, like – well, a robot. Here was a human acting like a robot in order to teach a robot how to act like a human.

This was not an ironic gag, but more of a necessity. Atlas Capture requires that the recorder’s hands remain visible, or else the video could be rejected and the recorder won’t be paid for that footage.

The guidelines on the app informed me that I should film in a bright area, move around and never sit down. Nobody else could enter the frame, either. To start earning, I had to pass a test by cleaning my living room, so when my headband arrived, I strapped in and angled my iPhone just right. I put away Lego and vacuumed the rug and never had any idea if my hands were visible.

Upon review, they were not. On my second attempt, I moved carefully like a man with a bad back and stared at my hands so that my head swivelled with my body. I was moving anything but naturally, even though I had to check a box before uploading the video certifying that of course, I was moving naturally.

Once the video was approved, a series of jobs then unlocked on the app. I could make a sandwich, use the blender, change the bedsheets, wash the car, empty garbage cans and more. Later, a task appeared for volunteering at an “old age home.” The guidelines stipulated I had to ask staff for permission – before lumbering into a retirement facility and jerkily doling out food or playing cards with baffled senior citizens wondering about this stranger with a camera on his head – while trying not to record their faces or capture medication labels lest I violate anyone’s privacy.

Instead, I set the table. Atlas Capture had updated the app by then to emit a beeping noise when it did not detect hands in the frame. Every time my phone chirped, I jerked my arms randomly in hopes of silencing it. I also had to record for 15 minutes, which is a long time to be setting the table. I slowed down, stacked dishes on top of one another and liberally sprinkled cutlery until I had created an AI hallucination of a dinner table. A red welt soon blossomed on my forehead from the headstrap, but at least my video was approved a day or so later.

I was only moonlighting, of course. The backbone of this operation are people in lower-income countries, such as the Philippines, Indonesia and Vietnam, who do this out of financial necessity. Browsing Facebook groups for contributors, I noticed most of the members were women, often with pictures of children on their profiles – entirely unsurprising, given the typical division of domestic labour.

One woman, a 46-year-old who lives north of Manila, had seen a TikTok video promoting Atlas Capture. She had questions about AI (some people in the Philippines blame it for replacing jobs in the country’s call centre industry, she said), but it seemed like a flexible way to earn money on top of her job distributing health products. Since signing up, she’s recorded herself cleaning, organizing and sweeping outside; when passersby inquired about the iPhone on her head, she recommended the app. Her hourly rate worked out to about US$2.40, which she felt could be higher, but she didn’t have any plans to stop.

Another woman I spoke with, a 30-year-old mother of three who worked as an on-call golf caddy in the Philippines, recorded for about two or three hours a day, usually cooking or washing dishes, while making sure her kids didn’t run into the frame. She found it hard on her back to be standing for hours at a time with the weight of an iPhone on her head, though, and had decided to pause for a while. The volume of rejected footage was frustrating, too; she could be compensated for all 10 hours of video recorded in a week, or just five or six, she wrote in a Facebook message. But when I followed up a few weeks later, she had started recording again.

Ariel Genita, a 35-year-old who lives in the Philippines, said many of his recordings have been rejected, including for not depicting “meaningful work.” He feels “exhausted” when a video he has spent time recording is not approved, he wrote in an email, but “rules are rules.” So far, he had earned US$203 for around 71 hours of approved footage, which works out to US$2.86 an hour. He and other recorders I spoke with said that payments have been delayed at times. Still, he seemed content to be paid at all, even if he didn’t think the rate was entirely fair. He maxed out on the platform, in fact. Atlas Capture had taken to limiting users to 50 hours of footage for reasons unexplained.

Atlas Capture has had more formal arrangements with recorders, too. Tamara Sarastiti, a 29-year-old in Bali, spent four months working for the company. Each day, she’d head to a villa somewhere in the area and don an iPhone and wrist cameras to fold clothes, clean shoes, peel cucumbers and many more chores for up to eight hours a day. She worked under a supervisor, who monitored the ten or so other recorders in the villa, and earned US$4.50 an hour. “I didn’t have a job, so it was good for side income,” she said.

For videos to be more suitable for AI training, they need to be paired with written descriptions. Atlas Capture maintains a platform for labellers, most of whom seem to be located in Nigeria and Kenya, who write short descriptions of every action in every segment of every video, marrying language with movement.

James Joseph, a 28-year-old in Abuja, signed up a few months ago. He would come home from his job at a phone repair shop and log on to Atlas Capture in the evening, sometimes labelling videos until well after midnight, and start again before work in the morning. He could earn up to 50 cents for every annotated segment, which can run up to two minutes long. The job entails vexing judgement calls about when one movement ends and another begins. At what frame, for example, does someone stop merely holding a package of hot dogs and begin to unwrap it?

Annotators have to follow Atlas Capture’s guidelines, which Mr. Joseph said changed often, or else they can be penalized. At one point, annotators had to maintain consistency between nouns and verbs across segments. If someone described a “cardboard box” in one segment, they could not say “box” in the next. The company then reversed course on that rule. Annotators also have to decide how general or specific to be. Should that knife be described as cutting a zucchini? Or slicing? Or chopping? Mr. Joseph said there were weekly online classes, in part because the guidelines always seem to be evolving.

Every annotator earns a quality score, and mistakes damage their rating. If their score dips below 70 per cent, they won’t be compensated for their work. “You’re just working for free,” Mr. Joseph said. This was a change Atlas Capture appeared to make earlier this year. “Work that comes in below 70% won’t be compensated for now,” the company wrote in a Discord server for labellers, according to a screenshot obtained by The Globe. “As your accuracy climbs back to 70%, full payment resumes right away.”

The system forces trade-offs for annotators. Because they’re not paid by the hour, volume is key if they want to earn more. But speed brings the risk of mistakes.

When I spoke with another annotator, a man in Kenya, he told me that he was waiting for an easy assignment so he wouldn’t put his score at risk. He sent me a video, about a minute and 40 seconds long, that paid US$0.46 to label. In the clip, someone sits at a table surrounded by piles of cotton swabs and methodically counts them out before slipping them into a small plastic bag. Accounting for every movement – when the right hand picks up a swab, transfers it to the left, puts it on the table – wasn’t worth it, he had decided. So he waited for something more straightforward, like a video of window washing, where there are fewer steps involved.

A 37-year-old man in Nigeria, who has a job but signed up to Atlas Capture to help cover his bills, said that you would have to be a robot to label more than two videos in an hour. Electricity and internet connectivity can be spotty in his area, making it difficult to work for long stretches, not to mention the challenge of staying focused for hours at a time. The most he earned in a day is US$6. He felt the setup was unfair, and stopped working on Atlas Capture earlier this year.

A woman in Nigeria, meanwhile, told me she had been on the platform for months and had no other source of income. Trying to keep track of all of the movements in a video, knowing that mistakes would damage her rating, was overwhelming, she said.

Some annotators work faster than others. A university student in Nigeria told me he could label a minute-long video in about seven minutes, and said the job helped supplement his income while he is in school. The pay could be better, he said, but it’s a job for people who have no other options.

Atlas Capture did not respond to e-mails. Mecka did not make any of its founders available for interviews, despite multiple requests over several months. The company did not answer a detailed list of questions, including about its relationship with Atlas Capture.

“Mecka’s aim is to help AI learn about the physical world,” a spokesperson wrote over e-mail. “We partner with individuals and businesses in many countries who support this work and can earn supplemental income for it, which is on average ~2-5x the local minimum wage. Contributors are paid for all approved work, payments run on a regular schedule, and anyone with a question about a payment can reach support staff directly.” (The spokesperson declined to give their name.)

The gig may not be consistent, however. In August, Atlas Capture halted labelling. Instead, annotators were supposed to verify existing labels and correct mistakes. Each clip, which runs a few seconds, paid US$0.0067. According to screenshots from Discord, the company set a target for people to verify 1,640 segments a day with 100-per-cent accuracy. A mistake would earn a strike. “If you have 5 strikes, do not resume working on the Human Verifier, you will NOT be paid,” according to a message posted to the Discord.

But even this switch didn’t last. Soon after, Atlas Capture paused the project.

When I spoke to recorders and labellers for Atlas Capture, some would ask if I knew of any other platforms where they could also make money.

One recording app gaining traction was called Human Motion Data Recorder, run by Vivek Viswanathan. He lives in Connecticut, previously worked in finance and incorporated the company in March after speaking with a robotics company that needed 50,000 hours of egocentric video as soon as possible. He launched the app in April and by mid-summer had collected 38,000 hours of household chores from 3,900 contributors.

The company has a large user base in the Philippines and pays US$3 for an hour of approved footage. The Philippines, he explained, is well off enough for people to own quality smartphones, but not too well off. “It’s not so rich that if somebody said, ‘I’ll pay you $3 to record you doing chores,’ they’d be like, ‘I don’t want to do that,’” he said.

Houses in the Philippines also bear a close enough resemblance to houses in the west for these videos to be suitable training material for domestic robots. “If you look at our videos from Kenya and Nigeria,” he continued, “I wonder whether the robotics company is going to look at that and be like, this is just too far off.”

I asked whether there was something discomfiting about paying lower-income people a few dollars to train robots that will clean the homes of affluent families, when the benefits could take a very long time to spread to the developing world. Mr. Viswanathan didn’t see it that way. “They’re getting paid better than they would get paid otherwise,” he said. Technology only gets cheaper, he continued, so families in the Philippines will some day have a robot of their own. “They’ll be able to tell their kids, ‘Hey, I trained that robot.’”

Other companies are expanding beyond household chores to collect from commercial settings. Claru, for one, partners with businesses to record egocentric video from workers, in addition to household videos. The platform is the data collection arm of California-based AI company Reka, which merged with Canadian AI video startup Moonvalley in June. Claru grew out of Moonvalley’s data operations, said John Thomas, its Toronto-based general manager. “We had the thought that we’ve found so much value in this function, is there demand for other labs and companies to use these same services?” said Mr. Thomas, who also co-founded Moonvalley.

Claru partners with carpenters, baristas, restaurant workers and more. This kind of video is useful for robotics models because it depicts the tumult of everyday jobs, including cluttered spaces, safety hazards and time constraints, which cannot be captured in lab settings, according to the company.

Its website includes a calculator so that businesses can get a rough idea of how much they can earn. A skilled trades outfit in Asia or Africa can bring in up to US$6,000 a month by having five workers record eight hours a week, while restaurants, retailers and cleaners can earn up to US$3,400. Revenue is usually shared with the employees doing the recording, Mr. Thomas said.

Businesses that participate are typically located in Asia, Africa and South America, in part because of the regulatory environment compared to North America and Europe. “If you were to go to India and you work with a fast food restaurant there, it is less regulated,” he said. “The unit economics are a lot cheaper, so the data is a lot cheaper.”

Mr. Thomas was not able to connect me with any commercial partners, which he said were cautious about speaking publicly. After all, it would appear that workers are being compensated to train robots that could replace them in the future. “It’s definitely a concern that people have,” Mr. Thomas said. But he added that jobs are going to change as a result of robotics. “If this thing replaces burger-flipping, you will go on to manage the robots that are flipping the burgers.”

How general-purpose robots will affect employment is far from certain. But the capture of egocentric data today forces us to rethink the value of what we do.

Our hands, organs that robotics companies are trying to replicate, are wonderfully complex, with some 17,000 densely packed mechanoreceptors that give our fingertips a highly acute sense of touch. Greek philosophers have argued about them. Anaxagoras said we are more intelligent than animals because we have hands; Aristotle posited the opposite, that nature bestowed us with hands because of our superior intelligence.

Hands contain the origins of communication (we can point before we can speak) and their ability to perceive, touch and grasp helped us develop our sense of agency and selfhood, wrote British philosopher Raymond Tallis, drawing a link between the hand and humanity itself.

The hand is now also a commodity, but in a way that many of us have not considered before.

With skilled labour, what often matters is the finished product. A framing carpenter builds the skeleton of a house; a mason lays the bricks. What has value now is physical movement itself – how someone swings a hammer, where they grip the handle and all of the other motions stemming from years of experience.

“The physical articulation of their body is a big part of their intellectual property,” said Daniel Wigdor, co-founder of Axl, an AI-focused venture studio in Toronto. He argued there’s a large opportunity that labour unions are missing. He’s tried to convince some of them, in fact, that they should equip workers with cameras to record their movements – but not to sell off to a robotics company. Instead, he pitched unions on partnering with Axl to launch a startup that would build its own AI model, and ensure union members own equity.

These conversations have not been successful. Unions seem set on forestalling robotics to protect jobs, he said. The optics of a union making its own members participants in a plot to build robots that could replace them are, of course, not great. “I get where it’s coming from, but it’s so short-sighted,” Mr. Wigdor said.

Robots are not going to swarm job sites any time soon, he continued, and even then, he’s not convinced that would portend a labour apocalypse. Instead, jobs will change over time – the carpenter of the future will manage robot apprentices – and unions can help define that relationship, he said.

If he’s wrong, at least workers could have a stake in a company bringing about economic revolution. “I’m shouting from the mountaintop saying, ‘I’ll bring the money and we’ll give you all this wealth,’” he said. “I realize as I’m saying this that I probably sound exactly like Christopher Columbus.”

The pitch might sound more plausible when you remember that companies are already collecting this data to build better robots. The extraction of all that is human to feed AI might be inevitable. What’s left is how to ensure the spoils don’t only go to the handful of companies mining the human experience.

In Nigeria, Omolara started using another recording app, and was surprised at its hyper-specific guidelines, right down to the type of sponge she must use for cleaning. The app’s guidelines did not consider her broom to be a broom. Nigerian brooms do not look like Western brooms, she explained. They’re much shorter, and can be made from the stiff spines of palm fronds bound together at one end, which serves as a handle.

Her plan was to stick to cooking videos, recording as she peeled and sliced, something that, for now, only human hands can do.