Robots Depend on Us More Than We Rely on Them

In the future with AI, people, not the computer programs they use, will be the main factor for a company’s success

Glenn Burgess

Think about searching for a specific picture in the National Football League’s huge collection of videos. In just one season, they make over 16,320 minutes (which is around 272 hours) of game videos.

If you add all the stuff before and after games, like pregame, halftime, and postgame shows, along with practices and interviews, you end up with an incredibly large amount of video.

And remember, this is just for one season.

The NFL joined forces with Amazon Web Services to make it simpler for their staff to create highlight reels and other media from their vast video collection. Initially, the NFL’s content team had to teach the AI what to look for.

They assigned tags to things like players, teams, jerseys, stadiums, and other things they wanted to identify in their videos. Then, they combined these tags with Amazon’s image-recognition AI, which had already learned from millions of images.

With both sets of data, the AI could identify important visuals in the video library, and the content team could easily approve each tag with a few clicks.

In the past, employees had to manually search for, find, and clip each video, store it, and add metadata tags, but Amazon’s AI automated most of this process.

Some top organisations are challenging the traditional belief that technology will replace people. Instead, they are harnessing the potential of humans working alongside machines to change their businesses for the better and boost their profits.

A few companies are not just surpassing their rivals in innovation using this method; they are also shifting more firmly toward AI technology that prioritises humans, completely reshaping the way innovation has been done in the past decade.

In the NFL’s situation, AI sped up the process of recognising images, but it was essential for employees to decide what data to use and give their approval.

The NFL didn’t just entrust the task of creating highlight reels to AI alone; experts in content creation still carried out the job, albeit with greater speed and ease, thanks to AI’s exceptional capability to swiftly analyse huge amounts of data.

A new approach to AI, one centered on humans, is reshaping the way we think about innovation. Companies like Etsy, L.L.Bean, McDonald’s, and Ocado are redefining the relationship between AI and automation, seamlessly integrating various cutting-edge technologies to adapt to evolving customer needs and operational challenges.

These forward-thinking firms have significantly increased their investments in digital technologies, particularly cloud services and AI, outpacing slower companies in revenue growth, according to Accenture’s surveys.

We’ve distilled our research into the IDEAS framework, which focuses on five key elements of the evolving technology landscape: intelligence, data, expertise, architecture, and strategy.

This framework offers guidance for both technical and non-technical leaders, helping them understand these elements and how to combine them effectively for innovation.

In this article, we explore real-world examples of businesses that have successfully harnessed human-driven AI to solve problems in e-commerce, online grocery delivery, robotics, and more.

You can follow their lead, leveraging the skills and experience of your own team to drive technological innovation across various areas, from research and development to talent management and business model development.

Intelligence: Make AI More Human & Less Artificial

Human intelligence and artificial intelligence complement each other. Even the youngest humans outshine AI-powered machines in their ability to learn, understand, and contextualise information.

For instance, drop an object accidentally, and a one-year-old will try to hand it to you. Purposefully throw it, and the child will likely ignore it. This showcases a remarkable cognitive ability in very young children – the understanding of human intentions, which seems almost innate in the human brain.

Furthermore, from a very early age, children develop an intuitive understanding of physics. They anticipate that objects move smoothly, persist in existence, and fall when unsupported. Even before they can speak, they distinguish living beings from inanimate objects.

As they acquire language, they demonstrate an exceptional capacity to generalise from just a few examples, quickly picking up new words after hearing them only once or twice. They also learn to walk on their own through trial and error.

Conversely, AI can excel at tasks that are challenging or impossible for humans, despite our natural intelligence. It can identify patterns in vast datasets, defeat world-class chess champions, manage complex manufacturing processes, handle numerous customer service calls simultaneously, analyse weather conditions, soil quality, and satellite images to help farmers boost crop yields, scan millions of internet images to combat child exploitation, detect financial fraud, predict consumer preferences, customise advertising, and much more.

Most importantly, AI has facilitated efficient collaboration between humans and machines, creating new, high-value jobs instead of the automation-related job losses often feared.

For instance, at Obeta, a German electronics wholesaler using a warehouse managed by the Austrian company Knapp, human workers are teaching a new generation of robot pickers how to handle items of different sizes & textures.

These robots utilise industrial arms, suction grippers, and a vision system, all powered by AI software from Covariant, a California-based startup.

To train these robots, Knapp employees expose them to unfamiliar objects and assess their ability to adapt. When they fail, they learn from their mistakes and explore different approaches. Success is rewarded with positive feedback, programmed by humans, to reinforce their learning.

When dealing with a set of items that differs significantly from others, the process reverts to supervised learning, where new training data is collected and labeled.

Thanks to Covariant Brain software, Knapp’s robot pickers are gaining versatile capabilities, including 3D perception, understanding how to manipulate objects, real-time motion planning, and the ability to learn a task with just a few training examples (few-shot learning).

They can perform their job – picking items from storage bins and adding them to individual orders for shipping – without specific instructions, even when handling items that haven’t been pre-categorised.

This adaptability is crucial, especially when dealing with electronics that require different handling methods, such as a light bulb versus a stove.

These robots are held to high standards in a commercial setting. Previously, Knapp’s robot pickers reliably handled only about 15% of objects, but the Covariant-powered robots now manage about 95% of objects efficiently.

They’re also faster than humans, picking around 600 objects an hour compared to humans’ 450. Importantly, these robots have not led to job layoffs at the Obeta facility. Instead, human workers have been retrained to understand and work alongside robotics and computers.

Data: Manage Info, Don’t Just Amass It

In 2018, McDonald’s faced stiff competition and needed a transformation. They partnered with Uber Eats to boost online delivery, adding $4 billion to annual sales by 2019. Their long-term vision involved becoming data-driven.

They turned their restaurants into data hubs, using machine learning for personalised orders and curbside delivery. Data analysis helped predict factors like weather and events affecting demand. This approach was a financial success in just two years.

To unlock AI’s value, organisations must establish a strong data foundation:

Modern data engineering: Unify data from various sources into curated datasets for versatile analytics. Frameworks for data ingestion and ETL ensure data quality and enable rapid development of analytic use cases.
AI-assisted data governance: Cloud-based AI tools automatically cleanse, classify, and secure data, ensuring quality and ethical handling.
Data democratisation: Make data accessible and easy to use for more people across the enterprise, with self-service, AI, BI, and data science tools, enabling personalised analysis for specific business needs.

These three capabilities address common data-related challenges, such as accessibility, trustworthiness, readiness, and timeliness. They enable real-time integration of data from big and small sources, agile reporting, and the application of AI to generate valuable customer, market, and operational insights accessible to all.

With a strong data foundation—comprising diverse, AI-managed data from various sources, accessible across your organisation—you can fully harness its potential. You can utilise data for increasingly precise and powerful purposes, but, similar to human intelligence, this demands active participation from your team.

Expertise: Unleash Your Employees’ Talent

Etsy, an online marketplace for handmade and vintage items, abides by the motto “Keep commerce human.” Understanding that aesthetic style plays a crucial role in purchasing decisions, the company turned to its human experts to teach its search engine to recognise and classify style.

This was a complex task for Etsy since most of its products are unique creations, often borrowing from multiple styles or having no clear style definition. With around 50 million items listed, previous AI systems struggled to make meaningful style-based recommendations.

One approach assumed items were similar in style if they were frequently purchased together by a specific demographic, while another used low-level attributes like color and material to group items by style. Neither method could capture the influence of style on purchasing decisions.

To tackle this challenge, Etsy’s merchandising experts developed 42 style labels, covering 15 categories from jewelry to toys. Some labels drew inspiration from the art world (like art nouveau and art deco), while others conveyed emotions (fun and humor, inspirational).

They curated a list of 130,000 items categorised into these 42 styles, leveraging their expertise to enhance the AI’s understanding of style.

Etsy’s team used buyers’ style-related searches to teach their system. When someone searched for a style like “art deco sideboard” and clicked, “favorited,” or bought an item, Etsy labeled that item with the chosen style.

In a month, they collected 3 million labeled instances to test their style classifications. Engineers then trained a neural network to predict styles for all 50 million active items on Etsy.com.

This approach became invaluable during the COVID-19 pandemic when people flocked to Etsy for masks. Buyers could specify the design they wanted, such as polka dots or animal faces.

Mask sales skyrocketed to $740 million, doubling the company’s revenue and raising its market value to $22 billion. It allowed buyers to find masks that matched their taste and style.

Machine teaching unleashes the hidden expertise within your organisation, making AI more accessible to a broader range of people. It’s customisable for your business, enabling innovation and advantage, rather than just playing catch-up with technology.

Machine teaching is especially useful when there’s little or no labeled training data for machine learning algorithms.

To make the most of both systems and knowledge workers, organisations should change the way specialists and nonspecialists work with machines.

Start by giving domain experts basic AI knowledge to transfer their expertise to company processes and technology and to come up with creative ways to use AI in the business.

Architecture: Build Adaptable, Living Systems

Old IT systems create barriers between business areas, making it hard for companies to adapt to new technologies and changing market conditions. Innovation projects often stall because of these rigid, outdated systems.

In contrast, forward-thinking companies are embracing flexible, boundaryless, and human-centered IT architectures. These modern systems break down barriers within technology, between companies utilising cloud platforms, and between humans and machines.

They rapidly adjust to business changes, scale innovation, and mimic human behaviors.

Take L.L.Bean, a retailer with a rich history. Despite its commitment to customer satisfaction, it struggled due to outdated IT. Its legacy system, comprising mainframes and servers, hindered seamless customer experiences across various channels.

IT personnel spent more time managing infrastructure than enhancing customer value. Transitioning to adaptable, human-centric systems enabled them to innovate and respond effectively to customer needs.

In the United States, 73% of consumers were using multiple channels for shopping. Research revealed that these multichannel shoppers spent more money, both in physical stores (4% more per trip) and online (10% more) compared to single-channel customers. They were also more loyal and likely to recommend their preferred retailer to others.

To compete with online giants like Amazon, L.L.Bean aimed to provide a satisfying omnichannel experience. They separated crucial applications from their legacy IT system and moved them to Google’s cloud.

This shift allowed for better data integration, more efficient handling of website traffic peaks, and quicker introduction of customer features. With the flexible cloud-based architecture, the company’s developers could focus on experimenting with new features and launching them quickly.

The path forward depends on choices made in your technology stack. Transitioning to more human-centered AI and automation is crucial. Start by investing in core technologies like cloud computing, data analytics, and mobility.

Rethink application development to leverage cloud capabilities and microservices for flexibility. Focus on creating highly valuable, reusable components.

By aligning business and technology strategies, organisations can develop unique offerings with exceptional agility.

Strategy: We’re All Tech Companies Now

Ocado, the world’s largest online grocery retailer, has developed advanced AI, robotics, and cloud technologies for over two decades. Their expertise includes more than 150 patents.

Their impressive achievement is even more remarkable given the complexity of the grocery industry. Online grocery shopping presents unique challenges, especially with varying product shelf lives & storage requirements.

Ocado’s customer fulfillment centers use advanced robotics, with dishwasher-sized robots moving at high speeds on a three-story grid to fulfill orders rapidly. Human employees also play a crucial role in order picking.

Ocado expanded its tech expertise beyond online grocery retail by creating the Ocado Smart Platform in 2015. This platform, running in the cloud, provides e-commerce fulfillment, logistics, and swarm technology.

It allows retailers worldwide to replicate Ocado’s model profitably and at scale in their regions, offering features like real-time stock projection, last-minute order processing, and intelligent delivery routing. The platform has been adopted by various grocery retailers globally.

Ocado’s strategy demonstrates the successful integration of business and technology, offering their advantages to other players and adapting to new market demands.

They’ve transformed into a grocery retailer and technology company, showcasing adaptability and innovation.

Similarly, other companies have integrated intelligence, data, expertise, and architecture into their unique strategies across various industries. There’s no one-size-fits-all approach.

Technology-integrated strategies require both careful planning and agility. Technology investments must be logical and well-sequenced, but in today’s fast-paced world, acting swiftly is essential.

As we’ve seen in various industries, AI is revolutionising how businesses operate, but it’s people who lead the way. Our framework offers a clear roadmap for companies ready to embark on this journey.