Melonee Wise, Agility Robotics CTO, on the evolution of robotics, the current “Cambrian Explosion” of innovation, and the barriers companies face inviting robots into the home
Melonee Wise, Agility Robotics CTO, on the evolution of robotics, the current “Cambrian Explosion” of innovation, and the barriers companies face inviting robots into the home
The past, present, and future of humanoid machines.
A machine capable of human-like motion has captured the imagination of scientists and storytellers for millennia. The ancient Greeks spun myths about a titan made of bronze built by Hephaestus, God of invention and blacksmithing, while Leonardo da Vinci sketched out a mechanical suit of armor controlled by an elaborate system of pulleys and cables at the tail end of 15th Century. It wasn’t until the advent of compute, however, that the ancient dream was realized.
Now half a century after the first modern robots debuted, companies like Agility Robotics are creating state-of-the-art bipedal designs to augment human labor. These companies are building a golden age of both technical advancement and market penetration. Robots now carry payloads through warehouses, deliver meals across town, navigate cars through traffic, and even assist in thoracic surgeries.
Melonee Wise, CTO of Agility Robotics, has helped shape and lead this movement. From her early days building robots at the University of Illinois, to her successful exit from Fetch Robotics, to her current work with Agility building the most advanced commercial robots on the market, few engineers understand the history and future trajectory of robotics quite like Melonee.
Atoms & Bits spoke with Melonee about her career and philosophy on building robots, where we’ll see robots next, and why it may take a few more years before a robot will do our laundry for us.
This interview has been edited for clarity.
A&B: Robots started out as messy hydraulic systems and moved on to articulated arms. Now they’re evolving into humanoid, bipedal robots. What is the advantage of these new designs?
Melonee Wise: Agility didn’t purposefully create a humanoid form. It was about using the right manipulation technology to do certain complex motions within relatively complicated environments. Bipedal robots bring a larger breadth of human-scale tasks to manipulation that mobile robots struggle with.
As someone who spent the better part of a decade with wheeled robots in these environments, there are certain limitations to those designs. Mobile robots quickly start facing stability problems and other issues related to reach. They can’t get low to the ground or reach high up in the air very easily. Those motions increase the complexity. Eventually, there are so many accessories on these mobile robots that they become unwieldy — cart connections, roller tops, shelfs — the list goes on and on. One of the things that those accessories are compensating for is the challenges of interfacing with the different equipment in the environment.
The advantage of bipedal robots is their ability to reach up and hold their balance while still only taking up a relatively small amount of space — all while traversing a complex environment with a payload.
Another problem that mobile robots face is a constant trade-off between payload and unloading. I see a lot of mobile robots with shelving units, but when they get to the endpoint a person has to unload them because there's no natural way to unload all those goods. The complexity of the mechanism would take up a large amount of the payload weight of the robot. What we’re seeing in some cases is our bipedal robot, Digit, becoming the new best friend of all of the AMR robots in the world, because it can unload them.
A&B: It sounds like we’ve come to the point where humans can design a robot tailored for a warehouse or other specific circumstance, and we end up creating something that looks a lot like ourselves — robots in human form to work in those spaces. What does that tell us?
Melonee Wise: If you look at Digit, our robot, it’s not exactly human. We just describe it as human; we anthropomorphize it. I don’t think it would be good if either our legs or knees were shaped like Digit’s. Its arms are very long compared to a human arm. The robots we build are superhuman in form-factor. They look human, but they’re definitely not.
A&B: It’s human in impression only.
Melonee Wise: Perhaps.
A&B: What originally got you into robotics?
Melonee: I’ve always been interested in building things that move. I grew up pretty poor, so I didn’t have access to cool robots, toys, and stuff like that as a kid. I finished high school in 2000 when FIRST wasn’t the big production that it is today. At the time, there was only iRobot and Boston Dynamics, so the robotics landscape was very limited. It wasn’t until college at University of Illinois where I was working on a degree in mechanical engineering and physics engineering that I built my first robot.
A&B: What did you build?
Melonee Wise: Derek, one of my friends and Fetch co-founders, came to me one day and said, “Hey, I’m thinking about building a robot. You want to build it with me?” And I was like, “Yeah, let’s do it.” And so, we built this terrible little robot called Zippy as part of a physics seminar on digital and analog electronics. We had to beg, borrow, and steal the parts. The cool thing about this seminar was that we spent a month or two working on whatever project we wanted with a budget of $500. Back then $500 felt like a huge sum. I thought I could build as many robots as I wanted. That was really a different time. There wasn’t an open source platform for robotics. Everything was done from the ground up.
A&B: How did your robotics career progress after college?
Melonee Wise: I went out to California in 2007 to work on an autonomous car project as part of the DARPA Urban Challenge. I was doing my PhD at the time, and the founder of Willow Garage offered me a job and to pay off my college debt if I’d leave my PhD. I came to work at Willow Garage, which eventually became the leading robotics incubator for a good decade. I got very lucky ending up there as the second employee. When I showed up for my first day, it was me in a giant, empty building with one other person.
A&B: But you eventually made the switch from autonomous cars to robotics.
Melonee Wise: Willow Garage pivoted from autonomous cars to open source robotics. From about 2008 to 2012, we developed an open source platform called the Robot Operating System or ROS. We worked on the PR2 mobile manipulation robot with two arms and a wheeled base that was famous for research. At that time, I started another open source project with a co-worked called the TurtleBot, which is an educational platform. To date, there’s probably been about 15,000 TurtleBots sold into the research and educational market. It has been fun to watch grow. I spent the majority of the middle years of my career as an open source developer. Red Hat did a documentary on ROS on some of the work that my colleagues and I did at Willow Garage.
A&B: What came after Willow Garage?
Melonee Wise: I ended up spinning out a startup called Unbounded Robotics. It had some IP issues and challenges, and I eventually shut down the company. After that, I started Fetch Robotics, which did autonomous mobile robots for the logistics industry. We started in 2014 and exited to Zebra Technologies in August 2021. I thought it would be fun and exciting to be a CTO, so I came to Agility Robotics. Here, I’m working on industrial mobile manipulation with Digit in a space very similar to where I was with Fetch — just this time with legs and arms.
A&B: Looking back, what would you tell young entrepreneurs and technologists?
Melonee Wise: Learn to program. I learned to code when I got to Willow Garage. Someone I worked for said, “Look, if you don't learn to program, you're out.” But I wish someone had clued me in when I was at the University of Illinois. It would have helped me so much in my career. I'm good at it now, but I'm not great at it. And I'll never be great at it, because there’s a whole lot of formal training that I just never got.
A&B: Do you still feel that way with the dawning of the AI era?
Melonee Wise: Yes, because AI can only help so much. It’s great for making a script and other little things. However, there are real challenges with the algorithms for robotics. How efficient and fast they run and the size of your CPU are extremely important, especially when you're carrying your battery around and power really matters. We'll see. Maybe AI will get there, but robotics still requires a lot of human creativity and invention.
A&B: Could you talk about the evolution from those early days of robotics when it was just Boston Dynamics and iRobot to where we are today?
Melonee Wise: The history of industrial and research-based robotics starts in the 1960s. The first robot was Unimate deployed by GM in a New Jersey facility. It moved things like flaming hot metal rods from one process to another. Those robots were fairly successful. You have to remember, though, that robots developed concurrently with computers. The first were hydraulic and ran off magnetic tape. It wasn’t until the late 1970s and early 1980s that we had the mathematics for doing inverse kinematics to apply to computers.
A&B: When did electronic robots arrive?
Melonee Wise: An all-electric robotic arm didn’t land until about 1976 or so. All the things that we take for granted about electric motors and being driven by a computer, none of that happened until much later than you might imagine. There’s an amazing video from the late 1970s, where this scientist says, “We have this new teach pendant that allows the robotic arm to move in a straight line.” That was groundbreaking, because they had figured out the transformation math in computers for moving an end effector point through a straight line while moving six degrees of freedom. When computers came on the scene, everything became electric and design wasn’t trapped by hydraulic messes anymore.
A&B: At what point did the market take off?
Melonee Wise: This “Cambrian Explosion” of robotics took place in the 1980s and 1990s when people started working on robots that flew, crawled, and drove around. In the late 1970s, SRI in Menlo Park put out a video of a robot called Shaky. That was the first robot to do a whole bunch of crazy things. They had an RF transmitter from a computer that took up most of a room and a robot that basically spent four hours thinking about how to drive five feet. Still, that was a big breakthrough in robotics.
In the late 1990s, designers got interested in humanoids. That’s when Honda and other players built devices like Asimo. A video of Asimo walking sent a shockwave through the research community. The video made it seem like this team had figured out the mathematics of human gait in robotics after researchers had puzzled over it for two decades. It turned out to be a mechanical trick, but it still triggered this big retraction. Researchers stopped sharing data to try and protect their research.
iRobot then had their Super Bowl success in the early 2000s. After that, some of these larger companies started doing more media about their products. Still, there wasn’t much going on until the DARPA Grand Challenge kicked off the autonomous car race. I remember when the first cars went 50 feet down the road. It felt like the technology’s moment had arrived. Around that time, the founder of Willow Garage said, “Why don't we have the equivalent of Linux for robotics?” He decided to start Willow Garage to start building the equivalent of the LAMP stack for robotics. We ended up creating ROS, which is not really an operating system at all. It's more like a meta operating system or a conglomeration of tools and packages that help support the deployment of robotics, and it’s all built on a Linux-based system. That was a huge catalyst. Suddenly, people didn’t have to reinvent every part of their robot. They could use the same software, cameras, lasers, and stuff like that.
After ROS, this moment of sharing happened. This was another “Cambrian Explosion” in startup land. Once the robotic software stack was available, people were building companies on it. All of a sudden, between 2012 and 2015, we saw hundreds of robot companies form, including Agility and Fetch. Then Amazon acquired Kiva, which excited the venture community since it seemed that investors actually saw value in robotics. That was serendipitous timing. And so, over the past seven to eight years, autonomous mobile robotics had its moment. A lot of the research developed in the early 2000s finally started making its way into the market. Now, we’re finally starting to see the commercialization of bipedal mobile manipulators.
A&B: You talk about these moments of expansion — these “Cambrian Explosions” — followed by eras of retrenchment when the industry becomes more insular. Where are we now? The last decade has certainly seen a fair amount of expansion. So it seems that we’re either in the middle of an expansion moment or the end of one.
Melonee Wise: We’re still in a very big expansion moment. There was an expansion in robotics from the early 1960s until about the mid-1990s followed by a decade of dormancy. Now we’re in another 30 to 40 year-long expansion phase. In robotics, I fundamentally believe that all the technology exists for continued progress; it’s just a matter of commercializing all of it. There’s nothing that prevents us technologically from success in the set of tasks that everyone’s going after in the semi-structured, light industrial market.
A&B: So you see no retrenchment on the horizon.
Melonee Wise: The next Dark Age will come when robotic companies feel like they have captured all of the light industrial market and before we reach the general public. Robotics will experience a lot of success in a number of sectors for the next decade. Teams are building into many parallel domains including light industrial, agriculture, construction, warehousing and logistics, as well as manufacturing. The expansion of robotics is still massive.
A&B: What triggers these seismic shifts in robotics?
Melonee Wise: Contractions happen when engineers run out of things to do, which will be a very long time from now. I believe you’ll see the next contraction in our industry around selling robots into the home.
A&B: Why is that?
Melonee Wise: The home poses a lot of challenges. Every consumer has an implicit value that they assign to the objects in their life. When you look at a TV or a car, you know exactly what value it provides. A $15,000 car gets you from point A to point B. When you buy a $1,000 TV, it provides entertainment. These things all provide measurable, intrinsic value. When you ask someone how much a robot should cost, it’s very dependent on that individual’s metric of intrinsic value. Today, if you ask someone how much it would cost to build a robot to pick up a gallon of milk, take it out of the fridge, and put it on the counter, you would get a lot of different answers. The reality is that robot costs around $30,000, and it can barely do anything.
A&B: Are there robots that are more desirable to the home?
Melonee Wise: When I was at Willow Garage, we spent about a year looking at problems in the home. And we found out the general public wants the tidy robot. They want a robot that can take a snapshot of your home in its most pristine state — the way that you want it always to look — and clean up. They would pay a lot of money for that robot to wander around cleaning.
A&B: Everyone would want it.
Melonee Wise: The only impediment to that robot is the insane amount of money it would cost. Would a consumer pay $50,000 for it? What about $70,000? Some things we look at for the home are robot adjacent. Right now, there’s simply too wide of a gap between the functionality people demand and the price at which we can deliver it to them.
A&B: But there aren’t the same gaps in warehouses?
Melonee Wise: When you look at industrialized or light industrialized verticals, you don’t have that same gap. We can build devices, robots, and solutions that get a return on investment. In the warehouse, there’s a large labor gap, labor turnover, repetitive strain injury issues, and problems with safety that all calculate into the return on investment. Now we have the technology to do it in a safe fashion. In industrial environments, we also have some regulation and control over how people behave. Employees are trained and have knowledge of the rules they’re expected to follow. All of that creates a very conducive environment for robotic technology.
A&B: I would have thought that it was a matter of the complexity of the motion. Motion in a warehouse is relatively predictable and repetitive. Whereas in the home, the motions are finer, more creative. A robot would need to spray down a window and scrub this particular corner of the counter. There’s a lot of motor skill involved.
Melonee Wise: We can design a lot of that fine motor activity today. The bottleneck is the overall cost of the machine. Honestly, you can build super cool, complicated, and capable robots for the home. It comes down to how much a consumer wants to pay for it.
A&B: How do humans use these machines?
Melonee Wise: Ironically, the things that have been successful in teaching children how to program have taught adults how to use robots. In the last 20 years, STEM education has focused on teaching children as young as three or four how to code using blocks that snap together to make toys move. A lot of those visual programming language tools have conditioned parents. At Fetch, we built a visual programming interface for our mobile robots. One of the first user studies we did was with warehouse managers. When the researchers asked one of the managers how it was to use the robot, he said, “It’s so easy. My kids use something just like it.” Visual programming languages are being brought to bear on things that people use every day — their iPhone, web browser, video games, and robotics.
A&B: What makes it so hard to build a successful robotics business?
Melonee Wise: First, you have to get the technology to work, and then you have to build the entire stack for it. That’s getting a little bit easier now, but you had to build everything from top to bottom in the first couple of decades of robotics. As an industry, it has been hard to get some things off the ground because it’s a full-stack activity. It’s like the early days of computers: engineers had to build the computers and all the media, content, and everything related to it. It wasn’t until about 20 years into the ecosystem of computers that we started to see platform integration, commoditization, and then separation of business concerns. The people who built the machines weren’t also building all the content. In robotics today, we’re not only building the technology, we’re building the full solution.
A&B: As robots advance, how will that affect the future of human labor?
Melonee Wise: That is a complex question. I’m a strong proponent that as time approaches infinity, and technology approaches infinite capability, work ends up defined differently than it did a century ago. Less than two months ago, we didn’t think there would ever be a job called “prompt engineer.” There are also good examples of technology creating jobs. The automobile was the worst thing that ever happened to the horse business, but it created a historic job engine. The problem emerges when people try to transition into new jobs. There’s clear data showing that in the last eight years, more jobs are available today in manufacturing and logistics than in 2016. In 2016, we couldn’t fill 600,000 jobs. Today, we can't fill a million jobs, and that’s just in the United States. Why is that? That has nothing to do with technology. We have been dumping robots and automation into this problem every year as fast as we can. Yet, the labor gap is still growing. I think that problem goes beyond what the technology can address itself.
A&B: Where is the robotics industry headed?
Melonee Wise: Over the next two to five years, we’ll see a lot of focus on making all this new technology more usable. Robotics is facing the hard fact that most people can barely use an iPhone, let alone a robot, which is like an iPhone on steroids with arms and legs. I think we’re going to see more and more startups. I think that right now agriculture and construction are extremely hot for robotics, especially assistive tools in construction. But that will lead to concentration. Already, most of the relevant autonomous mobile robot companies have been bought up. We just went through a startup contraction phase where there was a land grab. All of the relevant Fortune 100 and Fortune 500 companies acquired all the robot technology companies they could. And now we’re going through that cycle all over again. Different technology branches like logistics, construction, and healthcare are being commercialized, and so we’ll probably experience another concentration of startups in these sectors.
A&B: Does the term “deep tech” have meaning to you?
Melonee Wise: Yes, deep technologies are the ideas that are conceptual and far away from the market. They are the tech that you can only find out how they work or whether they’re valuable to anyone by building the thing and putting it in the field and seeing if it will solve a problem.
A&B: Tell us a little bit about battle bots.
Melonee Wise: Over the years, I’ve worked on many robots — boats, cars, and battle bots. One of the battle bots was a 250-pound machine with a weapon made out of a water jet cut out of hardened steel. We had some good battles with a lot of different robots. At one point, Willow Garage got a really high-end laser cutter, and I decided that I was going to design a laser-cut robot. I challenged myself to build an entire robot made out of laser-cut sheets riveted together. When I was at Fetch, we had a weaponized battle bot league with only plastic robots that we live-streamed. I challenged myself to build a mostly 3D-printed bot and called it Rooster, which exploded fabulously. We also hosted 10 different companies and their employees to create battle bots. These days, I do it to challenge myself. I don’t know what I'm going to build next, but some of my co-founders have been building “trash bots” with whatever leftover pieces of plastic they could find.
A&B: Is there anything you take from these robots to your day job at Agility?
Melonee Wise: I would say the most you could apply to Agility or other robotics jobs is these bots help you learn to solve problems. Sometimes I will give myself a constraint — like only building out of a certain material or only out of scrap pieces — which helps me think about challenges and how to build and think differently. It breaks you out of your usual mindset and forces you to adopt new perspectives.
A&B: Thank you for your time.
Melonee Wise: Thank you.