Robot gardeners? Landscaping with large models

Artificial intelligence is reshaping the way we interact with the world, and this wave of innovation is also sweeping across landscaping.

San Francisco startup Electric Sheep Robotics has launched the new Verdie robot , which is equipped with a powerful artificial intelligence system that can autonomously complete gardening tasks such as mowing the lawn, trimming bushes and blowing fallen leaves, and can operate in any outdoor environment without the need for teaching.

“With the debut of our Verdie robot, the landscaping sector’s first AI robot for tasks like mowing and edging, it’s exciting to see our ES1 technology power multiple robots that can work alongside crews without the need for engineers to be on site to set specific paths for them,” said Nag Murty, co-founder and CEO of Electric Sheep.

The core technology of Electric Sheep is their self-developed ES1 artificial intelligence system, which is an advanced large-scale world model that operates in a similar way to ChatGPT, but is designed to solve spatial reasoning and planning problems instead of focusing on natural language processing like ChatGPT.

ES1 provides the robot with the capabilities needed to autonomously navigate and perform tasks in outdoor environments by semantically understanding the environment, creating maps that can be used for path planning, and identifying the boundaries of the workable area, working in a manner similar to making dense predictions about the state of the world through a single model.

To tackle this daunting challenge, Electric Sheep uses two key technologies: ES1, a large-scale world model, which provides the agent with a concept of outdoor work, and reinforcement learning through simulated environments, which allows the robot to learn to solve specific new tasks, such as string trimming, by repeatedly training in a simulated environment.

In the example of string trimming, they used reinforcement learning to train a small fully connected policy network on top of the embeddings of ES1. The network receives as input the time series projections of ES1 and then outputs the speed commands that control the movement of the trimming tool. All training is done in the NVIDIA ISSAC simulator, so the entire policy can be efficiently trained on a single desktop GPU.

Through this innovative AI architecture, Verdie and other Electric Sheep robots already have a certain degree of autonomy and task execution knowledge when they are "out of the factory". They can be used out of the box without the need for on-site engineers, greatly reducing deployment and operating costs.

Moreover, what is truly unique about Electric Sheep is that it combines technological breakthroughs with business model innovation. Rather than developing the entire gardening robot product on its own, the company cleverly acquired several traditional landscaping companies and gradually transitioned their operations to automation and robotization.

Electric Sheep said it has acquired traditional outdoor service providers to incrementally transform operations by deploying its software and robots. Electric Sheep acquired two of the landscaping companies in October 2023, and then acquired two more a few months later in December.

The acquisitions are also a way to gain data that the company can use in its reinforcement learning operational sandbox, where Electric Sheep can build on its base model and apply it to the Verdie and RAM robots, Murty said.

This strategy has brought three major benefits to the company: a ready-made customer base and revenue source, ensuring its own cash flow; obtaining a large amount of training data in actual business scenarios, providing important material for reinforcement learning; and retaining the professional human resources of traditional companies, providing the necessary expertise support during the unmanned transformation process.

With this unique "M&A + AI" dual-wheel drive model, Electric Sheep started making money from the first day of the acquisition because it ran existing businesses with existing customers and accumulated a large amount of practical experience data. It also received funding from Tiger Global and Foundation Capital, and its revenue increased eightfold, laying a solid foundation for the eventual realization of fully automated operations.

Last words

Of course, Electric Sheep’s ambitions extend beyond commercial success; it aims to become the first large-scale outdoor maintenance company driven by artificial intelligence and robotics.

From the perspective of ecological protection and reducing carbon emissions, autonomous gardening robots such as Verdie are undoubtedly the future development direction. With the improvement of computing power and energy utilization, they will be more efficient, environmentally friendly and economical. Perhaps in the future they will partially replace the traditional human model. The future landscape will no longer only have the marks of human cultivation, but also be dotted with the footprints of robots exploring the world.

However, robotization is not something that can be achieved overnight. Before achieving full autonomy, Electric Sheep still faces challenges such as insufficient cognition and adaptability to complex environments and how to meet the personalized needs of different customers.