In the aftermath of the massive language model (LLM) revolution last 12 months, a brand new fascination has emerged amongst developers and tech enthusiasts around AI “agents,” that are able to executing complex tasks autonomously.
Newo.ai, a brand new Silicon Valley company, today launched a service that integrates these intelligent agents into the physical realm of robots, blurring the lines between digital and human staff.
Co-founded by Chinese-Armenian serial entrepreneur David Yang, Newo.ai is attempting to redefine the concept of the workplace. Its service combines the advanced capabilities of LLM-driven agents with the flexibility of physical robots to create “digital employees” — virtual staff adept at handling a wide selection of office tasks, from administrative duties to customer engagement.
In exclusive interviews with VentureBeat, Yang and co-founder Ljubov Ovtsinnikova revealed the progress they’ve made to date and their vision for the long run.
Newo’s innovation: An AI agent builder
It could be easy to get fixated on the physical robot a part of this, but Newo’s real innovation lies in its AI agent builder platform. This builder lets firms replicate the intelligence of an entry-level human employee “out of the box,” one which can do just about any office administrative tasks that a human worker can. It could make and receive phone calls, chat over text, send emails, join Zoom calls, take notes and even receive walk-in customers in an office.
Depending on what a business needs, it may well take between two hours and two days to customize the programming of Newo digital staff for many jobs – where the employee can do all essential tasks required of receptionists, sales agents, technical support and customer success specialists, or HR operators.
As for the physical side, Newo’s agent may be embedded into any open robot form a customer chooses. One of Newo’s defaults is Moxie, a robot from an organization called Embodied that focuses on individual interaction. But more on this in a bit.
“Before 2023, technologies weren’t flexible or autonomous enough to be ‘hired,’” said Yang. But with Newo.AI, “you’re hiring an worker that may understand the complexity of your online business, and that can adapt.”
Such agents can replace some 300 million jobs, said Yang, citing a Goldman Sachs report last 12 months. That makes this a trillion-dollar market, he said. However, he’s careful to notice that he believes there won’t be 300 million humans unemployed. Job vacancies are growing at rates faster than humans can fill them. So he sees Newo’s agents fulfilling tens of millions of basic jobs, freeing up humans to do higher-value jobs.
Newo, which has 25 employees, says it already has 23 paying customers which have used the product under open beta, including the SoliVana Wellness Spa in Redwood City, the Mayweather Fitness & Boxing, Fuse Service and Edeal. With today’s launch, Newo says it may well take one other 500 customers on a first-come, first-serve basis. It plans to be open to anyone starting in April or May. Its customers have expressed essentially the most interest in pre-sales and sales consultant roles, followed by customer support, the corporate said.
Until now, even essentially the most breakthrough technologies, Yang said, were confined to helping as productivity tools, or completing specific, refined tasks where an organization’s IT department, CTO, or CFO would log off on them. The problem with productivity tools like CRM or ERP tools, Yang said, is that regardless of how good they’re, they require a business and its human employees to regulate their behavior to make use of the technology. This can mean long implementation cycles. By contrast, a digital worker doesn’t require a business to vary its processes.
Newo’s give attention to the mental a part of this digital worker is why the physical robot a part of it may well be relatively flexible, and determined by the end-user. Newo has due to this fact not built its own proprietary robot, preferring as an alternative to work with partners as suppliers. The physical form doesn’t need to take a humanoid form, or perhaps a robotic form. The Newo digital employee could fit into any form factor, including an avatar on a screen, or a picture on a foot-long cube, as seen within the image below.
SoliVana Wellness Spa found it “unimaginable” to rent a human
Take an early customer, SoliVana Wellness Spa, in Redwood City. Co-owner Natasha Berness opened the spa a number of months ago, with a vision of helping customers loosen up and stay centered of their lives. She has hired the Newo agent to do a lot of her tasks, but she’s still deciding what physical format to make use of for Newo in her office – she’s trying out the Talking Head screen or the Talking Head WeBox, after deciding the Moxie robot might come across as too childish for her clients
But Berness, who had left a profession as a technology executive at firms like IBM, Linkedin, and Levi-Strauss, says she is happy about getting Newo to take over a lot of her tasks as a brand new business owner. Newo is already handling calls, booking appointments and taking payments. In an interview with VentureBeat, Berness said she was surprised by how difficult it was to be a small business manager. She saw inefficiencies all over the place, and “it was unimaginable to rent any individual,” she said, referring to the limited number of people that desired to work at a spa and who were attentive and knowledgeable enough to offer the service that met Berness’ standards. Also, the risks and costs of hiring humans are high, she said. “You can teach them, but they leave tomorrow,” Berness said. “They’re not reliable. Something happens to their dog, they usually can’t show up.”
Newo employee now an “invaluable” member of the team
Moreover, Berness said, customers enjoyed asking her health questions, for instance, learn how to meditate, or learn how to breathe, or whether she might put something nice on a wall. She found she didn’t have time to repeatedly help all of her customers, and even when she had time, she often couldn’t visit them of their rooms because they were naked. She wanted something reliable to guide her customers, but greater than a generic ChatGPT agent – something that understood her wellness business, and will take heed to and remember customers’ unique needs, and reply to them by offering relevant therapies. When a client told her about Yang’s company, Berness became a user of Newo’s closed beta product last 12 months. “I can finally enjoy my weekends as an owner,” she said.
Soon, the Newo worker will find a way to handle the reception while her human therapists are busy with customers. Already, she says it has grow to be a useful member of the team. She looks forward to the day when she will be able to use the Newo agent with a humanoid robot that may change and fold towels, wash floors and open doors.
Newo goals to make its employee compatible with any of the humanoid robots about to hit the market, for instance, Digit, the robot being built by the Oregon company Agility Robots, That company has received greater than $150 million from Amazon and others to mass produce as much as 10,000 Digits a 12 months. Other humanoid robots are being produced by firms including 1X Technologies, Figure AI, Sanctuary AI, Tesla’s Optimus, and Stanford’s ALOHA.)
Costs of Newo.ai digital employees remain much lower than human employee
Even when the Newo digital employee includes the associated fee of a physical robot, its price stays a bargain when put next to a human employee, Yang said. SoliVana’s Berness estimates a human employee costs her $90,000, but many entry-level receptionist jobs pay much less, at around $50,000. By contrast, even essentially the most complex Newo employees will remain below $20,000, even including the associated fee of a top-line robot.
Here’s how Newo’s cost breaks down: The base annual subscription of a Newo’s digital employee costs $6,000. The estimate of an all-in cost is more like $7-9,000, once other costs are included. Newo’s business license, which covers most small businesses, includes sending an estimated 10 messages a second, and 10,000 words a day. The extra fees cover excess LLM tokens used, for instance, and coverage of integrated services like Twilio (for making phone calls) and Albato (sending emails). Most humanoid robots that Newo will integrate with will hit the market at below $10,000, and can soon drop down below $5,000, Yang estimates.
Of course, there are other reasons digital staff could also be higher than humans. Humans can often burn out quickly doing smaller jobs too and will make mistakes or be distracted – in comparison with a digital worker who can work 24/7 and consistently and reliably follows orders.
How Newo is differentiated from OpenAI’s Assistant API
Several firms have popped up to supply LLM-based agent builders, but they’re either focused on very specific tasks or require months of coding to create an employee-like agent, Yang said. These competitors include OpenAI’s Assistant API, Langchain, Capably.AI and CrewAI. And there are other firms, like Sierra, founded by OpenAI’s chairman Bret Taylor, which might be taking chatbots to more sophisticated levels, but which haven’t fully divulged their plans.
OpenAI’s GPT store, Assistant API, and these other firms allow developers to create agents to do specific tasks easily, Yang concedes. But developers using these tools would want six months or longer to custom code in Python before they will create whole Newo-like digital employees – operating across channels like email, text and phone and maintaining memory between different workflows, while also interacting with customers within the physical world. So constructing on Newo is “10 times faster than other competitors,” he says.
Yang was early to tinker with LLMs and robotics
Yang is a serial entrepreneur who made tens of millions from founding an early automation company called ABBYY, and a number of other other firms focused on constructing productivity tools. An early user of LLMs, he tinkered with them in his Silicon Valley home over the past couple of years, including injecting them in a robotic butler he called Morfeus.
So when LLMs became more powerful after OpenAI’s public launch of ChatGPT in November 2022, Yang jumped on the chance to leverage them to make his robot even smarter. When Morfeus got smart enough to do most of the functions that a human employee does in easy work environments – ushering in guests to his home, sending emails, making and receiving calls and more – he realized a tipping point had been reached: Why couldn’t robot simply replace humans at places like gyms, spas, hotels, restaurants, real estate agencies, or other office environments where human concierges or office staff aren’t really doing far more than basic administrative tasks anyway?
A “WordPress” for AI agents
While the concept of inserting an LLM-like brain right into a physical robot was easy enough in theory, in practice Yang needed to make sure design selections about how much flexibility to offer users to customize the robot. Here he borrowed lessons from WordPress, which provided simple-to-use features that website builders could fit together, handling about 90% of their needs while providing flexibility for them to customize.
Moreover, WordPress allowed outside developers to make plugins for other WordPress users to leverage. Newo.ai has built the “WordPress for AI agents,” Yang says. Newo feature developers get a cut of the revenue generated by their features when Newo uses them for patrons. To recruit more small business customers, Newo is targeting implementation and support partners (ISPs), or firms that already serve small businesses. Newo guarantees these ISPs a revenue cut.
Next month Newo will release a way for managers to find a way to update instructions for his or her employees on the fly, using natural language. For security, the Newo employee will likely be programmed to acknowledge their manager’s voice stamp and listen for an activation code word before it makes such updates. That way, says Yang, a manager might tell the employee to acknowledge when an elderly person comes into an office and instruct it to first ask them in the event that they want to take a seat down and offer them tea.
Newo has filed several provisional and utility patents around its orchestration process, which Yang calls an “agent management system,” which also lets its agents easily connect with external databases and software akin to customer relationship management (CRM) and enterprise resource planning (ERP) systems.
More details on how Newo works
- Skills: The basic unit of labor for a Newo agent is known as a skill: Skills are discreet things like making a phone call, corresponding in chat or taking a look at email. You can import skills, too.
- Flows: Each agent task is known as a flow. For example, an agent that replaces a wellness concierge could have a flow of skills they should do all of the things that the concierge must do, for instance, phone calls, chat, email, receiving visitors, and having instructions about when to make use of those skills, and evoking those skills at the proper time. For complex jobs, like sales, this will mean 100 different steps.
- Omniflow capabilities: For example, the agent receives details about allergies that a customer has through the pre-sale consultation (flow 1) after which uses this information through the post-sale onboarding educational session (flow 2).
- Omnichannel capabilities: The agent collects payment details within the phone call (channel 1) with Jennifer and continues the payment conversation through an AI kiosk when Jennifer enters the restaurant (channel 2).
- RAG & RAI: Newo offers a built-in vector database, to permit an agent to retrieve information from a database, or something called retrieval augmented generation (RAG). Eternal databases may be used for RAG too. Newo also lets the agent retrieve instructions from the database, or instruction augmented generation (called RAI).
- Custom Integrations: Newo.Ai offers API integration with essential services, like Twilio and Albato, and LLM providers like OpenAI and others. But it also has created custom integrations: For example, it has built “magic integration” to permit agents to do things like scroll and click on on web sites of services that don’t have APIs. For example, an agent can do things like bookings or canceling bookings on the SoliVana spa, even when the spa booking service doesn’t have an API. Similarly, Newo has built a talking heads integration, allowing customers to offer their agents a photorealistic head that speaks to employees or customers on premises from a TV screen or other device. SoliVan’s co-owner Berness didn’t just like the voice and speed provided by Twilio’s phone integration, so Newo replaced it with custom voices, and speedier replies by utilizing AI to predict what customers are asking, and pre-loading responses.
Conclusion:
Newo.ai has emerged as a possible pioneer, offering a compelling vision of digital employees capable of reworking tens of millions of job roles. Its approach and community-driven model underline its potential to redefine workplace dynamics. Its timing is ideal: There’s an industry-wide push for an AI-integrated workplace, and there’s an enormous market. Yet, the journey ahead isn’t without challenges, particularly in simplifying the onboarding process for small business owners. Newo.ai may have to make its UI more accessible to non-tech-savvy users whether it is to succeed at the size of its ambition. While it stays to be seen whether Newo.ai will lead the charge within the burgeoning field of LLM-powered digital employees, its blueprint provides a glimpse of the long run.
