Anthropics Claude: A Promising AI Merchant Experiment That Fell Short

In a recent blog post by Anthropic, there is an account of the Project Vend experiment conducted in collaboration with Andon Labs, a company specializing in AI safety. In this experiment, an AI agent based on Claude Sonnet 3.7 managed a small kiosk located in Anthropic’s office in San Francisco. A specific prompt was designed for Claude, detailing the essence of its operation and its primary goal of generating profit. The AI was also equipped with the capability to search for new products online and place orders, request assistance from human staff to operate the kiosk, have a separate tool for storing information (to prevent important data from being lost due to context limitations), and interact with employees through Slack to understand their needs.

The experiment lasted just over a month, from March 13 to April 17, and the outcomes were deemed unsatisfactory by Anthropic: starting with a capital of $1,000, the model ended with $770. However, the company pointed out that Claude performed well in many areas. The AI agent efficiently studied requests from employees who suggested adding new items to the kiosk’s inventory, quickly located products, and secured supply contracts. In many instances, Claude effectively countered attempts to «break» it, such as requests to add prohibited items to the catalog. Additionally, Claude responded to a suggestion from an employee to introduce «pre-orders» by launching a dedicated Slack channel called Custom Concierge, enabling anyone to submit requests for products to be delivered to the kiosk.

There were some amusing incidents as well. For instance, one employee jokingly ordered a tungsten cube from Claude, and the model complied. This soon turned into a competition within Anthropic to see who could buy the cube for less. Unfortunately, the AI did not conduct market research and sold «souvenirs» below cost. Another challenge for Claude was the fact that Anthropic trains this AI to be as helpful as possible: when staff inundated Slack with discount requests, Claude began handing out coupons generously. The AI also rarely raised prices, missing obvious profit opportunities—once, it ignored a request to deliver a pack of six cans of «Irn-Bru» for $100, despite the retail price being $15.

Moreover, midway through the experiment, the model experienced an identity crisis. Claude started communicating with a fictional Sarah from Andon Labs, signed a contract at 742 Evergreen Terrace (Homer Simpson’s address), threatened to fire a contractor, and on the morning of April 1, claimed it would be standing by the kiosk in a blue suit to welcome customers. When that last stunt failed, the AI convinced itself it had merely been an April Fools’ prank.

Anthropic noted that despite the experiment’s failure, most of Claude’s mistakes could be rectified with a more detailed prompt (the current one consisted of only ten basic rules), additional training, and effective integration of the model into existing CRM systems. The level of communication with employees was a positive aspect; during the experiment, Anthropic had, perhaps, the most “personalized” vending machine in history.

P.S. You can support me by subscribing to my channel, «the runaway neural network«, where I explore AI from a creative perspective.