Databricks, a San Francisco-based startup last valued at $38 billion, on Friday released open-source code that it said companies could use to create their own chatbots along the lines of OpenAI’s ChatGPT.
The code is an AI model, an algorithm that is trained on sets of data and can then learn from new data to perform a variety of tasks.
Databricks CEO Ali Ghodsi said the release was aimed at demonstrating a viable alternative to training a kind of AI model called a large language model with enormous resources and computing power.
A large language model underpins OpenAI’s viral chatbot ChatGPT. OpenAI, valued at $29 billion, trains its AI models with huge troves of data on a supercomputer from investor Microsoft. The computing costs are “eye-watering”, OpenAI CEO Sam Altman has said.
OpenAI charges business for access to its models for their own applications and has projected $1 billion in sales by 2024.
Databricks’ effort comes with caveats. Ghodsi told Reuters that, while the open-source chatbot displayed impressive capabilities at such tasks as drafting blog posts, the company had not released formal benchmark tests to show that the bot matched ChatGPT’s performance.
Databricks sells cloud-based data mining and analytics software to businesses and said last year it had surpassed $1 billion in annualized revenue.
Databricks wants enterprises to train their own AI models using its software. Ghodsi said the company’s researchers had taken a two-year-old model that was freely available and trained it with a small amount of data for three hours on single computer that anyone with a credit card could rent.
“The future will be that everyone has their own model, and they can actually train it, and they can make it better,” Ghodsi said. “And that way, they also don’t have to give away their data to someone else.”
Databricks’ move comes at a time when startups are raising millions of dollars of venture capital investment to train their AI models and as big tech firms such as Google and Meta Platforms rush to shrink the size and cost of AI models while improving their accuracy.
“My belief is that in the end, you will make these models smaller, smaller and smaller, and they will be open-sourced,” Ghodsi said. “Everyone will have them.”
