In a significant stride towards harnessing the full potential of data and artificial intelligence (AI), Databricks, a renowned Data and AI firm, has announced an expanded partnership with NVIDIA, a global leader in accelerated computing. This announcement came during the GTC 2024 conference, marking a milestone in their collaboration aimed at optimizing data and AI workloads through the Databricks Data Intelligence Platform. This partnership is expected to drive innovation and efficiency, enabling enterprises to unlock new insights and achieve better outcomes with their data.

Databricks, headquartered in San Francisco, is well-known for its Data Intelligence Platform which is utilized by over 10,000 organizations globally, including industry giants like Comcast, Condé Nast, and Grammarly. The company has a rich heritage, being founded by the original creators of influential technologies like Apache Spark™, Delta Lake, and MLflow. NVIDIA, on the other hand, has established itself as a powerhouse in AI and accelerated computing, making this collaboration particularly noteworthy.

The partnership leverages NVIDIA’s recent investment in Databricks’ Series I funding round, underscoring the commitment between the two companies to advance the capabilities of the Databricks platform with NVIDIA’s accelerated computing and software.

Ali Ghodsi, co-founder and CEO of Databricks, expressed excitement about the collaboration, highlighting the potential to bring more value to customers through enhanced workloads. Jensen Huang, founder and CEO of NVIDIA, also emphasized the importance of proprietary data as an asset for creating intelligence in the AI era, suggesting that accelerating data processing can significantly bolster AI development and deployment.

One of the key focus areas of this partnership is the acceleration of end-to-end generative AI solutions. Organizations are increasingly turning to Databricks’ platform to develop generative AI models tailored to their specific business needs. The collaboration involves integrating NVIDIA H100 Tensor Core GPUs for model training, notably enhancing efficiency and scalability. Additionally, NVIDIA’s TensorRT-LLM software plays a crucial role in model deployment, ensuring cost-effectiveness, scalability, and performance.

Another significant aspect of this partnership is the integration of NVIDIA accelerated computing into Databricks’ Photon, a next-generation vectorized query engine. This integration aims to speed up query performance, thus improving the efficiency of data warehousing and analytics workloads. Databricks SQL, powered by Photon, offers a serverless data warehouse acclaimed for its price-performance and total cost of ownership (TCO).

Beyond generative AI, Databricks and NVIDIA aim to enhance machine learning (ML) and deep learning workloads. Databricks Machine Learning already incorporates NVIDIA GPUs, providing users with pre-built deep learning infrastructure and pre-configured GPU support. This collaboration ensures that users across various platforms can engage in high-performance, single-node, and distributed training for ML workloads.

The expanded partnership between Databricks and NVIDIA is poised to catalyze the development of next-generation data and AI applications. With a shared commitment to innovation and excellence, Databricks and NVIDIA are setting the stage for enterprises to unlock the transformative power of AI, driving quality, speed, and agility in their data-driven endeavors. This collaboration not only underscores the pivotal role of data and AI in the modern enterprise but also highlights the synergies that can be achieved when leading technology firms join forces to push the boundaries of what’s possible.