AI startup launches fastest data processing engine on the market.

AI startup launches fastest data processing engine on the market.

Pathway: The Fastest Data Processing Engine on the Market

Pathway

Introduction

In the world of artificial intelligence (AI) and data processing, speed is crucial. Paris-based AI startup Pathway has taken this requirement to heart and has recently announced the general launch of its groundbreaking data processing engine. According to reports, this engine is up to 90 times faster than existing streaming solutions, making it the fastest data processing engine on the market. But what sets Pathway apart from its competitors?

The Secret: Real-time Learning and Adaptability

Pathway’s data processing engine has a unique ability to mix batch and streaming logic in the same workflow, mimicking the way humans learn and react to changes in real-time. Traditionally, the complexity of building batch and streaming architectures has led to a division between the two approaches. This division has slowed down the adoption of data streaming for AI systems, limiting their intelligence to a specific moment in time. Additionally, the inclusion of generative AI workflows has added even more complexity to the equation.

Pathway’s CEO and co-founder, Zuzanna Stamirowska, emphasizes the critical need for rapid data processing and more adaptable AI systems. By enabling real-time data processing and providing developers with a simple experience, regardless of whether they work with batch, streaming, or generative AI systems, Pathway aims to bridge the gap between these approaches and meet the growing demand for flexibility.

Revisions without AI Retraining

In the past, machines had difficulty “forgetting” incorrect or outdated information in real-time because models were trained on static data uploads. Unlearning would typically require the retraining of the entire model. However, Pathway has developed a groundbreaking solution to this problem. They can make revisions to certain data points without requiring a full batch data upload, similar to updating a single cell within an Excel document. Instead of reprocessing the entire dataset, only the cells dependent on the updated data points are processed.

This approach has proven highly effective for Pathway’s clients. For example, German logistics specialist DB Schenker reduced the time-to-market of their anomaly-detection analytics projects from three months to just one hour. Similarly, French postal services company La Poste saw a significant 16% reduction in fleet CAPEX.

Democratizing Data Pipelines

Founded in 2020 by Polish-French duo Zuzanna Stamirowska and Claire Nouet, Pathway is a female-led deep tech startup that has quickly gained recognition in the industry. With a pre-seed round funding of $4.5 million (approx. €4 million) in December of last year, the company has assembled a team of over 20 employees across Europe and North America.

Pathway’s vision goes beyond revolutionizing data processing; they aim to become a “lingua franca” for all data pipelines, including stream, batch, and generative AI. By doing so, they hope to not only cut costs for their clients but also democratize the ability for developers to design streaming workflows. Historically, this task has required a specialized skill set, but Pathway seeks to make it accessible to anyone in the industry.

Conclusion

Pathway’s data processing engine is a game-changer in the field of AI and data processing. Its ability to combine batch and streaming logic, learn and adapt in real-time, and make revisions without the need for full AI retraining sets it apart from existing solutions. With a talented team and a clear vision, Pathway is well on its way to becoming the lingua franca of data pipelines, empowering developers and revolutionizing the industry.