Python is becoming a primary language for real-time data science and machine learning, challenging traditional Java-based tools like Kafka.
Companies hesitate to adopt real-time streaming due to high initial infrastructure costs, the mental shift from batch processing, and inefficient resource usage.
A common but complex streaming architecture involves using Debezium for change data capture and Kafka as a message broker, which presents DevOps challenges.
Combining Kafka for messaging and Apache Flink for computation creates significant operational overhead, requiring specialized roles and complex infrastructure management.
Modern Python frameworks unify the message broker and stream processor, abstracting away infrastructure complexity and enabling developers to focus on business logic.
Real-time Python pipelines can power various applications, including clickstream analytics, ad enrichment, vector database updates, and anomaly detection alerts.
A step-by-step guide shows how to create a real-time data pipeline using a visual editor, a Python transformation function, and webhooks for integration.
Watch a live demonstration where new data inserted into a Supabase database is instantly processed by a GlassFlow pipeline to predict a price using AI.
Python-native frameworks provide self-sufficiency for data teams, reduce infrastructure management with serverless execution, and accelerate the development of real-time applications.
46:43Bobur Umurzokov
38:50Lucia Cerchie
54:29DeveloperSteve
39:14Alan Mazankiewicz
59:43Paul Graham
43:26Ayon Roy
50:38Radovan Kavický
52:37Kateryna Hrytsaienko
Jobs that call for the skills explored in this talk.
CONTIAMO GMBH
Berlin, Germany
SYSKRON GmbH
Regensburg, Germany
evoila Frankfurt GmbH
Mainz, Germany
