DEV Community

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Building a Real-Time Crypto Data Pipeline with Debezium CDC
aineahsimiyu profile

Building a Real-Time Crypto Data Pipeline with Debezium CDC

#python #kafka #cdc #dataengineering
Comments
5 min read
Undestanding Kafka Lag, Why It Happens and How To Fix It.
mutindakioko profile

Undestanding Kafka Lag, Why It Happens and How To Fix It.

#kafka #programming #datascience #dataengineering
2
Comments
4 min read
The Myth of Distributed Computing as a Silver Bullet for Big Data
esproc_spl profile

The Myth of Distributed Computing as a Silver Bullet for Big Data

#discuss #dataengineering #performance #architecture
5
Comments
10 min read
The State of Apache Iceberg, Polaris, and Arrow: November 5-11
alexmercedcoder profile

The State of Apache Iceberg, Polaris, and Arrow: November 5-11

#opensource #database #dataengineering #data
Comments
5 min read
Understanding Kafka Lag: Why It Happens and How to Fix It
oliver_samuel_028c6f65ad6 profile

Understanding Kafka Lag: Why It Happens and How to Fix It

#dataengineering #monitoring #performance
Comments
4 min read
Right Approach to JSON Log Analysis: A Hands-on Guide to Efficient Practices with Alibaba Cloud SLS
observabilityguy profile

Right Approach to JSON Log Analysis: A Hands-on Guide to Efficient Practices with Alibaba Cloud SLS

#analytics #cloud #tutorial #dataengineering
Comments
7 min read
Understanding reasons behind Kafka lag and how to minimize it.
robbin_murithi_f75005db58 profile

Understanding reasons behind Kafka lag and how to minimize it.

#dataengineering #monitoring #performance
Comments
3 min read
REST API Calls for Data Engineers: A Practical Guide with Examples
chaets profile

REST API Calls for Data Engineers: A Practical Guide with Examples

#discuss #rest #dataengineering #json
1
Comments
3 min read
Why Idempotence Is So Important in Data Engineering
chaets profile

Why Idempotence Is So Important in Data Engineering

#architecture #dataengineering #systemdesign #etl
Comments
6 min read
Reducing Consumer Lag in Apache Kafka
aineahsimiyu profile

Reducing Consumer Lag in Apache Kafka

#kafka #dataengineering #python #eventdriven
5
Comments
3 min read
Analyzing and Optimizing a Parquet ClickHouse Ingestion Pipeline
mohhddhassan profile

Analyzing and Optimizing a Parquet ClickHouse Ingestion Pipeline

#dataengineering #etl #clickhouse #python
3
Comments 2
3 min read
🚀 Day 1: Introduction to Apache Spark
sandeepk27 profile

🚀 Day 1: Introduction to Apache Spark

#spark #python #dataengineering #bigdata
1
Comments
2 min read
Shine in Your Next Data Engineering Interview with Pandas
betoalien profile

Shine in Your Next Data Engineering Interview with Pandas

#interview #dataengineering #performance #python
Comments
10 min read
Writes, 3 ways: Postgres, Apache Kafka® and Apache Iceberg™
celestehorgan profile

Writes, 3 ways: Postgres, Apache Kafka® and Apache Iceberg™

#postgres #dataengineering #learning #database
1
Comments
10 min read
Building Custom MCP Servers with Python: A Data Engineer's Guide 🛠️
qvfagundes profile

Building Custom MCP Servers with Python: A Data Engineer's Guide 🛠️

#dataengineering #software #python #ai
Comments 1
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.