Taylor's Life Log

Showing 2 of 2 posts

programmingAugust 9, 20267 min read

Sorting a File Too Big for Memory

I sorted 2.2 million rows two ways to see a classic computer-science trade-off in practice: use lots of memory and finish quickly, or use a little memory and take the scenic route through disk.

Language: RustDifficulty: beginner

data-engineering algorithms external-sort rust

programmingJune 13, 202614 min read

Stream Joins Without Joining: How Paimon Inverts the Problem

Keeping one always-current row per product, assembled in real time from five systems that never talk to each other, without drowning Flink in join state. How Apache Paimon moves the merge into the storage layer.

Language: SQLFramework: Apache PaimonDifficulty: advanced

flink paimon streaming data-engineering lsm-tree lakehouse

Acadia National Park
Congaree National Park
Cuyahoga Valley National Park
Gateway Arch National Park
Glacier National Park
Grand Teton National Park
Great Smoky Mountains National Park
Hawaii Volcanoes National Park
Indiana Dunes National Park
Mammoth Cave National Park
New River Gorge National Park & Preserve
Olympic National Park
Shenandoah National Park
Virgin Islands National Park
Yellowstone National Park
ai
algorithms
camping
claude
cli
data-engineering
deno
external-sort
flink
games
hiking
lakehouse
lsm-tree
national parks
nodejs
paimon
rust
streaming
typescript
wordle

Filters

Sorting a File Too Big for Memory

Stream Joins Without Joining: How Paimon Inverts the Problem