Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Is Iceberg involved in every read/write? I thought it was mostly metadata?


DataFile(parquet) is not enough for table with update/delete, (they are part of iceberg "metadata"). for CDC from OLTP use-cases, the pattern involves rapidly marking rows as deleted/ insert new rows and optimizing small files. This is required for minutes-latency replication.

And for second latency replication, it is more involving, you actually need to build layer on top of iceberg to track pk/ apply deletion.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: