5/10 - Scalable Data Access: Scaling Operations TODO | Notion

DB Systems

Non-deduplicating project

Screen Shot 2024-05-16 at 11.37.49 AM.png

Screen Shot 2024-05-16 at 11.38.03 AM.png

Simple SQL

Screen Shot 2024-05-16 at 11.38.32 AM.png

Screen Shot 2024-05-16 at 11.38.47 AM.png

Groupby

Screen Shot 2024-05-16 at 11.39.21 AM.png

Screen Shot 2024-05-16 at 11.39.32 AM.png

if hash table > DRAM size, program will crash
to scale larger groups, divide and conquer

Matrix Sum / Norms

Screen Shot 2024-05-16 at 11.44.02 AM.png

all stores have same I/O cost because we need to square and sum all cells
generally, tiled partitioning is better because it scales both dimensions, more DRAM & Cache efficient

ML Systems