A Chonky Problem As mentioned in a previous post, we have a CHONKY 14 GB dataset. This is too large to even load into the memory we have available, so we’ve been working on downsizing the dataset. Earlier, we sampled 5% of the 46M datapoints to get the dataset down to 600 MB in size. Progress! 💪 But there’s still one problem…
Sparse and Spurious
Sparse and Spurious
Sparse and Spurious
A Chonky Problem As mentioned in a previous post, we have a CHONKY 14 GB dataset. This is too large to even load into the memory we have available, so we’ve been working on downsizing the dataset. Earlier, we sampled 5% of the 46M datapoints to get the dataset down to 600 MB in size. Progress! 💪 But there’s still one problem…