It’s been a busy week
US will have Greenland ‘one way or the other’, says Trump – Europe live
Trump tells Cuba to ‘make a deal’ or face the consequences
Homeland security sends more agents to Minneapolis as protests erupt in US
There is a developing consensus that this is the tail wagging the dog: US justice department has released less than 1% of Epstein files, filing reveals
Tasks
- Trash – done
- Look through the bank stuff and see if there is enough to open an account – completely forgot
- Progress on getting the Alienware set up as a Linux box. I also asked them how much RAM they could stuff in since that seems to be an issue these days for me
SBIRs
- Work on loading arrays.
- See how big all the files are. Iterate over all the pkl files but don’t keep anything, just increment the memory size value and the number of vectors. create a list of dicts
- And the answer is: 262,626,464 bytes, in 30,556,975 vectors
- Sort the list by memory, and try loading up all the small ones until 14GB is passed. See if that works. If it does, use those to create a mapping
- Based on the overall size of the pkl footprint, determine an optimal subsampling strategy – looks like 1:50 ratio. That’s not bad
- See how much it would cost to use a bigger box – at least $110/hr. Or I could get a box for about $15k that could handle this. It would pay for itself with a week of compute. Hmmm
- Maybe try the NN approach? Possibly in steps until the array is the size that can fit in memory? Talked to Aaron about this. Some neat ideas.
- See how big all the files are. Iterate over all the pkl files but don’t keep anything, just increment the memory size value and the number of vectors. create a list of dicts
viztales 