Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
@C0D4 i've found that converting some file types to 3 byte per pixel PNGs can end up dropping the size by up to 10% and still being further compressable. I was stress testing my engine with a 64GB VHDX containing most of my malware samples.
1TB isn't enough for this because Python doesn't do memory management well, and Numpy and PIL don't help matters. -
@Parzi Write it in C, ofc. Or Perl. Perl is very fitting.
-
@Root C is too low-level for me to get anywhere in. Perl can eat a fat one (blame the standard Linux rename binary.)
-
@TheCommoner282 healthy skepticism. I like it.
https://github.com/ParzivalWolfram/...
also here's before and after conversion of a decent-size file (I zipped up my floppy images to quickly use as an example.)
Conversion from PNG back to the original file isn't implemented yet but would be simple to add (famous last words!)
so apparently Windows 10 Pro build... 1803? 1806? (somewhere in there) refuses to allow over 1TB of pagefile if there's more than one volume with pagefiles. The system just hangs when pagefile is used if this condition isn't met.
I know because *i'm batshit fucking insane and am trying to cram 64GB of data into a PNG in PYTHON*.
(this might also only apply to me. your mileage may or may not vary.)
rant