Why are you trying to multithreaded c++ file i/o? If you can't write c++ code that's faster than your hard drive, please just don't write c++.

Literally no complex calculations, just some insane string formatting.

I could see writing multithreaded c++ disk i/o because disk i/o IS slow.

What is the context here?

@Demolishun if i/o is the bottleneck, then you can’t make it faster by doing it in parallel.

But I don’t see how it applies to c++ because it’s true for any language.

@Lensflare but if you are waiting on i/o you can make your main code slower. Which is the only reason I can see doing that.

This is a tough topic. Depending on the underlying tech it might be beneficial to do async/multithreaded io. There are a lot of fancy scheduling and caching techniques going on in decent operating systems like Linux. Same goes for RAID and SSD controllers. Networked storage is another beast again.

That said using multiple threads can cause a serious slowdown too. Especially if the process is sequential in nature the only thing you are doing is making it harder for the processor and OS etc.
If performance is that important benchmark and tune. If not a python script will probably suffice.

Y'all are better engineers than the ones I'm working with.

There's some partly multithreaded c++ computations.

Then once that's done, a load of threads are spun up (16). Each writes to its own file. No computation done, but some janky slow string conversions. Then once that's done it's all combined into a single output file by a single thread.

IT'S WAY WORSE THAN YOU ALL THOUGHT!

@atheist Your explanation makes sense why you would question their approach. But the upvotes are mostly for using the word "janky". There is just a ring about that word that I cannot explain.

@atheist hahaha yeah I did not expect that. Writing twice with 16 threads (that is a shitload or context switching for io operations by the way) is never going to be as fast as writing once in a single thread in the slowest language out there.

@atheist I'm pretty confident in saying that the dev who wrote this thought: peeeeerfffoooormmmaaannnceee.

:)

The usual "over engineering a problem without understanding the solution"

Are the intermediate files written to an in memory filesystem at least? If you are waiting on this crap you can at least ensure that and enjoy a serious performance boost.

@hjk101 nooooppppeeeee. I just ripped it all out and will roll with it if someone complains.