It seems Java on Linux runs slower than Windows... Same code takes about 2x longer to run. And well this code is for a batch job...

Does anyone have experience with JVM tuning/optimization or can confirm/disprove this is the case?

Which java? OpenJDK or Oracle Java?

@Demolishun Oracle

I seriously doubt JVM is causing the difference since most Enterprise Java apps run on Linux

Have you profiled the app on Linux to see what seems to be going on?

@asgs how to best do that? Most of the good ones are paid?

But yes how can the same code be 2x slower on an Enterprise Linux server vs my windows dev PC...

@billgates ok, there are a few things to clarify first. What is the parameter you are using to detect it is slower or faster? Are the machines and JVM settings exactly the same? Are the use cases identical when you are measuring the said parameter? What does the app do? Is it I/O intensive or CPU/Mem intensive? Is the behavior consistent or are there certain conditions that cause this?

As for profiler, the free one, JvisualVM, is good enough. If you are looking for commercial ones, YourKit is better and they do offer a trial version

@asgs I run the job in eclipse in Windows vs Linux. Job takes 20 minutes in Windows, 40mins in Linux.

One thing though I just remembered, the server machine is Redhat 5 which is like really old? but it's running Java 8...

Maybe that's the problem? Plus I think Eclipse used a special version of Java though I'm pretty sure I have it set to load Java binaries from a path.

@billgates I think neither the OS nor the Java version is making a difference here, although we can't rule it out altogether. Yes, Eclipse comes with its own Java Compiler, but there's nothing fancy about it

So, what does that Job do? Can you add loggers to see where that extra 20 min is spent? Also, are you sure you are not running the job twice on Linux? There's something we are totally overseeing here

@asgs yes I tried that but it seems both read to DB and write are slow... But I also checked the distance and stuff and well the ping times from my machine and our servers to DB are similar. So doesn't explain how there's a 2x diff.

I guess I'll test with a simple app but not sure what to put in it.

Basically Do X 1000, time how long it takes.

@billgates ok, so DB calls seem to be the bottleneck here. Comment out all the DB calls. You should now see same job times on both the OSs. If so, that eliminates JVM from the list of suspects

Now, on to DB calls, are the data read or written exactly the same? If not, the queries from Linux might not be using the DB indices the queries from Windows use. If they are same, as you said, write a test app which does only the DB stuff and see how it pans out

@asgs it's seems so but need to test the jvm I guess first. What's a good call to do? That's takes a good amount of time but also consistent?

I read something like getTimesInMillis takes longer on Linux which maybe the problem too...

Wait.. What is the hardware behind both linux and windows? I find it very hard to believe :)

as for profilers -- jprofiler is quite nice. And free. Yourkit is awesome but only gives a 14days trial

@netikras well the Linux is RH5, but seems now have to break it down to the basics... Wire a Java app to test OS, OS Version, MongoDriver seperately

@netikras @asgs I dunno maybe if this is a bad test but just wrote some code using BigDecimals to do factorial(10^5)

Takes 7s but the final product is blank (# to big I guess)... So not sure if it's any good....

Currently desktop beats server by .5s...

@billgates You are beyond comparing apples to oranges. In your case it's more like apples and pumpkins. Making such comparisons with THAT many different variables is no way to draw any conclusions.

@netikras ok how about writing selection sort on an array with 400 000 elements, in reverse order. Aka there worst possible case for it?

First, you are comparing different operating systems.
Second, are you comparing the same java version? The same build?
Third, you are comparing VERY different hardware. Very different grade, class of hardware!
Fourth, I guess the server is a virtual env.?

I did not ask about OS version. I asked about the hardware for a rason.

@netikras ah yea vm .... Maybe. To many did leiffs... But yes it failed selection sort too...

@billgates What did I just say?

Frankly you haven't even answered my question about the hardware. But since you are saying it's a server, I'm assuming you are running your tests on a server (RH5) and a desktop (Windows 10).

No matter how nice of a test you craft out, your test runs will be INCOMPARABLE if you run them on your server and desktop.

A bonus note: do not expect servers to be more performant than desktops. Servers are supposed to be a fortress: reliable, stable, 24/7, HW error prone as much as possible. Servers cost loads of $$ and new releases tend to come out quite rarely.

Now compare this to desktops, where every bum can buy a newer hi-end piece of hardware every week for almost a few pennies. My point is that PCs are VERY cheap compared to servers when it comes to comparing power. So naturally you will most likely have better test results on your desktop!

If you do in fact stop and think for once and compare your server's and PC's hardware (CPU: cores, threads, cache sizes, freq, built-in features easing particular types of calculations; DIMM: freq/CL ratios), you might find the answer to why you have the results you have.

And then add the fact that you are losing calculation time to (at least one) hypervisor!

@netikras only other test left then would be to try running it on a Windows server? But wait actually all or machines are now virtual workspaces so yes... Already got that covered too...

@billgates
```
But yes how can the same code be 2x slower on an Enterprise Linux server vs my windows dev PC...
```

```
But wait actually all or machines are now virtual workspaces so yes... Already got that covered too...
```

Smth does not add up.

Same sort, same version of virtualization engine? On the same hardware? If yes - then sure, go ahead. If NOT - your results will not be able to confirm nor deny anything.

@netikras yes seems it's the hardware and maybe Linux os. Just ran this code on a different Linux vm, newer server, 30% faster and also beat the desktop.

@billgates See? You are still left with MAYBEs. Your testing environemnts/conditions are wrong if you cannot draw conclusions out of your test runs.

And yes, I am certain it's the hardware :)
Spin up a windows VM on the same hypervisor to have decent comparison. ASSUMING (!!!) you are not running on a shared environment, where Jack or John runs some batch jo on the same physical machine and can affect your test results.

Average out of 3 seems a fair bet. So that's 6 runs: 3x on a Linux VM and 3x on a Windows VM, o the same physical machine, same hypervisor, exact same software version (JVM) with all the same runtime parameters. All your tests (if there are more than one) should also be run in the same order.