Architecture for Java REST API going to build/port from existing NodeJS one.
So Spring Boot + *

Lots of concurrent requests and large MongoDB calls. Current APIs use like 4GB memory for each instance because they don't use stream/pipe the response. Hold all data in memory and then return it all at once to user.

And well we expect more load in the future, so want to do this the right way.

So my understanding since this morning, is there's the blocking? MongoClient, (find* returns List) and now a Reactive MongoClient which is very async and like JS promises. Based on Pub, Sub model.

But the downside of JS promises was callback hell.

So actually 2 questions.
1. For each request, the db call done using the same MongoClient/db connection such that if there are 2 requests one would block the other?
2. Reactive Mongo would be non-blocking by design so would be better to support streamed responses?

If you do decide to go down the reactive route, then considering you're using Spring the Webflux / reactor stack would be the most logical choice, as it's built right into Spring 5. If that's the case then yeah, you can use the ReactiveMongoRepository and it'll return a Flux, which is a stream that can return 0 to many elements before terminating.

Big upside is that it won't store it in memory, so that problem is solved. Downsides are that the response type will be a stream of Json by default (so a bunch of records separated by new lines rather than a JSON list), so you'll need to decide if that matters or work around it. Reactive stuff in general in Java land is also harder to read, requires a different style of programming, and is a much more "niche" area, so you'll find it hard locating Java devs with an expertise in it in future. Debugging is also a monumental PITA, so I'd think ahead on that front and make sure you have checkpoints marked at strategic locations.

@AlmondSauce ah ok, do you know if you can stream a response even without reactive?

I know PITA since nodeJS, promises are basically reactive and our developers are sorta script kiddies.... Don't write very clean code...

I wrote some streaming in nodeJS too for large responses, but to keep it consistent, sorta hacked the response writer so it prints it as a json array.

Is it acceptable to just return line delimited json or have some functions not streaming so regular json and others that are?

@donuts Not really - you can, but then you just end up writing callback hell style code that's worse than actual reactive code.

Would pagination solve your issues? That way you can return a chunk of results at a time rather than having to keep the whole thing in memory, but you avoid the reactive drawbacks at the same time.

@AlmondSauce we use pagination now but it's not efficient I think because for example there's 100,000 documents in mongo and say page size is 100.

In order to paginate the caller appends a lastId field to the query string. And our mongo queries are all *.sort(id: -1)

And well sometimes the query itself is really complex so asking mongo to run it again every call is expensive especially if the data goes out of memory (lots of page faults)

With streaming, the cursor I believe is left open so the data is never moved out of memory? Although then the issue is what happens when the db runs out of memory for all the open cursors?

@donuts Yeah, there's not much more you can do if you're already using pagination.

Been a while since I've looked at reactive mongo, but I don't think you'd have any memory issues with open cursors if you don't have any with pagination, as it should just perform the lookup then push everything it can to the client (it's then the clients responsibility to do whatever it wants with the streamed results when they arrive.)

It all varies though depending on loads of factors. If this was me then I'd suggest a spike rather than a user story for this sort of work initially - that should enable you to play around with options and profile the DB with a few different approaches to see if it has an adverse affect on memory, then decide accordingly.

@AlmondSauce what's a spike? The only reason I have time to play around with this stuff is technically I'm on vacation this week...

@donuts Ah, just a research / investigation / proof of concept type task rather than final implementation.

Whenever you're making a decision like this it pays to spend some (work!) time looking at the options imho. Though if you have to do it all in your own time that obviously puts a different slant on it!

@AlmondSauce the problem with work time is that a it's structured more like a stack or priority queue... And spikes are low priority until they are paired with an urgent deadline.

And then it goes to the urgent queue.. With all the other urgents...