How do you keep shared libraries used by multiple microservices in sync?

For example, a model class in a shared lib used by some of your microservices. If a new field is added, how would you quickly identify which microservices need to be updated, redeployed? And do it quickly.

Or say the model library has many classes, used by different services. 1 class changed, and only 2/10 services reference it. Do you target only three 2 or so of them? Or would those be bad design?

A bit unsure what you mean...

Usually you should have a dependency management and thus versioned packages of libraries.

Or utilize VCS commit IDs as "versions" (avoid submodules, they'll do more harm than good.)

Or am I misunderstanding the issue at hand?

@IntrusionCM
Library has data models Car, Person, House to represent records in their respective tables in a database.

And you have 5 services and say they're names are obscure so A, B, C, D, E
2 use Car
2 use Person
1 uses House

Now you need to add some fields into the Car model so update the library.

You need to update the services that use Car. Don't really care about the others.

And well unless you inspect all the code for references, you wouldn't be able to tell.

You wouldn't typically have the situation of having to manually keep shared library versions in sync across microservices...

The only real thing that micro services share is the public API definition. If and only if that explicitely changes do you have to update another service that depends on that... and then it's pretty obvious that you should update the shared library as well (if you depend on that)

@12bitfloat but say each Microservices if responsible for adding something to the final Car that gets returned to the user. Like owner, location. Or like Facebook's user info. Each service that would get the object, do something with it and return it to the next, but all need to know the model structure? speak the same language.

@donuts Yes, but before you would worry about the internal data structures you have the problem of the service API changing -- i.e. suddenly a new required field num_steering_fields was added. So updating the model is more of less implicit when having to accomodate the new API version anyway

@donuts As to how to easily detect that... good question

It's hard to explain as for proper examples I'd need to write a lot of examples.

If you use GIT, you can get the list of modified files from commits or a commit range.

Look at e.g. git diff , especially in conjunction with the porcelain parameter.

You have now a list of _changes_ in all files between commit X and Y.

Now you can utilize grep (even better ag /
https://github.com/ggreer/... ) to search for imports of changed files.

The "messier" the code base, the harder it will be.

The approach can be extended. For fun, I e.g. wrote a wrapper script based on PHP tokenizer that extracted class and function names from the changed files and looked for their usages.

It's highly dependent on your language, of course.

If you e.g. have a file with quadrillions of classes, you're fucked. :)

A manual approach would be simpler, as you can narrow down the changes.

All in all, this is where unit testing would come in more handy...

There is no easy way imho. Static code analysis could help to e.g. detect parser bugs / compile errors / code smell, Unit tests to detect broken behaviour.

@IntrusionCM yes guess units tests would help except if the issue can only be detected using a functional test that to make sure the final Car that comes out is what you expect...

I'm not quite sure how git helps though. Each service would be it's own project so would be in their own repo.

But like you said feels like need to scan all the repos and basically generate a datacube of all imports in each project and when the library changes, get the classes in the lib that changed and use them to add the search params....

@donuts yes.

I usually have all git repos checked out and as such can search in all repos.

Hence ag for multi threaded search, it's really a great tool.

@IntrusionCM I'm wondering though if I'm thinking about it wrong. Structuring projects libs the old way where the number of dependant projects... Usually 1, is manageable manually...

I mean Microservices have been around for a long time now so this should be a solved problem?

@donuts

I think your assumptions are wrong.

There is no "perfect" solution.

Architecture is a lot about compromise.

Let's say we have a monolith.

The monolith consists of 1000 files.

The monolith has one dependency - a bundle which consists of framework and library functions.

Semantic versioning is enforced.

As such, bumping a major version of dependency...

- requires unit testing
- requires static analysis
- requires checking of project for possible conflicts

So far these 3 points are identical to the microservice approach.

The difference:

- a monolith consists of far more files than a microservice
-- a microservice should be as minimal as possible
-- a monolith usually suffers - due to the mass of files - from interdependencies. Changing one class in the monolith requires manufacturing of dependent classes

- first point leads to several consequences

If a microservice breaks, one small unit of work is broken. Others still work and are useable as they are independent.

If a monolith breaks, the whole application is broken.

Microservice is - if done right - the compromise of having small unit of works and thus many projects - vs. one large codebase.

The many projects concept of microservices lead to "more management work" (organization, workflows, ... and of course the stuff you mentioned here) - but the positive outcome is that due to the small project size, it's more maintainable.

Running low on coffee, hope It was understandable?

We struggled with this for our microservices project as well.

We went with a standalone project/repository which the other services would import. It was definitely not ideal as you'd have to update this shared library, then update all the microservices with this new version.

We often discussed alternatives but it's that type of thing that works just well enough to not be a priority. The best thing is definitely to avoid such a dependency in the first place, but that's easier said than done for complex domain models where you want strict type checking.

If we had the time we would probably look more at having some kind of contract/definition set you could point to in runtime instead of having to import libraries and classes. We had some luck with graphql schemas and openapi, but again - just couldn't afford to prioritize it.

Speaking from experience:
You can solve by using a cascade CICD pipeline => when the base lib is changed, trigger a rebuild for all relevant microservices.

My own take on this - is to use a Single Binary for all the microservices. The basic idea is to use a parameter, or env var passed to the binary at run time, to select the mode of running - run as service1, run as service2, etc.
The reasons are very simple: Binary will be smaller. If you use Docker, the image will be smaller. Much simpler to deploy. Faster to build. Easier to sync across the chosen solution.

We use private NPM packages, or private Bitbucket repos we link to as dependencies. Our linters will yell at us if something in the interface has changed

We have the same experience, shared libs turn to dependency hell pretty much everytime.

The best way is to avoid using them and keep the services so small and specialized they don't need to use shared models, but sometimes you realy can't avoid it.

I suspect this comes down to a design issue and there's some magic limit to what can can't be a microservice. Like the separation between your app and oAuth is very distinct and no sharing is required, so It's a good candidate for a micro service but eh... Reality is rarely so clean cut -_-

Im actually kinda intrigued by @magicMirror 's approach... I think I'll try making a PoC of that to see how I like it

@Hazarth In order not to break my Team workflow - I actually use softlinks + examine how the binary was executed in the code as the selector.

@magicMirror not sure what you mean by binary. You put projects in a single project/image? Basically multiple classes with main(). And a master script that can run each?

In a single project though would make it a monolith?

@SudoPurge you have a link to an article, example? Not sure what you mean about linking bitbucket repos.

@donuts
Hmmm. you could say that about my approach.
when building many binaries - you can choose to deliver each in its own docker image, or put all of them in the same huge image. Both approaches have pros, and cons:
1 binary/1 image means a lot of images to update for a major change, but smaller footprint for a minor change. complex dependencies, and versioning. Use this approach if you are deploying multiple Small increaments to production each day.

all binaries/huge docker means an easier versioning/dependency management. much bigger footprint for each change. no differnece between small and big change. use this approach for a small number of updates each month.

My solution is to use a hybrid approach that takes the pros of each (small footprint for deploy, easy depencies and versioning) by having two main() for each microservice, calling the same actual MainMicroService(). You can build each small main, for tests, or build the Big main for deployment.

I don't use scripts.

@magicMirror in that case though, it would just be the startup configs that init a dependency are different but but the "backend logic" and all the jars it depends on are the same.

I guess you could DI different implementations but you can't say 1 main uses log4j v1 and another v2.

@donuts Java. it is always Java.
Java suffers from that special kind of it own DLL hell - with the screams of damnned souls when the Maven POM xml hits them square in the face....

Yup - in the case of Java, that uses different depndency versions across the stack, you are pretty much stuck with the single service/single docker solution.
You can still cascade the CI pipeline though.