Not a rant, but seeking advice...

Should I abandon 2 years' worth of work on migrating a personal project from SQL (M$) to a Graph database, and just stick to SQL? And only consider migrating when/if I need graph capabilities?

The project is a small social media platform. Has around ~50 monthly active users.

Why I started the migration in the first place:
• When researching databases, I read that for social media, graph is more suitable. It was, at least in terms of query structure. It was more natural, there were no "joins", and queries were much simpler than their SQL counterparts.

• In case the project got big, I didn't want to have to panic-deal with database issues that come with growth. I had some indexing issues with MSSQL, and it got me worried that at 50MAU I'm having these issues, what would happen if I get more?

• It's a personal project, and the Gremlin language and graph databases looked cool and I was motivated to learn something new.


Why I'm considering aborting the migration:
• It's taking too damn long. I'm unable to work on other features because this migration is taking up all my free time. Sunk cost fallacy is hitting me hard with this one.

• In local testing within docker, it's extremely slow. I tried various graph engines (janusgraph, official tinkerpop, orientdb), and the fastest one takes 4-6minutes to complete my server tests. SQL finishes the same tests in under 2 minutes, same docker environment. I also tried running my tests on a remote server (AWS neptune) and it was just as slow. Maybe my queries are bad, but can I afford to spend even more time fine tuning all queries?

• I now realise that "graph = no scalability issues" was naïve of me, and 100% wishful thinking. Scalability issues don't care what database I use, but about how well tuned and configured the whole system is.

• I really want to move on. My tech stack is falling behind and becoming outdated. I'm unable to maintain dependencies.

• I'm worried about losing those 50 MAU because they're essential to gaining traction once I release the platform. I keep telling them about the migration but at some point (2 years later) they're going to get bored I feel.

I guess partially it's a rant because I feel like I shouldn't stop now having spent 2 years on this, but at the same time I feel like I'm heading towards a dead end.

If you made it this far, thank you for reading:)

  • 0
    @Alexanderr Can't afford to spend the time on rebuilding it a third time lol
  • 1
    @bananaerror Optimize your queries. Use indexes and shit. That'll fix the problem.

    The highest volume I've reached is 10K MAU and 45 million webhook entries going into the db every month. MySQL did hold itself up with no visible issues.
  • 1
    @SidTheITGuy Thanks. So you think I should ditch graph? Currently SQL performs well (I fine tuned and set up indexes) my main concern is if it comes back if/when I'm growing.
  • 1
    @bananaerror Learn GraphQL and be ready with it should the day come. I wouldn't ask you to ditch it, dude.
  • 0
    @SidTheITGuy Oh no, I believe there's been a misunderstanding. I'm migrating from an SQL based database to a graph based database, so graphql is not involved (unless this was a typo and you meant graphdb).

    I can't have a graph db version of my server ready because I'd be maintaining two versions (the graph one and the sql one) and that sounds like a nightmare. 😂

    I think I understand what you mean though so thanks for your input nonetheless!

    Edit: unless by "be ready" you meant just knowing it then I'm there! I'm at the data migration stage and wondering if I should proceed or stop.
  • 1
    @bananaerror For the learning curve, I think you should proceed. It will only help you with learning.

    Btw GraphQL = Graph Query Language, how you would query the graph database. I meant it that way.
  • 1
    Move to supabase and have both?
  • 0
    @SidTheITGuy I thought you meant GraphQL the client/server API technology.

    I can't commit to both because time constraints unfortunately.
  • 1
    @emyu10 Thanks! I haven't heard of supabase before, will look it up.
  • 0
    @bananaerror and let us know what you think
Add Comment