I finally managed to get my Wireguard setup to work in both ways! Beforehand I could ping from A to B, but not the other way around.

A network 10.1.0.0/16
B network 10.2.0.0/16
(both actually use multiple /24 subnets, but I reserve a /16 for each site for the sake of simplicity)

Lots of fiddling later this is my configuration:
A interface 10.1.199.1/32
A allowedIPs 10.2.0.0/16
B interface 10.2.199.1/32
B allowed IPs 10.1.0.0/16

ping from 10.1.1.1 to 10.2.1.1 => 172ms
ping from 10.2.1.1 to 10.1.1.1 => 172ms

it works, yay! now to add more sites...