I suspect the way to go would be to match the 4 and 5 cubes where you have a central ball that pushes pieces out that are disconnected and they are all tongue and grooved around the central core. I also think given the variance in density with wood, a lot of sanding and some sort of sealant between the cubies. as @pubultrastar said.
I actually own way too many cubes, including an official wood rubik. If I choose to use it I have to work a lot on it as humidity and general crud makes it very hard to turn.
ok my mind just though about the tongues and grooves, and with odd pieces you don’t have a good way to use interlocking to keep the pieces in. either the edge or center will be loose. So you’ll need some sort of a mechanism with axels to turn the central face cubies.