"Go play on another server" is an off topic statement because we are discussing what is best for the game and what is more fun or less fun game design wise. Saying "it is part of the game" when the game is in alpha and subject to change is utterly vapid.
The interesting part of the crafting system is surviving in this open PvP full loot sandbox game as you harvest materials. The interesting part is finding recipes, whether by killing zombies, raiding rad towns, killing people to research their loot, or fighting over supply drops. The interesting part is not pressing "Go" on the craft screen. The interesting part is not waiting for a progress bar.
The bottom line is that having to play a different video game or do something else while I craft is strictly a negative on the game, making it a worse game than it would be otherwise. An important case study is Darkfall.
This game will do better, have more players, more action and get you (developers) more money if you make the game more fun by removing crafting timers.
There is one consideration, and that is "crafting in combat". Nobody would do this because it is stupid, so it's not really something to worry about. But, there are conceivably some abuses to instant crafting a house on the go or something. This can be 99% negated by requiring the workbench for more things. Remember there is no status quo to fervently defend, so they should implement what has the fewest downsides and most upsides, and that is hands down making crafting times super fast.
I find it hard to believe that with all things being equal, players won't play this game because of crafting times. Not all game mechanics are implemented to be 'interesting', some are there to create 'balance.'
I don't find whacking a log pile 5-6 times to get wood 'interesting', but I think it'd be lame if you could just hit it once and get all the wood.
There aren't many things that I craft that I feel 'this is taking way too long', it's only when I am crafting a large number of items. This is good for balance is well. If you are stockpiling metal and gunpowder for a metal house and c4, I don't want you to turn them into grenades in 2 seconds to throw at me as soon as you hear the tick-tick-tick of your door about to be blown off.