You can’t draw cards from the whammy deck or interact with it in any way other than by effects that specifically interact with the whammy deck. Effects that affect cards you own not on the battlefield don’t affect cards in the whammy deck.