Thats not easily doable.
the easiest way is to provider two application entry points - one for non-domain
users, and a separate app for domain users that converts the windows account
details to a forms auth cookie.
another option is to modify the pipeline and inject some modules that "bypass"
forms auth - this needs some kind of indicator if windows auth should be
used or not - like an IP address range for domain users.
i described both approaches and trade offs in detail here - but feel free
to ask more questions:
Dominick Baier, DevelopMentor