For every request that comes to IIS, IIS will pass it to the ISAPI filters that are hooked into IIS. This is true for all such filters (SE ISAPI, SE AS, or some other vendor's ISAPI filter-based program). Each ISAPI filter gets to tell IIS "this request is for me, don't hand it off to any subsequent filters that come after me", or "this request is not for me, pass it to the next filter". This can be thought of as the request-filtering phase of the request handling process.
If the SE Filter decides that the request is meant for SE, it will make an ISAPI callback to perform an internal redirect (not a round trip to the client) to invoke either ServletExec_ISAPI.dll (For SE ISAPI) or ServletExec_Adapter.dll (for SE AS). The Virtual Directory which points to where that DLL resides is used for this internal redirect.
In this way the DLL is invoked again... only this time it runs in a different "mode". It can be thought of as the request-servicing phase.
This is where the native DLL code passes the request info to SE's Java code (running inside the JVM) which then builds the response that the webserver then sends to the client.
Here is some more detail on the Virtual Directory:
With the default IIS website, the ServletExec filter looks for the very FIRST virtual folder/directory that points to the physical folder containing the ServletExec DLL and uses that. Normally this is the virtual folder named "Scripts".
When using ServletExec with subsequent IIS virtual Servers they must have a Virtual folder that points to the physical folder containing the ServletExec DLL, AND that virtual folder must have the same exact name as the virtual folder in the default IIS website. So for example if you have more than one virtual folder defined in the default IIS website that points to the folder containing ServletExec DLL, then the one that is listed first is the one that is actually being used by the ServletExec filter. Therefore you would have to create a Virtual Folder in the 2nd IIS virtual server having the same name in order to use ServletExec with that 2nd IIS Virtual Server.