Decouple abstract class details from extended implementations

I have a PluginClassLoader which is an abstract class that provides 99% of the functionality for my project's class loaders. I have 2 subclasses (ServiceClassLoader, and ChannelClassLoader) which extend PluginClassLoader and are little more than wrappers plus some customized logging. 99% of the logic is the same between the two implementations.

I then have a PluginManager which is also an abstract class which has 2 implementations that extend it (ServiceManager and ChannelManager) which are wrappers plus customized logging and a more convenient constructor.

The trouble I'm having is, in my PluginManager is must be able to instantiate a new class loader type, of either ServiceClassLoader or ChannelClassLoader. I'm trying to avoid having my PluginManager coupled to it's current implementations (ie. I want the flexibility of being able to add future implementations but not change PluginManager's logic), so trying to avoid passing in some Enum and using some:

if (classLoaderType instanceof ClassLoaderType.SERVICE) {
    // do logic for instantiating ServiceClassLoader
}

Sample class hierarchy:

public abstract class PluginManager {
    // logic for managing plugins and when to load them
    // ...
    // somewhere deep in a loadPlugin(final File directory) method
    pluginLoader = new PluginClassLoader(); // <-- not valid, can't instantiate 
                                            //         an abstract class, 
                                            //         and it's of the wrong type!
}

public abstract class PluginClassLoader extends URLClassLoader {
    // class loader logic
}

public class ServiceManager extends PluginManager {
    // wrapper for PluginManager with some customized logging
}

public class ServiceClassLoader extends PluginClassLoader {
    // wrapper for PluginClassLoader with some customized logging
}

Trying to avoid doing something like:

public abstract class PluginManager {

    private final PluginType pluginType;

    public PluginManager(final PluginType pluginType) {
        this.pluginType = pluginType;
    }

    // logic ...

    // somewhere deep in the loadPlugin(final File directory) method
    if (pluginType instanceof PluginType.SERVICE) {
        pluginLoader = new ServiceClassLoader();
        // more logic
    } else if (plugintype instanceof PluginType.CHANNEL) {
        pluginLoader = new ChannelClassLoader();
        // more logic
    }
}