You should use an interface instead of an abstract class:
public interface IPlugin {
void Start();
void Stop();
}
There are many reasons for this, as detailed in https://stackoverflow.com/questions/56867/interface-vs-base-class
I would also separate the plugin logic out of the form, and create a "PluginResolver" class to handle these concerns (This is where you hide the ugly reflection stuff you don't want to touch again).
In terms of the rest of the code, it's pretty basic, so not much to add.
In terms of creating a "plugin architecture" it's kind of a waste without using dependency injection (or a dependency resolver of some kind), because there's no obvious way for plugins to communicate with each other, unless they get coupled together (which defeats the whole point of the plugin architecture), instead by having a set of root services that can be injected into the particular plugins, you can have a disconnected way of giving them all access to say "the primary error logger".
