This would need to allow the user to select a DLL and then through reflection allow them to select methods to run.
Console app would be configurable via the command line or a configuration file.
Both should save history of tests run (using the DLL version to keep history) so that older version can be compared to newer versions of the same method as you develop it, instead of making 2 versions yourself just for comparisons.