Needs smoke tests

[RobinDaugherty]

#480 and #481 fixed an issue that I introduced by referring to a constant that wasn't loaded yet.

CI didn't fail, even though every app failed the moment it tried to load.

I think the solution here is to add a smoke test that runs an app of some kind (Rails/Sinatra/pure Rack) and make sure the app starts. This actually might be more effective than all of the specs we have, which currently have to cover a lot of different cases (with Rails, each version of rails, etc).

[peaonunes]

That's a fair point. I cannot think in multiple ways to easily validate that, one idea I just had was:

1. Rails app, or anything, with the dependency pointing to the development.
2. A rake task that simply puts on stdout.
3. A spec that navigates to the app and run the rake.

The rake would load the Rails env so if the gem was corrupted in any way the rake would not execute and would fail.
I'm just not 💯 sure on how to make the step 1 to work, but I would be happy to give it a try. What do you think?

[RobinDaugherty]

I think this might be a very simple thing actually. I have various Rails and Rack apps I've built to manually test in. One or more of these could be downloaded in our CI, then:

1. Start the app server in a background process
2. Wait for the app server to listen on the configured port (and time out otherwise)
3. Make a request to known-good and known-bad routes
4. (nice-to-have) make an "internal" Better Errors request to get variables or run a console command
5. Terminate the background process

This could be done using a build stage in Travis, or maybe just tacked onto the existing CI run, running on each combination of Ruby/Rails/binding_of_caller/haml.