Edit: As several commenters on reddit and hacker news pointed out, dokku provides a way to pin down a specific version of a buildpack. The maintainer also indicated that dokku wont download a new buildpack version by itself, for that you need to get a new version of herokuish. In our case, it was the use of a multi-buidpack addon (not provided by the dokku project) which downloaded a new version on deploy. Dokku is clearly not at fault here. In conclusion: always keep control on all your dependencies, pin down versions, and review every changes before applying it.
At puka, we use dokku to deploy our application on internal servers. Its fast and efficient, we just need to push the current dev branch to a git remote and everything is taken care of. NPM and Bower will download all our front end dependencies and maven will manage our backend code.
As we are nearing completion on the new version, I pushed the code to the remote and waited for the docker image to spin up. The first part went without a hitch, however the application didnt boot up.
I checked the logs and interestingly it failed early, unable to connect to the postgresql database (another docker image, managed by a dokku addon). The exact exception was
org.postgresql.util.PSQLException: The server does not support SSL
Pretty strange given we didnt change anything in our database configuration. I checked out the low-level code and for some reason, it was trying an SSL connection even if we didnt have any flag for it activated anywhere.
While trying to find a solution, I decided to deploy our earlier version, which was working correctly. To my surprise, I got greeted by the same error. A version that was running up to this point couldnt be deployed again. Before going to the bottom of this, I managed to change our configuration to explicitly disable SSL, and the application connected correctly to the database.
Obviously, something had changed somewhere which was causing our application to connect over SSL by default. The previous version wasnt working anymore so it was external to the application itself. After some searching, I thought that the buildpack, scripts that dokku uses to build and run applications, might have changed.
Sure enough, two weeks earlier a small commit was made, fixing a typo to enable the postgresql addon on the java buildpack. It turns out that this addon will, as I found out in the heroku docs, enable SSL globally on postgresql connections made from the application.
This is the dokku trap : by depending on external scripts to build an application, you cannot be sure a given version will be deployed exactly as before, and you are left to deal with the consequences. I guess well go back to good old tomcat for now.