Dynamically Configure Your Plugs at Run-time-Alex Koutmos

Intro

In this blog post, we’ll be talking about the various ways that Plugs [1] can be dynamically configured, why you would need to configure your Plugs dynamically, and I also introduce two libraries that I wrote that attempt to solve a particular set of these problems. If you have been a reader of the blog for some time now, this post is a slight deviation from the usual in-depth project based tutorial that I usually present. Hopefully you enjoy this format as well and still learn a thing or two :). And don’t worry…my next tutorial based blog post should be coming out shortly after this one :D. Without further ado, let’s dive right into things!

Enjoying the blog post so far? Follow me on Twitter for more content Follow @akoutmos

What do you mean by “dynamically configuring” Plugs?

Every once and a while on the Elixir forums I’ll see posts pop up asking how to configure Plugs at run time based on some specific requirements [2], or even in popular Open Source projects like the Prometheus Elixir library [3]. There have been several suggestions on the forums that I have come across in order to tackle this problem, but most leave much to be desired from a developer experience perspective. Some of these suggestions that I have encountered include:

Compiling out a particular Plug using compile time checks [4]:

if Mix.env() == :prod do
  plug BasicAuth, realm: "Admin Area", username: "admin", password: "pass"
end

Wrapping existing Plugs with your own Plugs:

defmodule MyApp.BasicAuthPlug
  @behaviour Plug

  @impl true
  def init(_opts) do
    []
  end

  @impl true
  def call(conn, _opts) do
    dynamic_opts =
      BasicAuth.init([
        realm: System.get_env("AUTH_REALM"),
        username: System.get_env("AUTH_USER"),
        password: System.get_env("AUTH_PASS")
      ])

    BasicAuth.call(conn, dynamic_opts)
  end
end

Use the run-time configuration options that are available to you from the particular Plug library (in this particular example Plug.Session uses an MFA to dynamically resolve some config values):

plug Plug.Session, store: :cookie,
                   key: "_my_app_session",
                   encryption_salt: {MyApp.Secrets, :fetch_encryption_salt, []},
                   signing_salt: {MyApp.Secrets, :fetch_signing_salt, []},
                   key_length: 64,
                   log: :debug

Let’s unpack each solution bit-by-bit and see where the approaches are valid and where they fall over. The first solution gives you the ability to compile out certain Plugs depending on the MIX_ENV that is available at compile time. In other words, once the application is compiled, there is no way to re-enable the Plug in question. This may work in some scenarios but may be less than ideal in others. For example, in a lot of the projects that I work on, we build releases of the applications (using Mix Releases for example Multi-stage Docker Builds and Elixir 1.9 Releases ) and lower-level deployed environments will have some tooling enabled, while higher level environments will not. For example, in a development or QA deployed environment perhaps Swagger docs and a Server Timing Plug (like the one I wrote https://github.com/akoutmos/server_timing_plug) are enabled, but in production they are disabled. This requires some sort of run-time configuration, as opposed to compile-time configuration given that one build artifact is portable across various environments and is configurable based on Environment Variables (this is very much in the spirit of the 12 Factor App https://12factor.net/config).

Example two on the other hand is a bit tedious given that you need to wrap existing Plugs with annoying boilerplate every time that some use case specific configuration is needed. In addition, I have seen people fall into the trap of performing these impure configurations in the init/1 function which can lead to some very interesting bugs at compile time given that :init_mode is generally set to :compile in a MIX_ENV=prod context https://hexdocs.pm/plug/Plug.Builder.html#module-options. This means that while you are developing, this will work as expected, but once you go to build the artifact, the results of your init/1 function will be used forever. I.e if you call System.get_env/1 in your init/1 function, whatever value was in that environment variable at build-time is what will be used at run-time. To solve this problem, you need to ensure that all of you impure calls are made in the call/2 function as that will be invoked on every request (pretty much as you see it in the sample code snippet).

The third example is the most ideal scenario, but unfortunately, it is also the scenario that you rarely find in the wild. Most of the time, Plug libraries that you encounter will either:

Not allow for run-time configuration of options,
Will have an opinionated way of configuring those Plugs that does not align with your usage.
You are only able to configure some of the options at run-time while others are configured at compile-time.

Of these problems, the lack of exposed run-time configuration for particular options is probably the biggest problem.

Introducing Unplug and Replug!

In order to solve the problems with examples one and two (and some other issues I have yet to mention), I wrote the libraries Unplug and Replug! If the Plug library I am using supports either Application config or an MFA tuple as an option value I will use that without question as it is the cleanest solution.

Unplug is used is used to conditionally execute any arbitrary Plug. This conditionality can range from anything contained within the current request’s %Plug.Conn{} struct, to Environment Variables, to Application configuration. How the Plug is conditionally executed is completely up to you. One very common use case for this (and something that I use in pretty much every single production application) is the ability to filter out logs depending on the incoming request. Specifically, I rarely want to output Phoenix logs when hitting endpoints that are only used for operational purposes. The endpoints that I generally want to filter out logs for are my healthcheck and metrics endpoints. That would like something like so with Unplug:

plug Unplug,
    if: {Unplug.Predicates.RequestPathNotIn, ["/metrics", "/healthcheck"]},
    do: Plug.Telemetry

Your operations & DevOps folks will thank you a million times over given they no longer need to write Regex rules to filter out logs via FluentBit/Fluentd/Logstash/etc. This eliminates cruft right at the application layer. Unplug also provides a wide range of pre-baked predicates (https://github.com/akoutmos/unplug#provided-predicates) but you can also write your own using the Unplug.Predicate behaviour. All in all, Unplug elegantly solves problem #1 from the previous section list as we can conditionally execute our Plugs and solves #2 as we no longer need to create additional intermediary Plugs to filter out Plug actions. In addition, it also preserves the semantics of :init_mode given that the if and else Plugs have their init/1 functions computed at compile-time to ensure the best performance possible.

Replug on the other hand solves the problems in #2 and #3 that Unplug is unable to solve. Replug allows you to configure a plug on the fly. Think of it as solution #2 without all the boilerplate of writing your own wrapper Plug. Unfortunately, Replug is unable to store the results of your end Plug’s init/1 function as Replug does not maintain any state anywhere, but for most cases this should be more than sufficient and allows you to avoid rolling your own Plug abstraction or even forking the original Plug library. To use Replug for example to run-time configure Corsica’s max_age you can do something like so:

# ---- router.ex ----
plug Replug,
  plug: {Corsica, expose_headers: ~w(X-Foo)},
  opts: {MyAppWeb.PlugConfigs, :corsica}

# ---- plug_configs.ex ----
defmodule MyAppWeb.PlugConfigs do
  def corsica do
    [
      max_age: System.get_env("CORSICA_MAX_AGE"),
      origins: System.get_env("CORSICA_VALID_ORIGINS")
    ]
  end
end

The :plug option provided to Replug can take a tuple where the first value is the Plug you wish to execute along with the options to that plug that are static (if your plug takes no options you can provide just the module name with no tuple). The :opts option takes a tuple with the form {Module, :function} so that Replug knows what function to invoke in order to get the required configuration. With that in place you are good to go and your run-time specific configuration will be invoked for each request. In this particular example, Corsica does support run-time config for :origins but not for :max_age. But given we require run-time config for both, that responsibility has been delegated to Replug.

If you want to use the built in Config providers in Replug (there are configuration providers for Application Environment configuration and Environment Variable configuration) you can do something like so and avoid writing your own module:

# ---- router.ex ----
plug Replug,
  plug: {Corsica, expose_headers: ~w(X-Foo)},
  opts: {Replug.Configs.EnvVar, :resolve, [max_age: "CORSICA_MAX_AGE", origins: "CORSICA_VALID_ORIGINS"]}

The difference here being that the :opts option takes a three element tuple with an Enumerable as the third argument that is extrapolated out via the built in Replug environment variable configuration module. This will replace all of the strings with their System.get_env("CORSICA_*") values.

Closing thoughts

Thanks for stick with me till the end and hopefully you enjoyed to slightly different blog format :). I am hoping you picked up a couple of cool tips and tricks along the way. To recap, if you ever in need of conditionally executing Plugs at run-time, give Unplug a look. If you are ever in need of a library that can resolve Plug options at run-time, reach for Replug. If the Plug you are using has all the functionality you require exposed via its own options, then use that directly.

Feel free to leave comments or feedback or even what you would like to see in the next blog post. Till next time!

Enjoying the blog post so far? Follow me on Twitter for more content Follow @akoutmos

Additional Resources

Below are some additional resources if you would like to deep dive into any of the topics covered in the post.