Uploading Files From a Phoenix App

How to upload files from a Phoenix app to Amazon S3 - a straightforward and ready-to-apply tutorial

Phoenix is known as a smart choice for a Elixir-based web framework, particularly for real-time applications. That’s why we wanted to take a closer look at how to upload files from a Phoenix app to the popular Amazon S3 storage. If you don’t have an Phoenix app you can play with already, don’t worry - we will start our tutorial from the very beginning, explaining how to create a basic app and configure it. Afterwards you will find out how to upload files from the app to Amazon S3. We'll use arc_ecto for handling file uploads and pushing them to S3 buckets. Let’s get stuck into it then!

Configuration

Creating a new Phoenix app takes just four steps:

mix local.hex
mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez
mix phx.new file_uploader
cd file_uploader

If you are creating the app for the first time, make sure your database config is correct - check the `config/{dev.exs|test.exs}` files and look for the `config :file_uploader, FileUploader.Repo` line to ensure it’s good to go.

Let's start by creating a resource that we can do CRUD (Create, Read, Update, Delete) operations on. We will use the `Picture` model (placed inside `Storage` context) in this tutorial:

mix phx.gen.html Storage Picture pictures title:string image:string

Let’s modify the migration created by the generator to include an `image` column:

# priv/repo/migrations/xxx_create_pictures.exs
defmodule FileUploader.Repo.Migrations.CreatePictures do
  use Ecto.Migration
  def change do
    create table(:pictures) do
      add :title, :string
      add :image, :string
      timestamps()
    end
  end
end

Then add all routes for the `PictureController`:

# lib/file_uploader_web/router.ex
defmodule FileUploaderWeb.Router do
  ...
  scope "/", FileUploaderWeb do
    ...
    resources "/pictures", PictureController
  end
end

And now we can migrate the database:

mix ecto.migrate

Creating the first uploader

Let's start by adding `Arc.Ecto` to our dependencies:

# mix.exs
def application do
  [
    mod: {FileUploader.Application, []},
    extra_applications: [:logger, :arc_ecto, :runtime_tools]
  ]
end
def deps do
  [
    ...
    {:arc_ecto, "~> 0.7.0"}
  ]
end

and then install them:

mix deps.get

Now we can create the `Image` uploader for our `Picture` model:

# lib/file_uploader/storage/uploaders/image.ex
defmodule FileUploader.Image do
  use Arc.Definition
  use Arc.Ecto.Definition
  @versions [:original]
end

Then update our `Picture` model to use that uploader. Let’s also add extended basic validation to include images in the default `changeset`:

# lib/file_uploader/storage/picture.ex
defmodule FileUploader.Storage.Picture do
  use Ecto.Schema
  use Arc.Ecto.Schema
  import Ecto.Changeset
  alias FileUploader.Storage.Picture
  schema "pictures" do
    field :image, FileUploader.Image.Type
    field :title, :string
    timestamps()
  end
  @doc false
  def changeset(%Picture{} = picture, attrs) do
    picture
    |> cast(attrs, [:title])
    |> cast_attachments(attrs, [:image])
    |> validate_required([:image, :title])
  end
end

In our `development` and `test` environments we will probably want to store files locally to avoid calling S3:

# config/dev.exs
...
config :arc, storage: Arc.Storage.Local
# config/test.exs
...
config :arc, storage: Arc.Storage.Local
# config/prod.exs
...
config :arc,
  storage: Arc.Storage.S3,
  bucket: {:system, "S3_BUCKET"},
  access_key_id: {:system, "AWS_ACCESS_KEY_ID"},
  secret_access_key: {:system, "AWS_SECRET_ACCESS_KEY"},
  s3: [
    scheme: {:system, "S3_SCHEME"} || "https://",
    host: {:system, "S3_HOST"} || "s3.amazonaws.com",
    region: {:system, "S3_REGION"} || "us-east-1"
  ]

We use environment variables for specifying S3 credentials and bucket settings to keep things safe for the `production` environments.

We also need to tell Phoenix to serve our files:

# lib/file_uploader_web/endpoint.ex
defmodule FileUploaderWeb.Endpoint do
  ...
  plug Plug.Static, at: "/uploads", from: "uploads"
  ...
end

Creating a file upload form

Let's add a file upload control to picture form in our application (remember to add `multipart: true` option):

# lib/file_uploader_web/templates/picture/form.html.eex
<%= form_for @changeset, @action, [multipart: true], fn f -> %>
  ...
  <div class="form-group">
    <%= label f, :image, class: "control-label" %>
    <%= file_input f, :image, class: "form-control" %>
    <%= error_tag f, :image %>
  </div>
  ...
  <%= submit "Submit", class: "btn btn-primary" %>
<% end %>

and modify the `show` template to include the `image`:

# lib/file_uploader_web/templates/picture/show.html.eex
<h2>Show Picture</h2>
<ul>
  ...
  <li>
    <strong>Image</strong>
    <img src="<%= FileUploader.Image.url({ @picture.image, @picture }) %>" width="300" />
  </li>
</ul>
...

Now it’s a good time to start our app to check if everything works as expected.

Run `mix phx.server` and open http://localhost:4000/pictures in your browser.

Uploading multiple files with the same name

By default, we won't be able to store multiple files with the same name because they will overwrite each other. To prevent this behavior, we will prepend a timestamp to the filename before saving it. Keep in mind this won’t prevent from simultaneously uploading files with the same name at a given timestamp, however for the purpose of this tutorial it is enough. For many incoming connections it is something you’ll need to think about.

Let's modify our `changeset` method inside the `FileUploader.Storage.Picture` module:

# lib/file_uploader/storage/picture.ex
defmodule FileUploader.Storage.Picture do
  ...
  def changeset(%Picture{} = picture, attrs) do
    attrs = add_timestamp(attrs)
    picture
    |> cast(attrs, [:title])
    |> cast_attachments(attrs, [:image])
    |> validate_required([:image, :title])
  end
  defp add_timestamp(%{"image" => %Plug.Upload{filename: name} = image} = attrs) do
    image = %Plug.Upload{image | filename: prepend_timestamp(name)}
    %{attrs | "image" => image}
  end
  defp add_timestamp(params), do: params
  defp prepend_timestamp(name) do
    "#{:os.system_time()}" <> name
  end
end

Here, we are doing some fancy pattern matching to extract a file name from the parameters structure and then append it back to the original `attrs` map.

Ensure files are cleaned up after running tests

To clean up the files created during test runs, let’s modify the `arc` configuration to store them in a separate directory, which we can then clean up safely without affecting existing files.

# lib/file_uploader/storage/uploaders/image.ex
defmodule FileUploader.Image do
  use Arc.Definition
  use Arc.Ecto.Definition
  @versions [:original]
  def storage_dir(_version, {_, _}) do
    if Mix.env == :test do
      "uploads/test"
    else
      "uploads"
    end
  end
end

The code above will ensure that files uploaded during tests are saved inside the `uploads/test` directory.

Let’s also create a simple helper module to clean up files after our tests are finished:

# test/support/file_test.ex
defmodule FileUploader.FileTests do
  def remove_test_files do
    File.rm_rf("uploads/test")
  end
end

Now we can include our `FileUploader.FileTests` module in `FileUploaderWeb.ConnCase` and `FileUploader.DataCase` modules during an `on_exit` callback:

# test/support/conn_case.ex
defmodule FileUploaderWeb.ConnCase do
  ...
  setup tags do
    on_exit fn ->
      FileUploader.FileTests.remove_test_files
    end
    ...
  end
end
# test/support/data_case.ex
defmodule FileUploader.DataCase do
  ...
  setup tags do
    on_exit fn ->
      FileUploader.FileTests.remove_test_files
    end
    ...
  end
  ...
end

If we run our tests (`mix test`) they will fail because of the presence validation of the image. Let’s fix them starting from the controller tests.

We need to provide some fixture file that can be used during tests - just put some `image.png` file inside the `test/fixtures` directory, then modify the `FileUploaderWeb.PictureControllerTest` module that was created by the generator to reference this file:

# test/file_uploader/web/controllers/picture_controller_test.exs
defmodule FileUploaderWeb.PictureControllerTest do
  use FileUploaderWeb.ConnCase
  alias FileUploader.Storage
  @create_attrs %{title: "some title", image: %Plug.Upload{path: "test/fixtures/image.png", filename: "image.png"}}
  @update_attrs %{title: "some updated title", image: %Plug.Upload{path: "test/fixtures/image.png", filename: "image.png"}}
  @invalid_attrs %{title: nil}
  ...
end

We also need something similar for our `FileUploader.StorageTest` module:

defmodule FileUploader.StorageTest do
  use FileUploader.DataCase
  alias FileUploader.Storage
  describe "pictures" do
    alias FileUploader.Storage.Picture
    @valid_attrs %{title: "some title", image: %Plug.Upload{path: "test/fixtures/image.png", filename: "image.png"}}
    @update_attrs %{title: "some updated title", image: %Plug.Upload{path: "test/fixtures/image.png", filename: "image.png"}}
    ...
  end
  ...
end

If we run our tests again (`mix test`) we should go back to green:

$ mix test
....................
Finished in 0.2 seconds
20 tests, 0 failures

And that’s us, over and out for our Phoenix file uploads guide - we hope you’ve enjoyed it! Full code for the tutorial can be found here.

If you’re struggling to complete your Phoenix app, looking to add features, or even want to build a brand new app then think of us at iRonin for all your Phoenix app development needs. We have world-class expertise and will gladly assist you - get in contact with us today.