Managing Different Environment Settings In Clojure With Leiningen

When developing applications in Clojure, one thing you might need is to be able to somehow keep different settings for each environment. For instance, you might want to be able to have different database settings for each environment.

This can be achieved with leiningen profiles:

:profiles {:dev-overrides {}
              {:dependencies [[com.cemerick/austin "0.1.5"]
                              [com.cemerick/piggieback "0.1.4-SNAPSHOT"]
                              [ring-mock "0.1.5"]]
               :plugins [[com.keminglabs/cljx "0.5.0"]
                         [com.cemerick/austin "0.1.5"]
                         [lein-cljsbuild "1.0.3"]]}]
             :test-overrides {}

Assuming you have that :profiles sub-map somewhere in your project.clj map, it will override what’s in the root of the project map.

For instance:

(defproject ring_to_curl "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url ""
  :license {:name "Eclipse Public License"
            :url ""}
  :dependencies [[org.clojure/clojure "1.6.0"]
                 [cheshire "5.4.0"]]
  :profiles {:dev
             {:dependencies [[com.cemerick/austin "0.1.5"]
                             [com.cemerick/piggieback "0.1.4-SNAPSHOT"]
                             [ring-mock "0.1.5"]]
              :plugins [[com.keminglabs/cljx "0.5.0"]
                        [com.cemerick/austin "0.1.5"]
                        [lein-cljsbuild "1.0.3"]]
              :test {}}})

Notice how there’s a :dependencies key in the root of the project map and there’s one in the :dev profile.

What this means is whatever is in your currently active profile in your [:profiles :] will be merged into the root project map.

Therefore, you can merge in dependencies like for say, browser REPLs in your dev profile. Merge in testing related dependencies in your test profile and so forth.

That’s the simple case of leiningen profiles. There’s a lot more you can do in the project.clj file.

Say I was using some library for databases and it was reading the database information from the project.clj map. I could then specify different databases for different environments.

I could have one for development, one for testing that gets rebuilt every time I run tests and so forth.

And this is fine for a single developer team.

What about for teams with multiple people?

This scenario comes up a lot more often then you would think. You could have a coworker that has a mysql database installed locally and another one that has their database installed in a docker container. They would have different connection information, usernames, and passwords.

In this scenario, one solution is to use what I call leiningen overrides.

There exists a file called profiles.clj in your ~/.lein/ directory. You might’ve played with it before if you’re an emacs user to add the cider nrepl middleware to it.

This profile gets loaded for every single leiningen project you work with.

You can override what’s in this folder by going to your project folder, the one that includes project.clj, and create another file named profiles.clj.

That profiles.clj in your project will take precedence over the one in your ~/.lein/ directory.

Also, notice that the map specified in the ~/.lein/profiles.clj file has a :user key (usually). That corresponds to the profile leiningen is using, which defaults to :user.

My ~/.lein/profiles.clj looks like this:

{:user {:plugins [[lein-ancient "0.6.2"]
                  [cider/cider-nrepl "0.9.0-SNAPSHOT"]
                  [lein-create-template "0.1.1"]]}}

Again, just like before, this map gets merged into the project.clj one. The :plugins key isn’t some random key that is specific to the ~/.lein/profiles.clj file. It appears in your project.clj map.

You can add more profiles for each of your environments in your project.clj file or your project’s profiles.clj file.

I make sure every environment my application will run in is defined in the project.clj file and then add overrides in the project’s profiles.clj file.

This is how it’s done:


:profiles {:dev-overrides {}
              {:dependencies [[com.cemerick/austin "0.1.5"]
                              [com.cemerick/piggieback "0.1.4-SNAPSHOT"]
                              [ring-mock "0.1.5"]]
               :plugins [[com.keminglabs/cljx "0.5.0"]
                         [com.cemerick/austin "0.1.5"]
                         [lein-cljsbuild "1.0.3"]]}]
             :test-overrides {}

There’s a few concepts that need to be explained in this map.

Firstly, by specifying the submap :dev-overrides, which isn’t a special name at all but one I defined myself, I can merge it into other maps in the profiles map.

If you look at the :dev, it’s a vector. The vector has a list of maps that will be merged together. So, by first listing the keyword :dev-overrides, I’m saying, get the value of the keyword :dev-overrides, which in this case is an empty map, and put it here. The next element in the vector is a map, so it merges that with the empty map I had in there originally from :dev-overrides.

With the knowledge that you can merge in maps in the project map, and the knowledge of profiles.clj, you might start to see where I’m going.

Say your project.clj file contains that :profiles map I specified above with the :dev-overrides key. You can now create a profiles.clj file in your project folder like this:


 {:dependencies [[reagent "0.5.0"]]}}

This is a simple, arbitrary case I came up with where I wanted the dependency reagent to be merged in to my :dev profile locally. Normally you might see things like AWS credentials and so forth, if for instance you’re using weavejester’s env library.

If you’re working with other people, what I would do is make a file called profiles.clj.sample or something and commit that to git. Then when a new developer comes to your project, they can just check it out, rename it to profiles.clj and stick in their own settings to be merged in.

Clojure data serialization

After deciding on the stack and framework for a new web application, I tend to ask myself, “What database do I use?”.

And I think that kind of thinking is normal for a lot of people. Many new database technologies are coming out, from SQL to noSQL. Off the top of my head, theres:

  • Mysql
  • Postgres
  • Datomic
  • RethinkDB
  • MongoDB
  • Riak
  • Couchbase
  • Couchdb
  • Redis
  • Hbase
  • … you get the point

But have you ever considered not using a database at all and delaying the decision until you have to scale?

Some of you might be wondering where / how you’d store your data without a database. You can use flat files and store it on disk. Not many people consider this as an option, because we tend to associate dynamic web applications with databases.

Hacker News was originally built with flat files, in case you weren’t aware. It’s possible to get to quite a large scale with just flat files.



  • When you really have to scale up, you would know what real life use cases are and where you actually need the performance. Are you read or write heavy? Is there more activity on a certain “cluster” of data? It’s much easier to scale and make the right decisions with some real world usage instead of what you think might happen.

  • You can achieve a faster speed to market if you delay your database decision for later. There’s less to deploy and less to maintain. You don’t have to be a DBA to manage files on disk.

  • You don’t have to worry about migrations like you would for relational databases. Getting the perfect schema is time consuming, especially in the early stages of a new application that’s rapidly being iterated on.


  • Similar to the cons of NoSQL databases without a schema, you don’t have an enforced schema. The problem with this comes when you add a field to an object or want to change the type of a field. You have to manually enforce and migrate your old data.

  • It doesn’t scale. If you need to scale immediately or think your app will rocket off the ground, this approach wouldn’t work. But the whole point is that it doesn’t scale. Remember, do things that don’t scale.

edn and Clojure

I’m a Clojure developer. In Clojure we have edn. You can think of it as a much more powerful version of JSON.

In edn, you can have tagged elements.

They look something like this:

#uuid "f81d4fae-7dec-11d0-a765-00a0c91e6bf6"

There’s a ‘#’ in front of the name of the tag, followed by the data.

The edn spec includes uuids and insts for free.

Let’s say we had a namespace like this:

(ns literals.core)

(defrecord ContactInfo
    [email address])

(defrecord User
    [username contact password])

(def cb (map->User {:username "cb"
                    :password "password"
                    :contact (map->ContactInfo {:email "[email protected]"
                                                :address "123 Fake St."})}))
;; => #literals.core.User{:username "cb", :contact #literals.core.ContactInfo{:email "[email protected]", :address "123 Fake St."}, :password "password"}

Just as an example, we have a User object with ContactInfo nested within it. Those are just two simple records.

When we create these objects, you can see they’re tagged with the namespace, #literals.core, along with the name of the object.

If you’re not familiar with records, they also work like maps. You can get a field by doing something like this:

(:username cb) ;; => "cb"

Having tagged literals is useful when you read the data back in. Your data is deserialized into the representation you use internally (the record). You gain all of the benefits of using records. For instance, if you had implemented protocols on the record then you can use them instantly after deserializing.

If you had serialized to JSON instead, you would have to rejiggle the data back into the records.

We can also create our own readers for custom tags.

You can add it in data_readers.clj, but I prefer to not make a global change to my readers. We can instead rebind data-readers.

Here’s an example:

(binding [*data-readers* {'our-name/our-type (fn [x] (pr-str x))}]
  (read-string "#our-name/our-type [123]"))

;; => "[123]"

With this in mind, you can create a large map structure with custom readers and writers for your data. And then you can do operations on your structure with clojure’s functions on maps.


  • You can easily write data to the file system and read it back into Clojure data structures.

  • You don’t have to have a database when starting a new application.

Additional Notes

  • You can use Prismatic’s schema library to validate your data. They even have a defrecord macro that allows you to associate a schema onto a record.

  • If you have a large structure that’s difficult to operate on and reconstruct, perhaps take a look at the specter library.

  • Using another language? Take a look at transit-json.

Announcing “Practical REST API’s In Clojure”

Hey everybody!

This is a super early announcement of an ebook I will be writing titled, “Practical REST API’s In Clojure”. 

I’ve been seeing a lot of people asking questions on the best way to approach building web applications in Clojure and figured it’s time for me to write a book on the lessons I’ve learned along the way.

I plan on taking you step by step through building a web application from the ground up. The application we’re building is likely going to be a mini Hacker News / Reddit clone. I figured it’s much more interesting than building an API for a blogging site.

Sign up here to get Clojure content in your inbox and be notified when it’s ready.

If you’re already on my mailing list then you’re already on the list.

Please tweet/share with your friends that may be interested in using Clojure for the web.


Two Clojure Idioms: seq and mapcat

It’s been a little over a month since my last post. But since then I’ve discovered two tiny idioms that I’ve started to use in my work.

The first is the seq function.

Returns a seq on the collection. If the collection is empty, returns nil. (seq nil) returns nil. seq also works on Strings, native Java arrays (of reference types) and any objects that implement Iterable. Note that seqs cache values, thus seq should not be used on any Iterable whose iterator repeatedly returns the same mutable object.

If you ever see a (not (empty? col)) in your code base, it’s a sign that you should be using (seq coll) instead.

The next idiom is mapcat.

Returns the result of applying concat to the result of applying map to f and colls. Thus function f should return a collection. Returns a transducer when no collections are provided

Along the same lines as the seq function, if you ever see an (apply concat (map ...)) in your code, it’s as simple as replacing it with a mapcat.

These two simple changes will make your code more idiomatic and cleaner.


Clojure Talks To Watch

I often come across videos on software engineering and Clojure that are recommended to me by colleagues.

Here are some:

Rich Hickey – Simple Made Easy

For those of you who don’t know, Rich Hickey is the creator of Clojure. This is probably his most famous talk. Definitely a must watch.

Stuart Sierra – Components Just Enough Structure

If you’re building lots of services or a web application, this is the talk for you. Stuart Sierra is the creator of the component library that allows you to keep the state of your application in components. 

David Nolen – Om Next

David Nolen, the creator of Om, a wrapper library for Facebook’s React.js, talks about what’s coming next. If you’re a user of Om or React.js, or interested at all in the topic, watch this.

A Tutorial Of Prismatic’s Schema Library For Clojure

I haven’t had many good ideas for blog posts, but I’ve gotten this bright idea to do tutorials/overviews of the libraries I use in Clojure.

The first one is Prismatic’s Schema.

Schema is an awesome library once you understand it.

Let’s go over some of the more important concepts via the schema readme.

The first thing that jumps out when you first see Schema is that it allows you to validate functions and data. You can think of it as optional type checking in a way.

(def Data
  "A schema for a nested data type"
  {:a {:b s/Str
       :c s/Int}
   :d [{:e s/Keyword
        :f [s/Num]}]})

The above Data schema validates a map. It says the key :a should be a map with keys :b as a string and :c as an integer. Additionally, key :d should have a seq of maps with :e as a keyword and :f as a seq of numbers.

  {:a {:b "abc"
       :c 123}
   :d [{:e :bc
        :f [12.2 13 100]}
       {:e :bc
        :f [-1]}]})
;; Success!

  {:a {:b 123
       :c "ABC"}})
;; Exception -- Value does not match schema:
;;  {:a {:b (not (instance? java.lang.String 123)),
;;       :c (not (integer? "ABC"))},
;;   :d missing-required-key}

Once you’ve defined a schema, you can validate it, by calling the validate function in the schema.core namespace.

If it’s a success, you’ll get your data structure back. If not, you get an exception.

The idea is that you can generate schemas to make sure the ‘shape’ of your data is the way you wanted. For instance, if you wanted to make sure your web server’s requests parameters had only these keys of these types, you can do that.

And there are lots of ways to create schemas. You can create schemas that validate on a predicate, meaning you can just pass in a function that returns a boolean and have it validate:

(require '[schema.core :as s])

(defn divisible-by-two? 
  (= (mod x 2) 0))

(def DivisibleByTwo (s/pred divisible-by-two? 'divisible-by-two?))

(s/validate DivisibleByTwo 2)

;; => 2

The second most important concept to nail down is coercers.

Coercion took me a long time to grasp, but once you have it, you’ll be amazed at how modular and reusable the code becomes.

Prismatic talks a lot of custom walkers for coercions, but I’ve found in practice that’s not really necessary most of the time. It’s there for more advanced users. For almost every use case, the default walker is fine. I would ignore that part for now, otherwise it will confused you like me.

Essentially what coercion means is you’re able to take a type and transform it into the type you want.

Let’s be more concrete. I won’t use the sample from the readme because I feel like doesn’t explain it well enough to beginners.

Say we had these schemas:

(require '[schema.core :as s]
         '[schema.coerce :as coerce])

(def RequestCommentId s/Str)

(s/defschema RequestParams
  {;; Path param /comments/:comment-id -> /comments/4
   :comment-id RequestCommentId
   ;; Form params
   :comment-body s/Str

(s/defschema DatabaseParams
  {:id s/Int
   :body s/Str})

The RequestCommentId is our representation of a string. You’ll see why we need this wrapper in a moment, but it’s also important that we have some semantics to our representation, other than strings, integers, etc.

RequestParams are the parameters that come in from a web request. For example, in this case, let’s say we have a simple blog API. Somebody sends a PUT request to update the blog post with the id they specify to “/comments/:comment-id”. If you’re using compojure, that :comment-id is going to be a string and you would need to coerce to an integer.

Then we have our DatabaseParams. In our little scenario, we’re getting an updated blog post from our API and coercing it to something we can insert into our database. In this case, that means renaming the keys from :comment-id/:comment-body to just :id and :body. Additionally, it means we want an integer for our :id.

Here’s the coercion part:

(defn RequestParams->DatabaseParams
  [{:keys [comment-id comment-body] :as request-params}]
  {:id comment-id
   :body comment-body})

(def RequestParams->DatabaseParams-coercer
  (coerce/coercer DatabaseParams {DatabaseParams RequestParams->DatabaseParams
                                  s/Int #(Integer/parseInt %)}))

(RequestParams->DatabaseParams-coercer {:comment-id "123"
                                        :comment-body "Hello World!"})

;; => {:body "Hello World!", :id 123}

I start by defining a function that renames the keys. This function affects the entire request-params map. In my experience, it’s much better to put separate out the functions that go into the coercion map, as they’re easily re-useable elsewhere.

Then I define the coercer.

The first parameter to coerce/coercer is the schema I’m coercing into. The second is a map that tells the coercer how to do its job.

The map’s keys are the types the schema wants, and its values are the functions that do the coercion.

Recall we made a schema for s/Str called RequestCommentId. We did that for two reasons:

1. We want our own type for it instead of just using s/Str.

2. The body is also an s/Str. There’s no way for schema to know how to differentiate between an s/Str that’s a comment body and an s/Str that’s a comment id. Since we called it RequestCommentId instead, we can have two different schema types that at the lower level mean String.

Back to the coercion map.

DatabaseParams is what we want and a key in the map. This means that when we want DatabaseParams, we get the entire parameter passed to the coercion function, in this case, RequestParams. Think about it as a way to grab the entire thing you’re coercion. It’s useful so you can massage the keys of the map into the shape you want before the types get coerced.

Finally, the other key, s/Int says, whenever an s/Int is specified in our schema map, apply this function to the value in that map. Recall the DatabaseParams map has an :id key with the value s/Int. Since we’ve renamed the key from :comment-id to :id, the coercion walker sees that :id should be an s/Int, looks into the coercion map for the function to do it, and returns the result under the key :id.

That’s it! An ultra quick overview of Schema, one of the libraries that we use a lot at RentPath. We’re hiring senior clojure developers by the way (100% remote), send me an email if you’re interested and are a U.S. citizen.

It might’ve been a little too quick so if there are any questions/confusions, please leave a comment.


A Neat Pattern With Juxt

I recently added the function juxt to my toolbox in Clojure.

A good pattern I’ve discovered is using juxt to perform operations on a data structure and destructuring its return value.

(defn read-files
  [^String path]
  (let [data-files (find-data-files path)
        [prices receipts] ((juxt find-prices-file find-receipt-files) data-files)]
    {:prices (read-prices prices)
     :receipts (apply merge (read-receipts receipts))}))

Setting Up A Node.cljs Project With Leiningen and NPM

Before we begin, make sure you can start a node.cljs repl first

One important thing I want to highlight is that you should have a stable version of node 0.12.X as the linked wiki page says. Additionally, you want to make sure you have npm installed so you can install node packages later on. Finally, I’m also assuming that you already have Clojure and Clojurescript setup with Leiningen.

Let’s begin.

Start by making a new leiningen project. 

lein new node_project

And then move into that directory.

Start by editing your project.clj file to look something like this:

(defproject node_test "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.7.0-RC1"]
                 [org.clojure/clojurescript "0.0-3269"]
                 [org.clojure/core.async "0.1.346.0-17112a-alpha"]]
  :profiles {:dev
             {:dependencies [[com.cemerick/piggieback "0.2.1"]
                             [org.clojure/tools.nrepl "0.2.10"]]
              :plugins [[lein-cljsbuild "1.0.5"]]
              :source-paths ["src" "dev"]
              :repl-options {:nrepl-middleware [cemerick.piggieback/wrap-cljs-repl]}}}
  [[:cljsbuild :builds 0 :compiler :output-to]
  :cljsbuild {:builds
              [{:id "dev"
                :source-paths ["src"]
                :compiler {:output-dir "out"
                           :output-to "index.js"
                           :optimizations :none
                           :source-map "src-maps"
                           :target :nodejs}}]})

It’s very important that you use versions of Clojure and Clojurescript greater than or equal to the ones I’ve listed. I’ve ran into a lot of issues trying a setup like this before finding one that worked. It was mostly due to old versions of piggieback and Clojurescript.

I added the piggieback related dependencies in dev for the node.js REPL we’re going to be using later. The clean-targets are for the command 

lein clean

Because cljsbuild deprecated its clean function.

A few more things to note:

We’re using no optimizations and targeting node.js. Having optimizations set to :simple, I ran into issues with core.async macros. 

Next, we need to setup the node side of the project. Inside the same project folder, run:

npm init

You can pretty much press enter for every single prompt, except the entry point one. There you want it to be named app.js, not index.js as we specified in the project.clj. We can’t run index.js directly due to optimizations being set to none, so cljsbuild will output multiple files.

Next, you can install a web framework using npm like express:

npm install --save express

Now that we have Express.js installed, we can just make a simple hello world web application in clojurescript.

Rename src/node_test/core.clj to src/node_test/core.cljs.

Then in the file put this:

(ns node-test.core
  (:require [cljs.nodejs :as node]))

(def express (node/require "express"))

(def app (new express))

(defn -main
  (.listen app
           (fn []
             (js/console.log "App Started at http://localhost:3000"))))

(set! *main-cli-fn* -main)

Then you want to compile your clojurescript to javascript with:

lein cljsbuild auto

Now that we have our index.js file, we need to make the file for node to run the application.

Make a file called app.js in the root of your project folder with this as its contents:

// Bootstrap node.js

// Our app compiled by cljsbuild

// The core of our code

// The core of cljs

// Run main
// NOTE: Dashes in namespaces are replaced by underscores.

If you’ve changed anything, replace the paths with the correct one.

Now, you can start the app with node.js:

$ node app.js
App Started at http://localhost:3000

And if you visit that url, you’ll receive an error message about not being able to GET /. That’s fine, we didn’t specify any routes.

If you liked this post and want to learn more about Node.cljs, sign up for my mailing list to get my latest posts in your inbox. Next post will be on how to develop this project shell further by adding in a cljs repl.

I’m also available for hire as a Clojure freelancer.

Becoming A Finisher

I’ve always known that I’m a starter and not much of a finisher. I’ll start a project and quickly get excited about a new idea and switch to the next shiny thing.

But I didn’t realize the extent of my problem until I had a conversation with some of my friends while playing Dota. My friends were talking about video games that they’ve completed, and that made me realize. I can’t even finish video games.

The games I’ve played are all competitive, like Dota or Hearthstone. No story to them. But even the competitive games with stories, like Starcraft 2, I can’t finish. Even single player games I can’t get halfway through. 

I started doing research on the subject by searching Hacker News, as it seems like a common problem and found a thread where Paul Graham gives some advice:

Taking on projects that are too big creates a situation like an airplane stalling because the angle of attack is too steep. Better only to take on projects so small that you know you can finish them, however humiliatingly small that may be initially. Then, with always finishing as a given, start to gradually crank up the ambition. But never so sharply that you stall.

Paul Graham

It may seem silly, but it looks like finishing is something you have to learn and practice. It’s a skill.

I’m going to take Paul Graham’s advice. 

Two things I’m going to start doing:

  • Using my Apple Watch as a pomodoro timer. Essentially the idea is you have periods of working and periods of relaxing. Usually it’s 25 minutes of work and 5 minutes of taking a break.
  • Breaking tasks up by single pomodoros to begin with. That’s humiliatingly small. 

Goal for tomorrow: Write the first post on Node.cljs in one pomodoro.

Stay tuned.

How To Refer To A Clojure Record In Another Namespace

I ran into an issue like usual. This time it seemed simple. I wanted to refer to a record that I def’d in another namespace.

The problem is, you can’t just do a :require and :refer it in regular Clojure. However, in Clojurescript this is how you would do it.

What you have to do is :import it. And the key trick to keep in mind when using :import on records and Clojure namespaces is that dashes are converted to underscores like packages in Java.

If you had a namespace like (ns foo-bar) in which there was a record called MyRecord, you would do something like this:

(:import [foo_bar MyRecord])

And you’ll be off to the races.