Growing Devs

Reflecting on Code School: Career Part Two, Software Boogaloo

2014-12-19T17:20:51+00:00

“I’ve been around the block twice, now my shot nice
No referee but best believe out here I got stripes
Now that they finally consider me a legit scorer
Would you believe I started off as a bench-warmer?
Have me feelin’ like Master Splinter
Now the haters and commentators say ‘there’s a winner’
Hey, hard work pay off, now I don’t take a day off”
– J. Cole The Plan

A little over four years ago, I was sitting in my office in a U.S federal government agency bored out of my mind. Thinking about an idea I had during a recent trip home to Pennsylvania, I started researching web development. I joined a bunch of e-mail lists and bought a whole slew of books. After a while I reached out to some engineers, and many of them suggested I look into Ruby on Rails. So I did.

I spent hours scouring docs, reading Stack Overflow, and reviewing some code a friend of mine wrote. After two years and some skepticism that I was doing the right thing, I entered LivingSocial’s Hungry Academy. It was, as the Washington Post described, an experiment. More late nights ensued, and after five months I became a full-time engineer at LivingSocial.

Looking back, while my path was a bit unusual at the time, my gravitation towards web development makes sense - especially considering my prior career in government. Facing the same options, Jack Dorsey described his rationale for choosing computer science over political science in a talk at Stanford:

I was deciding between political science at the time and computer science because I have always been fascinated by cities and at some point want to maybe potentially go into government. I’m still not quite sure if I have more effect there or more effect in programming. But I went away from political science because I realized that there are a lot of parallels between what you do in politics and what you do in government and writing policy and laws and what you do in programming. But the difference is the time scale. So, I could write a policy as a senator or as a mayor and I could see the effect maybe in eight years. But I could write that same policy and write a simulator around it and write populations around it and I could see that effect instantly with a computer. So, I went down the computer science route.

Dorsey’s observation is spot-on. If I were in my previous government job, it might take years to see something come to fruition. Now I can go home with an idea, crank out some code and have a working prototype, all in a matter of days. (Granted there’s still a ton of work to go into making a prototype a reality, but I digress.) That satisfaction is what attracted me to web development and it’s one of the reasons for which I’m grateful for having been a part of Hungry Academy. But like everything else, the road to becoming an engineer wasn’t without its challenges.

Much like the experiences that come from them, the challenges that code schools provide are unique. While you’re going through it, you’re aware of how much the skills you’re learning are in-demand. Then when you leave, you don’t know how in demand you are because you’re new. Looking back, there were things that were new to me that I struggled with as well as things that I had experienced before but were reinforced.

First, the things that were new:

Imposter syndrome is real

Coming out of a code school, it can be quite difficult to not suffer from imposter syndrome. You graduate and end up working alongside incredibly talented people who are on your same team, but whose journey getting there was very different: years programming as a child, college degrees, etc. Believe me, I get it. Hungry Academy wasn’t your typical code school experience. We had a sweet deal thanks to LivingSocial. If there was an ideal way to learn to code, Hungry Academy was it.

It’s hard to remember, but no matter how you got there you have to do the job you were hired to do. Period. As Mike Tomlin says, “The Standard is the Standard”. If I’m given the job title of Engineer, then I have to contribute to my team just like any other person with that title.

It’s up to your manager and company’s performance-based processes to address your performance and make sure you are judged on your current and future work as opposed to your past.

Empathy is necessary

One of the things I struggle with as an engineer is the concept of efficiency. I’m not opposed to it. As a matter of fact, I’m well aware that a part of my job revolves around creating it. However that doesn’t mean it always sits well with me when I do.

This is because, in software development, creating efficiency sometimes correlates to someone else’s job being made obsolete. I get that this is all a part of change and innovation. I get that it is the outcome of a changing workforce that is adapting to the challenges of an information economy.

But it’s still tough.

A word that’s mentioned in a good number of blog posts of late is empathy. More often than not it is in respect to the people who you deal with directly. To me, it’s just as important to practice empathy with those who are both directly and indirectly affected by your work.

I’ll save this for another post, but while we all believe in the power of the internet that doesn’t mean we can - and should - dismiss what it can do to existing jobs through our work.

While the importance of empathy and the reality of imposter syndrome were eye-openers, the following lessons were reinforced during (and after) Hungry Academy and have been just as important to my growth as an engineer:

Learning is failure

And vice versa.

Simply put: you likely won’t be able to get through a code school if failure scares you. After all, code school is like climbing a very high, steep hill at a frantic pace. It’s worth it at the end, but going through it just sucks at times.

Reminding yourself that failure is a part of learning and accepting it goes a long way in the life of a new engineer. That applies to code school and beyond.

Diversity is important

During Hungry Academy, one of the hardest things to do was to not compare myself to others. There was my friend who wrote Java in his last job, and my other friend who was picking up Ruby like she had written it for years. It wasn’t by design, but when you’re in an environment where everyone knows they are vying to look attractive to an engineering team, some of us compared ourselves to each other. We knew who was doing the best and we wondered where we’d end up when the music stopped.

The problem with that is that we lost sight of something: Everyone brought something to the table. That’s why we were selected in the first place.

When you have a good team, it is rarely the case when a part of that team doesn’t add something to the whole. So for me to constantly get caught up in whether I was as good at something as one of my peers wasn’t productive. On its own, it didn’t really matter whether I excelled in the technical or interpersonal aspects. What mattered was that my contributions helped the team move forward in the best way possible.

Balance is paramount

Sometimes, to compensate for imposter syndrome you end up working around the clock. I know I do. If you can relate to that, do yourself a favor: stop reading this article. Go for a walk. Go call someone you care about. Do something.

Seriously, this article will be here when you get back. So will the new feature that might get you promoted. So will the language you think you have to learn in order to get a better job. It will all be here.

What may not be is your sanity.

Unlike with imposter syndrome, I wouldn’t rely much on managers doing much here. Your managers might want you to take vacation and encourage you to do so, but you’re the one who has to prioritize your life and well-being. Burning out is as real as imposter syndrome, and it won’t help your team if you do.

So those are a few of the takeaways I took from my time in - and my career since - Hungry Academy. I don’t pretend to think that I speak for everyone, but I hope my perspective helps. Especially if you’re thinking about engineering as a career or even going into a code school for the first time. If you have any questions, you can find me on twitter @travisvalentine.

A basic infrastructure with Docker, Chef, and Rails

2014-07-26T17:48:08+00:00

Before we get started

This tutorial expects you to have knowledge of the basic Docker concepts such as images, containers, volumes, and linking. If you aren’t familiar, take a look at Getting Started with Docker. In this post, we’ll be building infrastructure in Vagrant with the lightweight resources provided by Chef-Docker.

Our chef kitchen, where we’ll configure our environment specific variables, can be found here: Example Chef Kitchen

What isn’t covered

Zero downtime deploys - Vamsee Kanakala’s talk on Zero Downtime Deployments with Docker at Garden City Ruby is helpful.
Backing up your containers
Deployment to production - I have been deploying to DigitalOcean via Chef solo, however that’s a bit outside the scope of this post.

What are we trying to build?

It’s hard to explain even a simple infrastructure without a diagram. So this is what we’ll be building step-by-step:

In this diagram, each square represents a Docker container built from a Docker image. Everything a container needs to run is either found in the container (e.g. configuration files) or by linking to other containers.

The rails application is linked to a gem cache container to avoid downloading gems everytime we deploy a container.

Gem Cache

Before each section, I’ll reference the chef recipe and the Dockerfile here:

In the Docker world, containers are ephemeral, which means data written to containers disappear when we redeploy a container. Docker volumes allow us to store data outside of a container. In our case, we never want to redownload a gem if it already has been downloaded.

Let’s take a look at the data volume recipe:

  
    include_recipe 'docker'

    # 1
    cookbook_file 'Dockerfile' do
      path '/tmp/Dockerfile'
      source 'gem-cache/Dockerfile'
    end

    # 2
    docker_image 'ubuntu' do
      tag 'gem-cache'
      source '/tmp'
      action :build_if_missing
    end

    #3
    docker_container 'gem-cache' do
      image 'ubuntu:gem-cache'
      container_name 'gem-cache'
      detach true
      action :run
    end

The first thing we need to do is copy over our Dockerfile into the context of the Docker build to provide the Docker daemon access to the files when building the image.
Next we create the Docker volume image. We set the context of the build to be /tmp, which is where we copied our Dockerfile scripts.
Finally, we use the image to run a container. This container is set to run bash via the CMD directive in the Dockerfile.

Postgres

Let’s look at the chef recipe to build our Docker image, since there’s a lot more going on:

  
    # 1
    FROM austenito/ruby:2.1.2

    # 2
    RUN mkdir /postgres
    ADD Berksfile /postgres/Berksfile
    ADD solo.json /postgres/solo.json
    ADD solo.rb /postgres/solo.rb

    WORKDIR /postgres

    # 3
    RUN bash -c 'source /usr/local/share/chruby/chruby.sh; chruby 2.1.2'
    RUN berks vendor ./cookbooks
    RUN chef-solo -c solo.rb -j solo.json

    # 4
    VOLUME  ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]

    # 5
    EXPOSE 5432
    USER postgres
    CMD ["/usr/lib/postgresql/9.3/bin/postgres",
         "-D", "/var/lib/postgresql/9.3/main", "-c",
         "config_file=/etc/postgresql/9.3/main/postgresql.conf"]

Pull a pre-built Ruby 2.1.2 image from Docker hub. It has ruby 2.1.2 and chruby installed.
Add files in the Docker daemon context to our image.
Run chef-solo to build and configure postgres with the files we copied into our image.
Expose directories to other containers to allow backups of our data. If we didn’t do this and the container is deleted, all of our data would be lost.
Run postgres.

Our chef recipe is similar to the gem cache with a few differences:

We expose a port (5432) to other containers to allow tcp connections to this container
We use env to set environmental variables before we run our container
We set which volumes from other containers that available in this container with volumes_from

  
    ...

    if `sudo docker ps -a | grep postgres`.size == 0
      docker_container 'postgres' do
        image 'austenito/postgres:9.3'
        container_name 'postgres'
        port "5432:5432"
        detach true
        env ["POSTGRES_USER=#{node['postgresql']['user']}",
             "POSTGRES_PASSWORD=#{node['postgresql']['password']}"
            ]
        volumes_from 'gem-cache'
        action :run
      end
    end

Rails application

Our Rails application container uses the link directive to provide container linking. This exposes environment variables with ip and port information of postgres container.

  
    ...

    docker_container 'rails-example' do
      image 'austenito/rails-example'
      container_name 'rails-example'
      detach true
      link ['postgres:db']
      volumes_from 'gem-cache'
      action :run
      port '3000:3000'
    end

When the rails-example container starts up, we want to be able to bundle, precompile our assets, migrate, then start our server. Specifying this inline via the CMD directive is a bit cumbersome, so we can specify a script to the CMD directive, run.sh. The script clones the latest rails-example repository, bundles, migrates, and starts unicorn.

Nginx

Unlike the postgres container, which uses a chef recipe for configuration, we will configure nginx manually via apt and copying over our nginx.conf file. Using the nginx chef recipe turns out to be a bit too compilicated for our simple infrastructure example.

Other than that, there is nothing new to see in the recipe or the Dockerfile. Our Dockerfile installs nginx and we run a container with nginx with our chef recipe.

Putting it all together

We’re ready to deploy our infrasture to vagrant. Run the following commands:

  
    git clone https://github.com/austenito/docker-chef-rails-example
    bundle
    berks
    vagrant plugin install vagrant-omnibus
    vagrant up

If everything went right the first time (it always does right?), you can visit the example app at http://localhost:8080/.

Let’s ssh into our vagrant box by running vagrant ssh and poke around. First let see what images we created:

  
    sudo docker images

    REPOSITORY                TAG                 IMAGE ID
    austenito                 nginx               a10b38bf0975
    austenito/rails-example   latest              38c2ed56c811
    austenito/postgres        9.3                 88c026cf2326
    ubuntu                    gem-cache           11f9a661754b
    ubuntu                    14.04               c4ff7513909d
    austenito/ruby            2.1.2               c794944b5fa2

These images were built by the docker_image directive in our chef recipes. They are used to create and run the containers below:

  
    sudo docker ps -a

    IMAGE                           PORTS                    NAMES
    austenito:nginx                 0.0.0.0:80->80/tcp       nginx
    austenito/rails-example:latest  0.0.0.0:3000->3000/tcp   nginx/rails_example,rails-example
    austenito/postgres:9.3          0.0.0.0:5432->5432/tcp   nginx/rails_example/db,postgres,rails-example/db
    ubuntu:gem-cache                                         gem-cache

What if I don’t want to build my images from the ground up?

Great question! Part of the power of Docker is the ability to push your images (ala git style) to Docker Hub so you don’t have to rebuild images. Ruby takes the longest to compile and I’ve set the docker_image directive in the postgres Dockerfile to point to austenito/ruby:2.1.2.

Ruby 2.1.2

How to Pick Languages

2014-07-14T01:03:43+00:00

Ideas for choosing your next language foray

My relationship with programming languages is sometimes problematic. You see, it’s quite easy for me to see that the grass must clearly be greener on the other side of the language fence. Thus, I spend a lot of time doing anything from a light tinker to a full-blown deep dive into new or interesting languages.

I’m a programming language aficionado. More often than not, a dilettante.

That confessional aside, I do have guidelines that help me decide when to read over language docs looking for interesting ideas, when to tinker with it if I get “the urge”, and when to do a full-blown weekend dive into a language.

WARNING this is for personal use only. Almost none of these guidelines hold water when it comes time to decide to deploy software to production.

My favorite reason to dive into a language is the creator has a very good taste in solving problems with computers or thinking about how developers think about writing programs. Though they sit on very different points in the language spectrum, Rich Hickey’s (Clojure) and Martin Odersky’s (Scala) both have very compelling ideas about how to design a programming language for developers such that it magnifies a developer’s ability to think about solving problems with computers. Other times, the intrigue of a language designer’s philosophy and work make it worth looking into a language even if I never intend to use it. I checked out Rasmus Andersson’s Move solely on the strength of the rest of his work.

Novelty of solution is an interesting angle. Some languages solve a boring problem with a novel approach. For example, CoffeeScript was the first language to treat JavaScript as a boring, lower-level target and layer a principled, tidier language on top of it via a translation layer. Building on top of that, I recently came across Iced CoffeeScript, which takes a problem that is new to many developers, confusing callback hierarchies, and layers some CoffeeScript syntax over rewriting the nested code as linear defer/await structures. Elixir is trending lately, and falls under the same scope, though it’s target is the Erlang ecosystem.

Languages with challenging, satisfying, or interesting principles are a source of intrigue. Haskell’s principles of extremely pure functional programming and type safety are a real puzzle if you’re coming from Ruby, which allows all manners of things Haskell outright forbids. Clojure’s principles, taken as a whole, can be very satisfying: write small functions passing immutable data structures, with very explicit handling of program state to compose programs that eschew complexity in favor of simplicity due to boundaries. Erlang’s emergent qualities are very interesting: use functions, pattern matching, and actors to implement reliable distributed systems. I saw a language that was all about making unit conversion, e.g. feet to meters, explicit in the language. Point is, some languages are fun playing around with simply to bend your brain in weird ways.

Languages that make previously inaccessible application domains simple can be a real ego boost. Right now, Swift is attracting a lot of eyeballs as it makes developing for Apple devices far simpler than it has been in the past. In the same way, Clojure makes Lisp a little easier, Erlang makes distributed systems a slightly less enigmatic, and Go makes systems programming less intimidating.

A language with a healthy culture is worth its weight in gold. If the values of the people and code in a language don’t align well, you’ll have a bad time. If they fit well within your own, you’re going to have a ton of fun. This is the most subjective of all parameters. I find the principles around Go and Clojure really intriguing and thus find them really interesting. I’m less jazzed about Scala or C++, so I “read into them” a lot less often.

Some languages are worth considering solely on the strength of the implementation. Lua is highly regarded for its simple of design and the modernity of its runtime given the tradeoffs required in a tool designed for embedding in larger software as a scripting language. Likewise, V8 and the Hotspot JIT compiler are well regarded technologies that make considering JavaScript and Java a rationale choice despite the shortcomings of those two languages.

Finally, not every language is cut out for an enthusiastic tinker. I’m likely to skip languages that are highly experimental, highly mathematical, come from boring vendors, or are too similar to languages I already know or use. That crosses languages like Seph, Agda, or Python off my list.

There’s half a dozen ideas to give yourself permission and treat yourself to learning a new language. Of course, the old standby excuses are pretty good too. Learning any new programming language expands your brain. Every language you learn gives you more vocabulary to solve problems, no matter what languages you use on a day-to-day basis.

Teaching yourself a new language is a winning proposition, you only have to decide where to start. And then learn the language. But sometimes the decision is harder than the learning!

Hopefully my guidelines, and perhaps some of the languages that satisfy them, will help you next time you get that itch to explore a new programming language.

Battling Complexity in Ruby

2014-07-05T23:14:42+00:00

Every one of our classes is an ambitious creature that seeks power through an accumulation of methods; knowledge through willful ignorance of the Law of Demeter; freedom by resisting testability; immortality through complexity. In short, they all strive for godhood.

When this happens, we find the power relationship between the developer and the codebase inverted. The code controls us. We become mere handmaidens to complexity. Although our instinct might be to treat problems of complexity as Gordian knots to cut through with a single stroke of cleverness, this is a fantasy that almost never plays out well in reality. The truth is that achieving simplicity takes a lot of effort. Our best chance to win victory over complexity lies in our ability to slowly and methodically break complicated things down into smaller pieces.

Code Analysis

Just as changing code without having tests as a guiderope is a recipe for disaster, so is refactoring code without deciding in advance on success criteria. One way to establish such criteria is through the use of code analysis tools.

The gallery of such tools is populated with evocatively-named rogues such as Flog, Flay, and Reek. Some, like Metric Fu, are meta-tools combining several analysis tools together to provide a more faceted view of code quality. And of course there is the SaaS solution of Code Climate, with its famous letter-grade ranking of complexity.

But the results of these tools can often be overwhelming. What do you do with the list of code smells, the failing grades, the cryptic numbers?

My recommendation is to start small. Identify a single metric that is easy to understand and communicate, and take steps to improve that metric.

If Only

It helps to define boundaries and sharpen our focus. A good first pass at reducing complexity can be to refactor conditional-heavy methods. The wedge code that most often undermines the simplicity, reusability, and clarity of our programs is the conditional. (A tax on if statements, with extra penalties for ternaries, would do more for code quality than an army of consultants equipped with Michael Feathers books.)

Reducing conditionals is a good way to reduce noise in the codebase, and reveal more gnarly complexity underneath, while making positive and demonstrable progress in promoting code clarity.

The Garden of Plenty

There are lots of code analysis tools that measure complexity, but none of them seem to agree on what complexity means. Running a suite of analysis tools using MetricFu can demonstrate general consensus on where hotspots in the code are, but it’s easy to be overwhelmed by the half-dozen independent metrics that component tools provide. It’s also impossible to compare the results of one tool against another, because each uses independent metrics and algorithms. Is a “D” in Code Climate better or worse than an 13.2 in Flay? Once again we face a signal-to-noise problem.

Surely there is a tool that specifically helps identify overuse of conditionals?

The Desert of the Real

The best approach to identifying complex conditional logic is examining cyclomatic complexity scores. Cyclomatic complexity is derived from an analysis of the number of possible execution paths through a given piece of code. There is even a Ruby tool for measuring this, called Saikuro, and it’s already part of MetricFu.

Unfortunately the output of the program didn’t serve my needs. I forked the repository with the intention of extending the program to produce output that I could better use. I discovered that the project was created in 2005 and appeared to be abandoned. The version that MetricFu used was a bespoke fork. And the code had not aged well– despite my best efforts, I found the program to be (ironically) complex and resistant to change.

Fukuzatsu is Born

I promise that I did not set out from the start to build my own code complexity tool. Really I didn’t. But in the end, that’s what I did.

Fukuzatsu (Japanese for “complexity”) is a command-line tool packaged as a Ruby gem. It is invoked by the fuku check command, which takes a file path as an argument. Output options can be specified by a --format (or -f) flag, and include html, csv, and of course text (default). To facilitate CI integration, the program can be passed a minimum threshold (via --threshold or -t), which in turn triggers an exit condition that CI applications can use to ensure that code meets your standards.

STDOUT Output

Good UNIX programs accept text input and return text output, allowing them to be used in the composition of powerful ad-hoc programs using the pipe operator (|). Fukuzatsu’s default mode accepts a file path and returns text output. So for example if we want to analyze the largest file in our project, we can do this:

$ find . -type f -exec ls -s {} \; | sort -n -r | head -1 | xargs fuku check $1

This composite program uses ls to list files in order of size, piped to head to grab the first on the list, and passing the result to fuku for analysis. The output is:

Analyzer    17
Analyzer  #initialize 0
Analyzer  #complexity 1
Analyzer  #extract_methods  1
Analyzer  #extract_class_name 2
Analyzer  #text_at  0
Analyzer  #find_class 4
Analyzer  #extend_graph 0
Analyzer  #methods_from 4
Analyzer  #parent_node? 1
Analyzer  #parse! 1
Analyzer  #parsed 1
Analyzer  #traverse 2

This says that the overall complexity of the Analyzer class is 17. Each method is listed with its name and complexity value. This output is suitable for piping out to other programs, or to a file for later perusal:

$ ls -S ./lib/*/** | head -1 | xargs fuku check $1 > results.txt

This could be handy for running via a cron job to measure how complexity changes over time.

CSV Output

If you want to pull your analysis into a spreadsheet, the csv option is right for you:

$ fuku check lib/foo/bar.rb -f csv
Results written to:
doc/fukuzatsu/lib/foo/bar.rb.csv

CI Output

To better integrate with a continuous integration system, set the -t (threshold) flag with the maximum allowed complexity level:

$ fuku check lib/fukuzatsu/complex.rb -t 7
Complex   17
Complex #initialize 0
Complex #complexity 1
Complex #extract_methods  1
Complex #extract_class_name 2
Complex #text_at  0
Complex #find_class 4
Complex #extend_graph 0
Complex #methods_from 4
Complex #parent_node? 1
Complex #parse! 1
Complex #parsed 1
Complex #traverse 2

Results written to:
doc/fukuzatsu/lib/fukuzatsu/complex.rb

Maximum complexity is 17, which is greater than the threshold of 7.

When a threshold is set, the exit status will be non-zero if the threshold is met or exceeded.

HTML Output

For an interactive report, use the -f html flag:

$ fuku check lib/my_project/ -f html
Results written to:
doc/fukuzatsu/index.htm

This will generate a tree of HTML files under the ./doc/fukuzatsu directory, complete with an index file at the root path. The generated pages display sortable and filterable tables of files or methods with their complexity values.

Wrapping Up

I’m finding Fukuzatsu to be a great tool for helping me identify complexity resulting from too many conditionals in my code. In fact, I use it regularly to measure itself and try to keep the numbers from going up from merge request to merge request.

I hope that you’ll check out Fukuzatsu and try it in your refactoring project. And of course bug reports, feature requests, and merge requests are always welcome! It’s available on RubyGems right now (gem install fukuzatsu), and installation and usage instructions are provided on GitLab.

Building a Conference From Scratch

2014-05-23T02:06:31+00:00

Anyone who knows me well will tell you that I have a terrible habit of reinventing the wheel. It can get really, really bad at times. I know there is a lot of great software out there, but there is always the smallest little thing that keeps it from being perfect. I know I can do better! So imagine the itch I get when someone says “Hey, we have to run a conference. What software do we need?”

TransparencyCamp is an open government unconference run by my employer, the Sunlight Foundation. We started TransparencyCamp in 2009 with just under 100 attendees. It was a scrappy little event, with most of us having very little experience running a conference. On May 30-31, 2014, we’ll be hosting over 550 people from across the world at TransparencyCamp 2014.

As the conference has grown, we’ve had the chance to build numerous projects that help us run the event. Sure, we could have used existing software to do it, but it’s way more fun to build our own.

The Badges and Registration

In previous years we used Eventbrite for registration and ticketing. It was a great system; they even provide a mobile app that can be used at the check-in desk to scan badges. Never wanting to settle for good enough, we figured out the format of the data stored in the Eventbrite-generated barcodes and encoded our own as QR codes that were printed on the badges. These badges could still be read by the app provided by Eventbrite, but when used in conjunction with the Eventbrite API, we could manage the process using our own systems.

This year we’ve gone with a completely homegrown registration and ticketing system writen in Python and Django. It uses Braintree for processing payments, sends email with Postmark, and has an API for accessing ticket information. The system also takes care of generating all of the QR codes that are printed on the badges.

Having all of the ticket information in our own system allows us to tie in other aspects of the event. We have several food trucks available for lunch (covered by the price of a ticket) and will have staff at each truck scanning the QR code on attendee badges when they get their food. This isn’t just to make sure people only eat one meal, it will also provide us with valuable data about which trucks were the most popular and how fast each truck was able to serve customers. We’ll be displaying this information in real-time at the event.

The Schedule

An attentive crowd in front of "The Wall" by sunlightfoundation

What would an unconference be without a giant wall for schedule making? Until last year we used a paper-based wall where people would write session information on cards and tape them to the wall. We decided to try something different in 2013 (partially due to the venue not allowing tape on the walls) and go completely digital. Sessions were submitted via laptops or paper and entered into a database. This database powered both the web site and a giant HTML calendar view that was cast onto the wall using three short throw projectors.

We’re moving back to paper this year due to readability issues with the digital wall (distance, text size, clarity, etc.), but sessions will still be submitted electronically. Cards for each session will be printed and hung on the wall by staff.

The Counselors

Who wants to come to a conference and see the staff fumbling through stacks of papers or have to stand around while a staff member that can help solve your problem is located?

Almost everything staff need to do at TransparencyCamp is handled by a custom iOS app, Camp Counselor. This includes:

scanning badges to check-in attendees
viewing check-in stats and other important day-of information
scanning badges to “rent” display adapters and other equipment
marking equipment as returned or sending a polite email to people that have run off with something
scanning badges (lots of scanning) to redeem lunch at food trucks
scanning badges to register attendees for API keys

All of this information is stored on a central server, Lodge, that has a simple JSON API. The Lodge is a Python web app written in Flask and hosted on Heroku. The iOS app itself is a small client to the API using AFNetworking for making calls to the server.

We distribute the app to staff devices via TestFlight. If an emergency fix needs to be pushed out, TestFlight can force people to upgrade, ensuring that everyone is using the latest version.

The Other Stuff

Transparency Camp 2012 by Burt Lum

Since it’s the small details that really make a difference, we’ve built out a number of other interesting things:

HTML-based information screens placed around the event that show current schedule information
HTML-based slideshows shown on screens that cycle through tweets and Instagrams from the event
live-streaming of keynote talks
a responsive web site, optimized for day-of use on mobile
an interactive SMS interface to the schedule

There is no doubt that we could have run TransparencyCamp throughout the years without spending all of this time on custom development. It would have been a perfectly fine conference, but we don’t want to settle for “fine” at Sunlight. The extra effort that we put into the systems and the experience sets the tone for the event. It shows that we care about the little details. It shows that we seek to inspire. It shows that TransparencyCamp is unlike any other conference you’ve been to.

It is important to understand all of the downsides that come along with the decision to embark on your own path and reinvent the wheel. It’s not something that should be taken lightly. However, if done with understanding and purpose, extreme DIY can yield fantastic (and super fun) results.

Source code for the Camp Counselor iOS app and Lodge server will be made available shortly after TransparencyCamp 2014 on the Sunlight Labs GitHub account. The TransparencyCamp site, registration, and scheduling systems have already been published. These projects, like all good software, are available under the BSD license.

Use Bower for managing front-end assets in Rails

2014-04-09T16:34:50+00:00

Rails’ history with front-end asset management is somewhat complex, involving a mixture of gems and manual file management. Fortunately, there’s now a better way—Bower.

Rails 4.0.4 depends on version 2.2.x of the jquery-rails gem which in turn bundles 1.9.1 of JQuery. Does that make any sense? The latest official release of angular-ui-bootstrap-rails is 0.9.0, which is at least a version behind the current release of the Angular UI bootstrap directives. When will it be updated? chosen-rails version 1.1.0 bundles an old version of chosen-jquery that has never been officially released. What?

As recent as a year and a half ago, this mess was the only sane way to manage front-end assets in a Rails app. You’d have to create a Rails Engine inside a gem that made the assets available.

When you want to use an asset in your Rails app, you have to navigate the insanity above to figure out what version you are using. If what you want isn’t provided by a gem, the “best practice” was to just download the files and throw them in vendor/assets. If your assets had inter-dependencies, you better watch out. And if you had a non-Ruby application - forget about it.

Now, there’s a better way—Bower.

Bower was created by Twitter to manage front-end assets. It’s not particular to Rails (and, like many JavaScript command-line tools, it uses NodeJS), but it works similarly to RubyGems and Bundler. You simply create a file in your project that describes what packages you want, and use the bower command-line app to install them. Bower also knows about inter-asset dependencies. Most importantly, Bower allows you to specify directly what version of an asset you want.

I’m not sure why Rails doesn’t provide a better tool for this. Perhaps someday it will, but for now, Bower is going to make your life so much easier, especially if you use a lot of front-end assets, or need to share private assets between your apps.

Setting it up

The handy bower-rails gem creates some Rake tasks for you, but also allows you to specify your dependencies in a succinct Gemfile-like format (called, naturaly, Bowerfile).

In your Gemfile:

gem 'bower-rails'

It’s important to note that this doesn’t include the bower command-line app, it just calls into it. You’ll need to install bower, likely with npm install bower (which requires installing Node and NPM. I know that’s a potential yak-shaving thing, but it’ll be worth it).

Then place Bowerfile in your root directory. Here’s an example of one that uses AngularJS, version 1.2.13:

asset 'angular', '1.2.13'
asset 'angular-route'
asset 'angular-resource'
asset 'angular-mocks'
asset 'angular-ui-bootstrap-bower'

You then install them with rake bower:install. If there are any verison incompatibilities, they will generate errors, just like you’d see in Bundler. Everything gets downloaded to vendor/assets/bower_components by default. Because this isn’t standard Rails, you’ll need to add this to your asset pipeline configuration in config/application.rb:

config.assets.paths << 
  Rails.root.join("vendor","assets","bower_components")

And, finally, add your newly-managed assets to application.js:

//= require angular/angular
//= require angular-resource/angular-resource
//= require angular-route/angular-route
//= require angular-ui-bootstrap-bower/ui-bootstrap.js
//= require angular-ui-bootstrap-bower/ui-bootstrap-tpls.js

That’s it! Now, your front-end assets can be managed in a sane way. But this is only part of it. Because Bower is so simple, it’s easy to use it to manage internal, private assets that you don’t (or can’t) share with the outside world.

Managing Private Assets

I work at Stitch Fix, and we have several applications that serve many different purposes. Still, they all interact with our inventory in various ways. Although much of that information is stored in a shared database, the official color swatches of our inventory aren’t stored there. They had been copied between various apps. We’re now using Bower to share them. This way, when we add new colors or update the swatches, the whole team knows, and we can roll out the updates in an organized way.

Here’s how to set that up.

Since a Bower version string can be a git url, a value like git@github.com/stitchfix/colors#1.0.3 will tell Bower to get assets from the stitchfix/colors repo, at whatever version is tagged 1.0.3. All we have to do is put a small file named bower.json into that repo:

{
  "name": "colors",
  "version": "1.0.3",
  "main": [
    "colors.css"
  ],
  "dependencies": { },
  "devDependencies": { },
  "ignore": [
    "**/.*",
    "bower_components",
    "test"
  ]
}

In your Bowerfile, you just use the git url:

asset 'colors', 'git@github.com/stitchfix/colors#1.0.3'

And, we add it to application.css.scss:

#= colors/colors

That’s it! Now you can manage internal assets without RubyGems in a sane way. This means that all your front-end assets, public and private, are managed from one clear location, and you can re-use your internal assets for non-Ruby apps, to boot.

Next time you are tempted to use curl to manage your assets, try Bower instead.

Refactoring Rails Views with SimplestView

2014-01-29T16:34:50+00:00

Most of the projects I’ve worked with grow in similar ways. Code often ends up where it is easiest to place it. For code in our Rails “Views”, under app/views, that’s either in the controller (with lots of instance variables assigned), in ApplicationHelper (shared with every view), or in the view templates themselves.

I’m going to try to address each of these scenarios, and show you how to refactor them into much simpler code, using SimplestView. But first, a little background.

The Rails Way

In Rails, our ERB templates go under app/views. There is a logical structure to how they are organized. This is The Rails Way. By convention, the views for a controller SalesController goes inside the folder app/views/sales. Further, each action gets a file inside the sales folder, using the name of the action as the filename.

The tricky bit is these Rails “Views” aren’t really view objects. The ERB templates we build, are rendered inside the context of an instance of an anonymous class, which inherits it’s rendering abilities from ActionView::Base.

But why an anonymous class? Why does Rails hide this away from us completely?

With SimplestView

Now, if we set up our Rails application to use SimplestView, we get clear, conventional access to these previously hidden view classes. We give them a name. By convention, that name will match the action, and be placed inside a folder matching the controller name.

For our SalesController’s #index and #show we would have classes named Sales::IndexView and Sales::ShowView (which inherit from ActionView::Base) inside of a folder in app/views/sales. Note: SimplestView has you move your ERB into a more aptly named app/templates folder to keep things cleanly separated, and to give you a nice Rails-y place to put your new views.

Now that we have that out of the way, let’s get into some refactorings!

ApplicationHelper Overload

How often have you had to dig through an ApplicationHelper that simply contains every single helper in the whole project? I’ve seen projects with an ApplicationHelper thousands of lines long.

Let’s look at a specific method that we can refactor, and get it out of there!

def format_tax_rate(value)
  '%.2f%' % (value.to_f * 100)
end

And it might be used in our ERB like so:

<%= format_tax_rate(@sale.state_sales_tax) %>

This is some very simple code. It’s a naive approach to formatting the decimal value as a percentage to two decimal places.

More importantly, it’s only really needed in one or two views to display a formatted sales tax rate. Why then, should it clutter up our ApplicationHelper in order to be shared? And what happens when you have 10, 20, or more small methods just like this one?

With SimplestView, we can move the method to a better place. If we were to handle this in our Sales::IndexView, for example, we could do something like this:

class Sales::IndexView < ActionView::Base
  def formatted_state_sales_tax
    '%.2f%' % (@sale.state_sales_tax.to_f * 100)
  end
end

So, what did we do here?

We made our view match the conventions. For the SalesController#index controller and action, we’ll use Sales::IndexView as the context for the template now found in app/templates/sales/index.html.erb.
We made a nicely named method which contains just the formatting we need.
We use the instance method for @sale, which we assigned in our controller, and is accessible to us inside of the view.

What are the benefits? Well, we get nicer encapsulation. We don’t have to expose the instance variable inside of the ERB for this particular case

Hello OO!

After this section, I want to make a quick aside to point out something important. With SimplestView, we’re back to using Ruby as the Object-Oriented language that it is! If we need to share code, we make a module to mixin. Or, if it makes sense, we can use inheritence.

If we have another method formatted_county_sales_tax, we can extract a method to do the formatting. Then, if that new formatting method is used in multiple views, it’s as easy as extracting a module like SalesTaxFormattable.

In addition, we can now easily test, and refactor this view because it’s just a Ruby class.

Controller Assignment

One of the more troubling ways I’ve seen developers cope with growing complexity in their ERB templates has been to move calculations and logic BACK into the controller.

What started out as an action that assigns a single instance variable, baloons into methods full of before_action (previously before_filter) calls to assign and calculate many instance variables.

Let’s refactor an example:

class SalesController < ApplicationController
  def show
    @sale = Sale.find(params[:id])
    @purchase_count = @sale.purchases.count
  end
end

Now, we can output both data about our @sale, but we can also output our @purchase_count number, too.

Instead, let’s do it with SimplestView.

class Sales::ShowView < ActionView::Base
  def purchase_count
    @purchase_count ||= @sale.purchases.count
  end
end

Now, we can still output our purchase_count in our template, without cluttering our controller!

ERB Templates

By now, it may be clear what I’m going to say. But, I’ll say it anyway.

Extraneous logic in your ERB templates is very likely to be a code smell. Extensive branching logic, local variable assignment, etc. Time to get it out!

Inside of our SalesController#show action, we’ll likely render the template found in show.html.erb. It might have some code like:

<% if @sale.starts_on <= Date.today && @sale.ends_on > Date.today %>
  On Sale!
<% else %>
  Sale Starts On: <%= @sale.starts_on.strftime("%B %d, %Y") %>
<% end %>

There are two things in this code that we really don’t need. Both the if conditional and the date formatting code can be encapsulated in our Sales::ShowView.

class Sales::ShowView < ActionView::Base
  def active_sale?
    @sale.starts_on <= Date.today && @sale.ends_on > Date.today
  end

  def formatted_start_date
    @sale.starts_on.strftime("%B %d, %Y")
  end
end

This is probably my favorite refactoring. And, it is likely to be the one you will use on a regular basis.

But Why?

The benefits of these changes might not be the biggest pain you feel in your code, day to day. But together, all of the little changes you get to make, the added ease of refactoring, and the extra convention, make it a whole lot easier to write well-factored, maintainable code, every day.

I’m using SimplestView daily, in as many of my projects as possible. Not because the code I already have is horrendous, and not because there are no alternatives to refactor and clean up my code. I’m using it because it enables functionality in Rails that should be available already, and it gives me the conventions I kept searching for.

If SimplestView doesn’t strike your fancy, you can go the decorator route with the fantastic Draper gem. Both Draper and SimplestView accomplish very similar things. Regardless, you should try using one of them, and see how it helps you clean up your code.

A Beginner's Take on RubyMotion

2013-11-05T17:25:14+00:00

“A little over a year ago I was in bondage, and now I’m back out here reaping the blessings and getting the benefits that go along with it”

Pimp C’s last interview, from Jay-Z’s FuckWithMeYouKnowIGotIt

Like many others, I initially started to program because I had an idea. After a brief stint with PHP, I dove into Ruby/Rails and started developing my app, YourTurn. During development, there was a lot I wanted to learn about the technologies (and their minutiae) I was discovering, but my focus was getting something tangible to appear on my screen. This not only helped me prototype my app, but also drove me to learn more. (Topic for a future post? You bet.)

When I set out to make an iOS app for YourTurn, I faced a similar decision: either learn a new language - Objective C - or utilize something I was more familiar with to get something done. To be honest, I didn’t really consider the latter until I heard about RubyMotion, and when I did the choice was easy.*

You can learn more about RubyMotion on its website, but I want to use my inaugural post here to highlight a few of the benefits from the perspective of a new Ruby developer. First: RubyMotion’s use of, well, Ruby.

For example, here’s Objective-C syntax:

[[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]];

Here’s RubyMotion:

UIWindow.alloc.initWithFrame(UIScreen.mainScreen.bounds)

I don’t know about you, dear reader, but I find the RubyMotion way more readable as well as easier to type.**

As a beginner, a technology is only as valuable to me as the community supporting it. From the folks at HipByte to the authors of its many wrappers and libraries, it’s clear that the folks behind RubyMotion want it to succeed. I’m less than a year into the toolchain but I can confidently say I wouldn’t have been able to release version 1.0.0 of my app if it weren’t for resources like Clay Allsopp’s tutorial and PragProg book, as well as BubbleWrap, Formotion, and Sugarcube. Resources like these continue to grow each day, and I know I’ll continue to lean on them as I release future versions.

For me though, the best thing about RubyMotion is that it removes extraneous complexity from learning about Apple’s APIs. By not having to worry about areas in which I really lack expertise (namely Xcode and Objective-C) I can focus on making my app better. Of course, as I become more familiar with the APIs I could move to learning more about Objective-C if I wanted. But again, I wouldn’t have that choice without RubyMotion. With it I can prioritize learning the things that help me reach my ultimate goal of improving my app quickly.

So am I saying RubyMotion is for everyone? No. It’s obviously best for people with varying degrees of Ruby knowledge.

Is it perfect? No. A few downsides are that:

It is not fully open source.
It has had some big issues like RM-3 (which, in fairness was prioritized and resolved by HipByte).
It is a bit tricky with blocks.
The license is pricey at $199.99.

Collectively, these issues represent exactly what RubyMotion is: a growing toolchain from a bootstrapped business. Aside from the price, they shouldn’t dissuade anyone from getting their hands dirty.

Simply put: for Ruby developers, RubyMotion provides a lower barrier of entry to iOS Development. For beginners like me, it’s the best tool to use if you want to get an app out quickly. For that, I’m grateful to HipByte and the community behind RubyMotion.

* I know there are other frameworks that utilize technologies such as HTML/CSS/JavaScript but I haven’t seen any that deliver experiences similar to those that come from native apps. Perhaps these frameworks are the future - and I’ll certainly keep an eye on them - but it’s just not there yet.

** Granted, this is because my background is in Ruby and that’s all I know. But still.

Ruby Resty

2013-09-30T11:50:43+00:00

I spend a lot of my time working with REST APIs. At home, at work, at hackathons, it’s GET this and POST that. Resty is an amazing tool to simplify sending multiple HTTP requests with the same data and headers. My favorite part is having per-host configuration so you never have to worry about remembering your cryptic, hashed API keys and header names.

It’s such a great tool, except resty currently doesn’t work in zsh.

After thinking about it, a shell script isn’t the best approach for community development. Coming from a Ruby background, shell script syntax is so hard to read. What about adding functionality to Resty? Can I add a new command that Resty supports? Can I have nice tests documenting functionality? How about some continuous integration?

And I embarked on a Ruby Port of Resty.

What started as a straight port, morphed into something I didn’t expect.

I started a spike with my own REPL. Thinking more about features, I realized I should be using something like the superb Pry. Pry allows Ruby Resty to be easily extensible with it’s Custom Command infrastructure. Since we’re using plain ol’ Ruby, we should use plain old ruby objects!

POST /api/nyan {"name": "grumpy"} # JSON String
POST /api/nyan {name: "grumpy"}   # Ruby Hash
POST /api/nyan data               # User assigned variable in the REPL

Pretty nifty.

One way Ruby Resty differs from Resty is host aliasing. Rather than specifiying a hostname, which maps to the config file, users can specify an easy to remember alias defined in ~/.ruby_resty.yml.

This is still a working in progress and I would love some feedback. If you’re interested, submit a ruby-resty issue or reply to this post.

Welcome to Growing Devs

2013-09-10T00:12:52+00:00

Welcome to Growing Devs. This is a blog for a group of devs to share their experiences – through writing and through code – in growing software, themselves, and other developers. Or, just to write about any nerdy thing we want to write about.

The group is fluid. Some devs will write only once, and we hope that new devs will join and write. All editorial review and publishing is handled through GitHub pull requests.

This is an experiment for us. But we think it has merit. In particular, we think this format has the benefits of:

inspiring and challenging us to write
sharing the burden of editorial work
sharing success, recognition, and pagerank
not binding us to a single language, topic, or company
providing space to share content, without maintaining a solo blog

You can follow our RSS feed, follow us on Twitter @growingdevs, or we would love for you to contribute.

The site is open source under the MIT license, all content is copyright of the original authors and not of Growing Devs (which is not really an organization of any kind anyway).

Thanks for visiting, we look forward to sharing some new and wonderful content in this experiment of ours!

And again, if you would like to contribute a blog post you’ve had in the back of your head for days or months, please feel encouraged to send us a pull request.