What the Hack!

Hackathons give development teams the opportunity to work in different ways to business as usual. Some of the strongest motivations for undertaking a hackathon are as follows:

A departure from current tasks

Hackathons give the opportunity for developers to get away from “ground level”, to explore tasks outside of their normal activities. Whilst seemingly simple, this profound shift in perspective underpins the many other benefits of the hackathon. Having dedicated time and space really unlocks an atmosphere that’s distinct from the day-to-day, where the normal rules don’t necessarily apply, and where creativity can take place in a detached, liberated manner.

Focused teamwork

Moving away from the typical organisational structure can help colleagues to work with people that may typically form part of different work streams. Blending disciplines in this way, such as a product manager with a developer and a graphic designer can yield an interesting final product and presentation in my experience. This factor is something that can be tailored for any given hackathon, and a key consideration that would support limiting the event to developers is that there’s potentially a higher likelihood of producing something production-ready during the event as opposed to something compelling but not directly usable. If usable innovation is key, then restricting participation can be a sensible trade-off, but cultural exchange is not necessarily maximised.

Embracing the unexpected

Whilst not necessarily a primary goal, all hackathons offer the opportunity to fail without major repercussions. This is a vital component of the culture underpinning a hackathon. Teams will manage the risk by employing disciplines that lead to predictable, successful, delivery day-to-day. This includes things such as TDD, task management, setting and testing assumptions and so on. At the same time, teams might be using tools that aren’t necessarily familiar. This can lead to major knowledge transfer, but it’s also something that can go wrong. Fostering an enterprising, inquisitive, approach to problems can really refresh motivation, and so can help company culture. Too much of the unknown can potentially limit the amount of directly-production-ready innovation within the confines of the hackathon.

Challenge

Coupled with embracing the element of risk, some hackathons ratchet this up by awarding prizes, sometimes organising direct competition by giving the teams similar objectives and starting positions and allowing each team to work (potentially in secret) to achieve the best-optimised outcome for a given problem. Competing in this way can yield interesting innovation outcomes, because it can potentially work around typical perspectives and viewpoints that might prevent solving a given business problem. It can also foster culture, if the business functions best through potentially competing delivery groups, or if–as is more typical–the hackathon is taking place at a community event where several companies that compete in real life can blend their teams together. This is not always the case, so sometimes hackathons may be better culturally if they have a more inquisitorial than adversarial nature, with presentations at the end but no awards ceremony.

Collaboration under pressure

In all hackathons, the short timescales can work well as a forcing function. This can give hackathons a common flavour, with certain rituals that can become familiar and so can lead to quickly forming new teams in the case where people have attended a hackathon previously (if not, it’s pretty easy to buddy-up). This is great for culture and can lead to great onward working relationships and stories to tell. Pressure helps innovation, too, because only the strongest or most interesting ideas can be explored with the fairly small team sizes available. This means that the ideas produced tend to be small but mighty, with a lot of surface area for further innovation after the hackathon.

It appeals to our nature

Curiosity is an important part of being a developer. Hackathons amplify this by providing a space to indulge curiosity, and to tackle new projects for the benefit of the group. So culturally it enables the organisation to recognise the forces that motivate developers, and from an innovation perspective allowing these discoveries to take place can lead to people gaining material insights that they can use to enhance innovation in their day-to-day work.

Work in different ways

As well as escaping the usual day-to-day workloads, hackathons offer the ability to work face-to-face when work is normally done remotely, using break-out areas, and in a competitive context may even go so far as to have a central “exchange” network that can present the relative progress of each team on an interactive leaderboard. Working in this way is hugely important for culture, because it enables everyone to work as an equal, and often subverts typical leadership hierarchy. People get to meet the person behind the video chat and interact freely. Sometimes you hear things said like “you’re taller in person”. Again, this can serve to humanise relationships between colleagues normally working remotely from one another and so can really help on a daily basis once the hackathon is over.

Putting theory into practice

So, having explored some of the theory, how does a hackathon actually operate at Form3?

To loosely paraphrase the old adage, one doesn’t grow an exciting fintech startup to scale without developing a few interesting ideas that would be fun to explore. Form3 hold regular hackathons to take on some of these challenges, for example security, API integration, incident support, visualisation of data flow, Slackbots, and creating new product ideas from data science fundamentals. We tend to have a healthy backlog of ideas to explore.

Our most recent hackathon was an excellent opportunity to pursue some of these challenges, whilst also providing a way to deepen those certain intangible aspects of cooperation free of the latency that even the very best web conferencing solutions can’t quite manage. The chance to break bread with people normally located thousands of miles away from each other didn’t hurt either. As a remote-first team we made sure to also include those colleagues that couldn’t make it by embedding them in many of the teams and by live-streaming and recording the presentations.

Getting started

To help ensure an interesting couple of days at the hackathon, task elaboration and selection began a few days before the event. We used GitHub issues to self-organise into teams that could focus on each effort. Whilst there were no particularly strict rules regarding team composition, the thirty or so of us all settled into teams of roughly five people that spanned our normal working groups.

Four of the teams were able to start their journeys on top of earlier efforts that emerged naturally from the various internal and external ecosystems that grow around the various elements of our tech stack. Some of the teams were using various interesting Linux distros and penetration tools, so they made sure to have everything configured prior to beginning the event. We also used a small selection of our common internal documentation tools to spin up a few slide decks, shared repositories, wikis and so on.

Once initial introductions were made, we settled in and began to engage with the task lists each team had created. Typically we had around ten tasks per team, to give each of the team members one significant milestone task per day, plus we then had at least one stretch goal per team member. Throughout the event, there were plenty of opportunities to walk the room, talk to everyone, and to exchange ideas for stretch goals. A significant number of people present were quite seasoned to the normal ebb and flow of a hackathon, but every team made an excellent effort to include everyone’s ideas and most seemed to settle upon regular ad-hoc stand ups to ensure that they were on track.

Variety

First up, “Red Team” used the opportunity to flex their security and penetration testing skills at hackthebox.eu using Kali Linux, and this made for one of the most interesting presentations. Having an intensive focus on these issues is important day-to-day, but being able to explore these issues in the abstract free from the constraints of an established system definitely made them more approachable and less intimidating.

Team “API CLI” was focused on implementing the 80% of functionality that we need from one of our commercial-off-the-shelf incident-support tools. The existing tool definitely works for us, but being able to expose exactly the right data in a high-pressure situation can help us to raise our game further. Go proved to be a great choice for a REPL that tied together several APIs to allow incident support teams to correlate events, navigate between business objects, and cross-check these items with relevant logging information. The team was able to meet all of their major objectives.

Team “Payments Flow” were aiming to visualise some of the interesting flows that occur through our platform, which is also something that can help to raise awareness of the normal behaviours that result from the emergent complexity of a sophisticated distributed system. Netflix Flux helped to inspire the team to provide a powerful visualisation using Processing.js but when we first connected it to the underlying data streams the payments were moving through the dashboard too quickly for the eye to follow!

Team “Slack Deploy” were enhancing Slack, which is a tool that also helps us to deliver precisely the right amount of information to the proper people, so a team built upon our existing stable of Slack integrations by developing a tightly-focused and useful Slackbot to assist with the ancillary tasks around deploying new features, ensuring people are aware of necessary steps and so on. We already have a mature release process, but we are always thinking of ways to enhance it further.

Finally, Team “Jupyter Weather” conducted some very sophisticated data science analysis of warehoused anonymised transaction data using Jupyter Notebooks and Python and were able to explore some interesting patterns that occur in our system, which will potentially help us to develop new aspects of our existing systems and potentially to approach other areas that are on our planning horizon.

No contest

Some hackathons focus on optimisation of a single problem within a tightly-scoped problem space. This can be an excellent opportunity to deepen teamwork, and can lead to some great innovation, but can limit the ground covered and there can be a degree of duplicated effort inherent in the process if the problem is not expertly framed and/or has been explored previously.

Other hackathons are broader in scope, and more collaborative, with each team going their own way but sharing ideas in a free-form way. This can allow for more ground to be covered, but occasionally can lead to differing levels of achievement depending on the relative difficulties of each problem. Although judging the presentations can help with the process of deciding which team is the winner, at Form3 we decided that the presentations themselves were fascinating but nominating a single team the winner would not necessarily be meaningful.

In keeping with the nature of the event, teams made sure to have lunch and to break promptly at the end of the day. All-nighters were definitely not in scope.

The presentations were delivered with great humour in a relaxed and egoless environment. They provided a great capstone to an interesting couple of days.

After that, the teams carried on some of the discussions and some tasks were formalised for more in-depth follow up as part of our normal development cycles. We also took the opportunity to reflect upon the other ideas in our backlog for the next hackathon. This list already has thirteen promising ideas and everyone is able to hang up their own shingle on the kanban board should inspiration strike.

Since each team was able to tackle most if not all of their planned problems with well-tested, production-ready code, and many stretch goals were also hit, we thought that the event had been very successful. A lot of the solutions produced can potentially be immediately applied, and for those requiring further development, strong potential was shown through the work done and the presentations at the end of the hackathon. Two of the solutions have made it to production already.

Given the success of the event, we plan to hold another in a few months, and we will probably continue to iterate on what seems to be a winning formula.