Peter Mellett

Cryptocurrency Exchange Website

2019-09-04T00:00:00+00:00

Cryptocurrency Exchange Website

My client was approached to facilitate a rebrand and update the website for a cryptocurrency exchange. Their current focus is on professional clients. I was brought in to implement the new design with an emphasis on motion and transition while adhering to low carbon emission and responsible web development principles.

Photo by Tomasz Frankowski on Unsplash

Requirements

Being a cryptocurrency exchange, security was of utmost concern to the client. They placed restrictions on the technology we were able to use and the approach we should take. This aligned with the internal team’s ability to review and audit the codebase along with use of standard metrics and error capturing tools.

To avoid introducing dependencies on too much third party code we avoided libraries where possible, instead choosing to implement lightweight alternatives which were easier to audit and maintain.

In order to provide a top quality and fast service from the site we elected to use a static site, only small amounts of the site are dynamically populated and have established APIs around them.

Node was expressed as a preference in early planning meetings, so it was the clear choice for rendering and delivering the static pages.

Methodology

Because the site was designed to be composed of modular blocks, I chose to use React with Serverside Rendering (SSR). This meant that if in future we wished for a fully interactive site then hydrating the application on the client side would be possible and not require rebuilding all the application code as a duplicate.

Initially any custsom JavaScript behaviour in the browser was written in plain JavaScript and injected into the page as a bundle. These scripts remained reasonably small as CSS transitions and animations were used to take advantage of hardware acceleration and the reduced CPU load per visitor.

Results

n.b. photography and text have been replaced with generic alternatives here, so please excuse any poor image selection!

The site contains some novel features using CSS animation and interactivity without drowning out the content.

Here is a small selection of some elements from the site.

A CSS-animated carousel with JavaScript triggers on the navigation arrows and blobs. It uses an overflowed container to allow the elements to be moved using a transition and translateX which is calculated from the available slides. The same system is used for the text.

Photo by Duy Pham on Unsplash

A “meet the team” section, designed to be fully responsive and accept any number of team members. Each has a personalised bio revealed on click or tap, with a hover effect to remove the blue hue.

Images from https://randomuser.me/

This is all very nice but doesn’t reveal the motion of the elements, here is a screen recording of the create your account section. It’s a series of animations tied together by a delay so they can all be triggered by adding a single class to the component.

Please excuse the autoplay video, the gif was too low framerate to show it off.

The project has certainly been a learning experience, CSS animations were something I used to avoid but they are very accessible and with a bit of thought behind them can have a big impact.

Delayed indexing with background_job and searchkick

2018-10-15T00:00:00+00:00

Photo by Icons8 team via Unsplash

The problem

A client had problems with their search indexes being out of date compared to the latest updates to their content. Due to the distributed nature of the app a blocking solution provided a less-than optimal user experience so a more appropriate solution was required.

The situation

The software comprises of three applications:

A Ruby on Rails JSON-speaking API
A React front-end for consuming the data as a user
A second React app for backoffice tasks.

The application is search-first in design so it’s important that data is findable and accurate in the search results. With the existing implementation, reindexing may not occur until a server restart which could be some time.

The search functionality is backed by Elasticsearch and interfaced using the searchkick gem.

The solution

I chose delayed_job as my queue manager, owing to the fact that this is a commercial project and delayed_job does not place any restrictions on use with the MIT license.

Following their readme, it was easy to fit to the specifics of the application. Since the application is using mongoid I needed specific support for this, which what provided with the delayed_job_mongoid gem.

# Gemfile
gem 'delayed_job_mongoid'

Because the application is using Rails 4, it is necessary to specify the queue adapter in application.rb

# config/application.rb
config.active_job.queue_adapter = :delayed_job

delayed_job needs a redis instance to manage its queues so I provided one on the default ports.

Queueing jobs

Now it is easy to delay any existing method by calling it through the delay method, like so:

# old
Article.reindex

# new
Article.delay.reindex

The real data models in question have some embedded models which needed to also be reindexed, as their data is provided to searchkick to represent one entity in the elasticsearch index.

In order to accommodate this and to automatically queue a reindex on save for the model, add an after_save hook with the reindex action.

# models/article.rb
def after_save
    self.delay.reindex
    # and any other relations that are embedded!
    # If you're using active record then ymmv
end

Now whenever our model is saved or updated, it’ll automatically generate a job to reindex the model. Neat!

Consuming the queue

To recap so far there is:

A method to queue our long-running jobs
A datastore to back our job queue
Integration with mongoid and the after_save callback

The next thing is for us to actually execute the queued jobs.

In local development this is pretty straightforward. Run our rails app again but use the provided rake task to wait and listen for our jobs and take the necessary action.

In the simplest form this can be done with

bundle exec rake jobs:work

The deployment environment understands Procfile so add a queue process to that and ensure it’s running as part of the deployment monitoring process.

# Procfile
web: bundle exec puma -C config/puma.rb
queue: bundle exec rake jobs:work

Monitoring queue health

The rake jobs:work task is fine, but not very chatty so it can be hard during development to ensure that jobs aren’t being missed or some other problem is occurring now that we’re relying on a separate worker process to do some of our work.

To ensure our queue is being consumed correctly I took a couple of steps.

Firstly in the API application I provided an endpoint to poll for the queue status. This simply returns the count of jobs in the delayed_job queue.

def queue_status
    data = {
        in_queue: Delayed::Job.all.length
    }

    render json: data, status: :ok
end

The admin application can then poll this endpoint and report on the state of the queue to the administrator. It should be obvious if the number is high, really anything above zero for more than a few seconds, or rising that something is not right with the queue.

Secondly for local and staging purposes including the delayed_job_web gem mounts a small sinatra application which provides stats on queue length, pending, processing and failed jobs.

It doesn’t go into huge detail but it’s a massive boost to confidence and being able to test out tasks and actions through the rails console with visual feedback.

If you’re using mongoid like I am then there’s an open pull request with mongoid support and you can use the proposed fork in your gemfile by targeting https://github.com/brunoporto/delayed_job_web

# Gemfile
gem "delayed_job_web", github: 'brunoporto/delayed_job_web'

Delayed job web quickstart

Conclusion

All in all this has increased confidence in the search indexing with visual feeback to the administrator and ensured that updates do not tie up the web server unnecessarily.

If you have a Ruby on Rails application with tricky bugs or unexpected requirements, why not get in touch?

Work Experience: Fun and Games

2018-07-20T00:00:00+00:00

Photo by Patrick Tomasso on Unsplash

For the last two weeks of school term, I took on Joe for some work experience. As punishment I made him help me write this blog post.

We decided to take on a project related to our shared interest of cycling and, hopefully, produce something fun for our cycling club to use. Joe’s words are in italics.

Our initial idea was to create something to use for the Wadebridge Coasters cycling club (as we are both passionate about cycling) and also use the strava API to create a website that will add even more fun into the club.

I proposed the idea to create an automated version of the club’s “challenges” page, which uses popular local Strava segments and has top-3 leaderboards for each. Joe agreed to the project so we set about planning how it would work.

We came about the idea of making a challenges page that shows the rankings of people in the club up climbs and loops around the local area. These challenges update daily and we hoped would bring a fun ‘get up and go ride’ addition to the club.

Over the two weeks the notebook would take a bit of a battering. Everything from planning, to designs, to “to do” lists and problems encountered. We made a habit of writing down as much as we could to help keep momentum during development.

Firstly Joe was tasked with implementing the HTML for the pages we designed. Using Bootstrap provided a good foundation and allowed Joe to lay out all the pages before we needed to worry about sourcing any data. It also helped to guide the design as we progressed.

I firstly got the layout of the pages and then added javascript and css to style these and get all of the features to work with help in the trickier more advanced coding.

Once this step was completed we used a template-first approach to creating our backend application. Firstly identifying the dynamic data in our design, checking it against Strava’s API documentation.

One of the main problems we came across was having personal profiles, this idea was made redundant as we did not have all of the access that we wanted and could not get a lot of data.

Strava have recently reduced the available data from their JSON APIs so we were unable to implement some features, such as linking from the leaderboards to club member’s profiles, or building our own profile pages with compiled ‘best results’ built from the challenge leaderboards.

This aside we worked hard to achieve all of our goals set out at the start and it turned out to work fully and I am very happy with the final result.

Very refreshing to create someting from scratch of our own volition and release it within two weeks. It’s a shame we had to cut some features but we adapted to the roadblocks well and ended up with an app to be proud of.

I’m sure there will be further development on the project and we’ll dust off the feature wishlist in future.

If I did it again things I would not change much as I really enjoyed it! But if I had to change something it would probably be making more notes on paper of what I wanted to do before starting the task as I started doing this before the end and it was very beneficial instead of spamming stuff into the code until it worked.

We spent some time on the final day putting together a retrospective of the project as an example of working in an agile fashion, although there is no ‘next sprint’ we went through the process of understanding where the strengths and weaknesses of our team are and where we would take the project next.

It’s been a very enjoyble two weeks and great to see Joe’s progress even in such a short time. Finally, some parting words from Joe:

From this experience I have learnt a lot. The main thing I will take from this is the skills of some more advanced programming features. And also how to manage time efficiently. But I have also learnt other important things like working in an office environment and how different jobs work.

I have definitely taken a lot from this and I have a clearer picture of what I want to do when I leave school and go to college/university. I would like to thank Pete for taking me on and giving me a very intuitive and hands on experience and providing a genuine work experience

You can see our challenges application at https://club-leaderboard.herokuapp.com

My First Month

2018-05-03T00:00:00+00:00

Photo by Nathan Dumlao on Unsplash

I’ve put my first invoices out. What a relief! It’s been a whirlwind few weeks so I thought I’d write about it a bit.

My previous employer, Foundry, unfortunately had to close doors last month. We had an amicable end to the business and made the mature decision as a team to do so. Still, that left the five of us at a loose end with not much time to do something about it!

I decided that the next step in my career should be to go freelance and see if I can use all the lessons learned over the past 6 years to manage my own workload and have a bit more flexibility in my life.

Being the boss

Now that I’m unemployed - Uh oh! I’m in charge and also responsible for 100% of my output. There’s a certain stress that comes from not having a team to work with if you have a bad day or need to flex your availability a little bit.

On the other side I get to pick how I fill my schedule, so it’s less a feeling of ‘you must do x today’ and more a feeling of ‘I have decided to do x today’, which gives a better feeling of control over my day-to-day.

Selling myself is a new skillset I’m having to learn on the fly. It’s a strange feeling to be your own salesman- it feels a bit like a conflict of interest.

I’ve also been able to pivot to selling my skills in more of a specialist role, targeting technologies that I prefer to work with while using the foundation of knowledge I’ve built over my career so far.

Work/life balance

I do all this to provide a good home for my wife and boys. Making sure to shut the laptop after a good day’s work is important.

Trying to establish the new normal with a routine that gets me out of the house every day is tricky too - I’ve got the option of working from a few different locations so I can mix up my week and hide from the teething baby.

An unfortunate side-effect is that cycling has taken a hit. Without my one-hour a day commute it’s been hard to keep the miles up. This is something I hope will settle down as my routine comes together and I can get back to keeping myself fit.

A good start

Through panicking, more than a bit of luck, some great friends, and sending buckets of emails I’ve been lucky enough to have a strong start.

I was up against the deadline that our meagre savings could provide while I established myself. There’s no safety net and guaranteed salary anymore.

Allowing myself more freedom to attend social events in tech locally, reach out to other freelancers locally and generally putting myself out there has given me a lot of confidence - and helped me make new friends!

It would have been all too easy to hide away and work without the social interaction, so I’m glad I didn’t go down that route!

Overall

So far, so good. The stress levels are coming down as I get myself established and build a good working relationship with my clients.

Thanks to those of you that have listened to me panic for all different reasons & my clients that have shown faith in my abilities - despite a slim portfolio initially!

If you’ve read this far then why not get in touch and tell me about your experience? I’m still finding my rhythm and any tips you may have are very much appreciated.

Injectable Angular

2018-04-16T00:00:00+00:00

Photo by Clint Adair on Unsplash

For a recent project I needed to be able to deliver an angular application as a stand-alone widget that could be dropped in to an existing website.

The danger here is that the third party website could be using any number of frameworks, including Angular, that may interfere with the bootstrapping of the code.

In this particular example the third party was using requirejs in their front-end code and the angular libraries are all module-aware.

This caused the application to attempt to load itself through require when in fact it did not need to do so. The application code was not expecting this and anything that would normally be bound to window - angular in particular - was not present at runtime.

This caused some headache until a solution presented itself.

Wrapping the application in an iife to give it its own scope while redefining define so that for the parse and execute time of the code it would not be present.

A pre and post snippet files were added to the build that opened and subsequently closed the iife were added to the gulp build on the project.

This meant that code my colleagues and I had integrated elsewhere did not need to change its knowledge of the widget in order to continue operating correctly, but the new sites that would be affected by angular would be safe from intefering with, or being interfered by the widget.

A similar approach was used for sites that pre-included jQuery, which is used to enable the fuller jQuery features over jqLite in Angular 1.x.

pre.js is the first file to be concatenated during build. It wraps all the code in an iife and reassigns define so it can’t be picked up by any of the application’s modules.

post.js is the last file to be concatenated in the build. It re-assigns the original define to where it was and closes the iife.

Additionally I considered another problem:

What if angular has already been bootstrapped?

In this situation angular will refuse to be included again with WARNING: Tried to Load Angular More Than Once which may (or may not) be a problem.

For this eventuality I decided that a version check should suffice to ensure that the detected version was equivalent to the bundled version. In that case I could use the existing angular version and not bother to attempt to run the bundled one.

In addition a check to ensure that, if the booted version of angular is new enough, it has been loaded with jQuery instead of jqLite. If not then angular.element can be extended.

pre-angular.js is added before the main angular library is included. This way it can detect an already-bootstrapped angular and take evasive action. The widget has a hard minimum supported version number so it has to fail if this cannot be met and a new copy of angular cannot be loaded to run it.

post-angular.js then comes after angular.js itself. It simply closes the iife from pre-angular but does not do any cleanup of its own.

For a more visual guide. Here’s how the logical blocks end up post-build:

note: I am no graphic designer!

At last the build provides a distributable, injectable application that is isolated from it’s runtime context without resorting to iframes! Phew!

I’ve got some years’ experience with Angular now, and this is the first time I’ve not had any control over the environment- beyond the browser that is!

Johnny-Five in Production

2018-03-28T00:00:00+00:00

I found this writeup that I never published. Oops! Here it is at long last

Photo by Mariana B. on Unsplash

Back in the halcyon days of 2015, I was involved with a project to build a virtual periscope. The brief was that motion of a periscope, sensed via a digital compass, would control a panorama of a cold-war bunker. Said bunker is actually there under your feet, but it’s not the sort of thing you want to be drilling a big hole in to install a real periscope in - you have to lower your equipment down on a bit of rope to get in. It’s certainly not the sort of place you expect to deploy software.

We required a relatively simple way to read from our magnetometer - an HMC5883L if you’re interested - and feed this heading data to our panorama running in the browser to position and interact with the virtual tour.

We chose a Raspberry Pi since it can run a large part of our application and interface with the magnetometer over its GPIO pins. This also opened up the choice of technologies we could use to prototype and trial the idea before committing to a solution.

The decision was neatly narrowed down by a quick glance around the office. We didn’t all have the Ruby or Go experience to use something like Artoo or Gobot, but we all have plenty of JavaScript experience.

I was aware of two robotics libraries for JavaScript: CylonJS, and johnny-five. In the end familiarity won out as I’d used johnny-five more than cylonjs while toying about with my own Raspberry Pi.

As an aside, all of the linked libraries rely on the firmata protocol in order to communicate with the host device. In the case of an arduino a program is flashed to the arduino that bootstraps and runs firmata, accepting instructions over a serial connection. With the Raspberry Pi it uses raspi-io, which uses a firmata-compatible API. It has the downside of requiring a serial connection to run your code.

Getting started with johnny-five is about as straightforward as it gets:

'use strict';

var RasPi = require('raspi-io'),
    five  = require('johnny-five');

var board = new five.Board({io: new RasPi()});

with the board object, we can now make connections over the Raspberry Pi’s GPIO and use asynchronous features of the language to wait for and respond to events from the hardware.

Firstly we need to tell johnny-five to expect a magnetometer to be connected- luckily there is a preset Magnetometer contructor (it’s actually a compass object with some presets for the HMC5888L sensor, see the example code for more details.) The default read rate is 50ms, which can be adjusted up or down depending on the datasheet of the sensor.

board.on('ready', function() {
  var compass = new five.Magnetometer();

  /* ... */
});

Now we have our compass object, which is an EventEmitter, we can listen to a set of events by binding callbacks and respond to them in our application.

The event we want to listen out for is called headingchange, this is fired when the magnetometer reading has changed since the last reading and contains a heading object bound to this. There is also a data event which fires on each read of the heading regardless of all previous. Since we don’t need the data if the reading has not changed, we can safely use headingchange.

compass.on('headingchange', function() {
  // Send the new heading to the waiting browser
  // to tell it to pan the virtual tour appropriately.
});

The best part about the johnny-five code is that it stays out of the way. A colleague with very little johnny-five experience was able to read and work on the code in short order since it introduces no new syntax and keeps the simple event/callback logic that is familiar.

I didn’t find many articles about this so I think it’s safe to say that most johnny-five (and really, most firmata-based) code is being used in a hobbyist or setting. Since we had an unusual environment to run our application and didn’t need to run our hardware in an environment where firmata would be impractical it was the perfect chance to use it in a production setting.

Cornwall Ruby

2018-01-01T00:00:00+00:00

Graphics kindly donated by @djnetherton

Cornwall Ruby is the RUG I’ve set up to find other like-minded developers and interested amateurs in the county.

We use Slack for our normal chat and arrange semi-regular meetups.

If you’d like to get involved don’t hesitate to join us. Everyone is welcome regardless of ability or level of interest - it’s always nice to meet someone with a shared interest!

How to build a custom session driver for Laravel 5

2016-12-06T00:00:00+00:00

Originally written for and posted on Foundry

Photo by Ilze Lucero on Unsplash

For a recent project built using the PHP framework Laravel a requirement was the ability to have certain resources be removed when the current session expires.

Using the built-in database session driver we maintain a pivot table linking Entities in a transient state to the current session. Later, if the current session logs in or registers an account we can store the processed entities against that account and remove them from the transients table, disassociating them from the current session.

This was brilliant and worked flawlessly-

-until the session garbage collection attempted to clean up some old sessions and didn’t know to remove their related models.

This garbage collection, which can be triggered at any time (with a default probability of 2 in 100 requests) will affect another user’s session if it causes an error. In order to prevent any foreign key errors from occurring we need to ensure that all would-be orphaned records are removed before the session itself is destroyed.

To accomplish this, we devised a custom session handler to extend the provided database session handler.

<?php namespace App\Extensions; // we have an Eloquent model that allows using the session table with the querybuilder methods and eloquent fluent interface- read only! use App\Session; // use the provided database session handler to avoid too much duplicated effort. use Illuminate\Session\DatabaseSessionHandler; class AppDatabaseSessionHandler extends DatabaseSessionHandler { /** * The destroy method can be called at any time for a single session. Ensure that our related records are removed to prevent foreign key constraint errors. * * {@inheritdoc} */ public function destroy($sessionId) { $session = $this->getQuery()->where('id', $sessionId); // tidy up any orphaned records by this session going away. $sessionModel = Session::find($sessionId); foreach ($sessionModel->myModels as $model) { $sessionModel->myModels()->detach($model->id); $model->delete(); } $session->delete(); } /** * Replicate the existing gc behaviour but call through to our modified destroy method instead of the default behaviour * * {@inheritdoc} */ public function gc($lifetime) { $sessions = $this->getQuery()->where('last_activity', '<=', time() - $lifetime)->get(); foreach ($sessions as $session) { $this->destroy($session->id); } } }

Registering the AppDatabaseSessionHandler is done through a provider, which is set up the same way as the built-in DatabaseSessionHandler.

<?php namespace App\Providers; use App\Extensions\AppDatabaseSessionHandler; use Illuminate\Support\ServiceProvider; use Illuminate\Database\ConnectionInterface; use Session; use Config; class SessionProvider extends ServiceProvider { /** * Perform post-registration booting of services. * * @return void */ public function boot(ConnectionInterface $connection) { Session::extend('app-database', function($app) use ($connection) { $table = Config::get('session.table'); $minutes = Config::get('session.lifetime'); return new AppDatabaseSessionHandler($connection, $table, $minutes); }); } /** * Register bindings in the container. * * @return void */ public function register() { // } }

For our application, it relies on this functionality so it’s fine to hard-code the config into the app. Otherwise we would edit our .env files. To make it a simpler default for the application, editing it in .env.example would be a good idea too.

Now when a session expires, it transparently cleans up any model that has not been committed to permanent storage. Neat!

For an alternative approach, and one that may become more valuable once the project increases in scope, would be to leverage queuing and implement a garbage collection job that looks for orphaned records and removes them from the database.

Comprobo

2016-01-01T00:00:00+00:00

How do you know your training was completed and understood?
Do you know who is really being trained or tested?

Photo by Craig Garner on Unsplash

Comprobo approached us with the aim to rebuild a prototyped interface for their automated invigilation and identity verification solution.

Choosing to approach the problem from a design-first perspective, we produced UX and UI design guidelines using atomic design patterns.

For the implementation we chose Angular for its stability and battle-tested APIs in business of all scales, from start-up to international bank.

Over the next few months we prototyped, reviewed and implemented sections of the user interface, as guided by stakeholders and prospective users.

Consuming a set of structured JSON APIs and responding to user events across Angular directives we built a modular system that can be expanded and reworked without massive rewrites and allowing us a degree of flexibility to replace modules that are unsatisfactory with improved versions as and when they are required.

In particular we were tasked to solve problems around capturing webcam streams, audio information and device and connectivity information to provide administrators a huge wealth of information to help them understand the nature of their user’s actions and to inform them on their subsequent actions.

Using Agile methodologies we regularly review progress and priorities with other stakeholders and remote members of the team.

My duties on this project include supporting third parties integrating with the software, architecting and implementing modules and features in the Angular codebase, scheduling, and managing ticket backlog.

I also provide some backup support for the AWS infrastructure and some back-end services when required.

MatchHero Cycling

2015-01-01T00:00:00+00:00

I was tasked to build a prototype for a social cycling application, used to advertise and respond to people looking for riding partners or clubs in their local area.

In order to facilitate rapid development the Laravel PHP framework was utilised and the core of the application based around affordable and powerful third-party APIs.

In order to give the best coverage for signups, the Strava API was used to build in social sign-up and login with ready-populated profile data including avatars, unit preferences and names.

Using the Strava API allows for future expansion of the platform to include more functionality based upon the data provided. It also provides access to the same privacy restrictions as the user would use with Strava, which we can then honour in the MatchHero application.

To build out the ‘post a ride’ and ‘find a ride’ features, the Google Maps API was used. Allowing a user to select their location by latitude and longitude, if known, or a street address if not using Geocoding services.

During the build phase we used a round of wireframe and whiteboard sessions to target the desired functionality and MVP features that we thought would be appropriate for the platform.

As keen cyclists in the team at the time, we all had a say in the design and initial functionality- and also a keen eye for what wasn’t working after launch. Some adjustments were made post-launch to improve the user experience and provide a solid application to extend and promote in the community.

Using a prototype and review process I put together the initial release in around two weeks of development time.

If you’re interested in the more social side of cycling then visit MatchHero Cycling Club.