Big Data, A Stick

dsc05804Airplanes streaming half a terabyte of data per flight…83% of travellers having a smartphone and generating three gigabytes per month…every backoffice system storing eons of log files…

“The Big, Complicated Tomorrow has arrived” say business theorists. “It’s data-driven.”

“What’s brown and sticky?” say I.

A stick.

Because technically, tomorrow becomes reality at midnight every day. And Big Data is just a stick.

Wooden poles have been around for a very long time and so has the idea of the data-driven enterprise (the 1970s, oddly a date some iPhones consider The Big Bang of computing, meaning they crash if you try to reset them to any earlier time). Now, however, there are more sources with an increased volume flowing per information channel.

And if we live in the information age, that of tidal waves of data continuously crashing over (and into) us, what are we so well informed about? Well, I say it depends on how you apply the wooden tool.

A stick pokes, digs, and uncovers unseen gaps. Let’s take an imaginary 6-hour sector which arrives late 65% of the time in the last 2 months. Reports to Operations Control dispatchers indicate that ground staff does not have enough time to prepare the aircraft during turnaround. Ground services have reported that police have to first remove at least two drunken and unruly passengers per arrival, hence the rotational delay.

A more involving question would be “Why in the last 2 months?” Let’s expedite the imaginary discussion: catering has started to load 30% more alcoholic drinks since the start of the new season combined with “in-your-face special offer” adverts, hence higher sales and QED.

Investigating this has taken two full weeks of the OCC manager’s time. Correlating the digitalized data sources (delay times, load factor, cause of police intervention, cost of postponed ATC slot) with the (initially unsuspected) catering company reports has been a manual task, not least because incoming and outgoing merchandise gets noted using pen and paper (a copyrighted process). This assembly has been required to answer the next and most important question: how much to reduce the volume of alcohol available without impacting sales considerably and simultaneously reducing the delay which started the investigation.

Here, a digitization gap cracked open without even focusing on Big Data’s preferred fuzzy source milieu (in this example, product placement).As with all digital fads, Captain Obvious would suggest to first establish where the enterprise should be in order to create senior-level guidance.

To get you started on creating magic with The Stick, I propose an action plan:

  1. Call in an Ops Engineer, a Technician, an IT Analyst, a Flight Dispatcher, a Sales rep,
  2. Ask each one to name the biggest (in terms of volume) source of data they would not be able to live without on a Post-IT. Even better, make the preferred origin an unstructured one.
  3. Stick these on a white-board table with the following columns:
    1. name, source, size, data type, transmission mechanism, storage point
  4. Ask each one to name what that parameter could be used for if anything could be done with it. Inter-connected (aka chewed) with others into a KPI,printed on a banner at the entrance, fed to an unstructured data analyser: anything.
  5. See if (and let me know if) anything interesting comes out.


Flight Operations Challenge…Accepted.