The Moving Target that is Enterprise IT infrastructures

Docker Logo

A flurry of recent Open Source Enterprise announcements, one relating to Docker – allowing Linux containers containing all their needed components to be built, distributed and then run atop Linux based servers. With this came the inference that Virtualisation was likely to get relegated to legacy application loads. Docker appears to have support right across the board – at least for Linux workloads – covering all the major public cloud vendors. I’m still unsure where that leaves the other niche that is Windows apps.

The next announcement was that of Apache Mesos, which is the software originally built by ex-Google Twitter engineers – largely the replicate the Google Borg software used to fire up multi-server workloads across Google’s internal infrastructure. This used to good effect to manage Twitters internal infrastructure and to consign their “Fail Whale” to much rarer appearances. At the same time, Google open sourced a version of their software – I’ve not yet made out if it’s derived from the 10+ year old Borg or more recent Omega projects – to do likewise, albeit at smaller scale than Google achieve inhouse. The one thing that bugs me is that I can never remember it’s name (i’m off trying to find reference to it again – and now I return 15 minutes later!).

“Google announced Kubernetes, a lean yet powerful open-source container manager that deploys containers into a fleet of machines, provides health management and replication capabilities, and makes it easy for containers to connect to one another and the outside world. (For the curious, Kubernetes (koo-ber-nay’-tace) is Greek for “helmsman” of a ship)”.

That took some finding. Koo-ber-nay-tace. No exactly memorable.

However, it looks like it’ll be a while before these packaging, deployment and associated management technologies get ingrained in Enterprise IT workloads. A lot of legacy systems out there are simply not architected to run on scale-out infrastructures yet, and it’s a source of wonder what the major Enterprise software vendors are running in their own labs. If indeed they have an appetite to disrupt themselves before others attempt to.

I still cringe with how one ERP system I used to use had the cost collection mechanisms running as a background batch process, and the margins of the running business went all over the place like a skidding car as orders were loaded. Particularly at end of quarter customer spend spikes, where the complexity of relational table joins had a replicated mirror copy of the transaction system consistently running 20-25 minutes behind the live system. I should probably cringe even more given there’s no obvious attempt by startups to fundamentally redesign an ERP system from the ground up using modern techniques. At least yet.

Startups appear to be much more heavily focussed on much lighter mobile based applications – of which there are a million different bets chasing VC money. Moving Enterprise IT workloads into much more cost effective (but loosely coupled) public cloud based infrastructure – and that take full advantage of its economics – is likely to take a little longer. I sometimes agonise over what change(s) would precipitate that transition – and whether that’s a monolith app, or a network of simple ones daisy chained together.

I think we need a 2014 networked version of Silicon Office or Hypercard to trigger some progress. Certainly their abject simplicity is no more, and we’re consigned to the lower level, piecemeal building bricks – like JavaScript – which is what life was like in assembler before high level languages liberated us. Some way to go.

The Internet of Things withers – while HealthKit ratchets along

FDA Approved Logo

I sometimes shudder at the estimates, as once outlined by executives at Cisco, that reckons the market for “Internet of Things” – communicating sensors embedded everywhere – would be likely be a $19 trillion market. A market is normally people willing to invest to make money, save money, to improve convenience or reduce waste. Or a mix. I then look at various analysts reports where they size both the future – and the current market size. I really can’t work out how they arrive at today’s estimated monetary amounts, let alone do the leap of faith into the future stellar revenue numbers. Just like IBM with their alleged ‘Cloud’ volumes, it’s difficult to make out what current products are stuffed inside the current alleged volumes.

One of my sons friends is a Sales Director for a distributor of sensors. There appear good use cases in Utility networks, such as monitoring water or gas flow and to estimate where leaks are appearing, and their loss dimensions. This is apparently already well served. As are industrial applications, based on pneumatics, fluid flow and hook ups to SCADA equipment. A bit of RFID so stock movements can be automatically checked through their distribution process. Outside of these, there are the 3 usual consumer areas; that of cars, health and home equipment control – the very three areas that both Apple and Google appear to be focussed on.

To which you can probably add Low Power Bluetooth Beacons, which will allow a phone handset to know it’s precise location, even where GPS co-ordinates are not available (inside shopping centres as an example). If you’re in an open field with sight of the horizon around you in all directions, circa 14 GPS satellites should be “visible”; if your handset sees two of them, it can suss your x and y co-ordinates to a meter or so. If it sees 3 satellites, that’s normally enough to calculate your x, y and z co-ordinates – ie: geographic location and height above sea level. If it can only see 1 or none, it needs another clue. Hence a super secret rollout where vendors are offering these LEB beacons and can trade the translation from their individual identifiers to their exact location.

In Apple’s case, Apple Passbook Loyalty Cards and Boarding Passes are already getting triggered with an icon on the iOS 8 home screen when you’re adjacent to a Starbucks outlet or Virgin Atlantic Check-in desk; one icon press, and your payment card or boarding pass is there for you already. I dare say the same functionality is appearing in Google Now on Android; it can already suss when I get out of my car and start to walk, and keeps a note of my parking location – so I can ask it to navigate me back precisely. It’s also started to tell me what web sites people look at when they are in the same restaurant that i’m sitting in (normally the web site or menu of the restaurant itself).

We’re in a lull between Apple’s Worldwide Developer Conference, and next weeks equivalent Google I/O developer event, where Googles version of Health and HomeKit may well appear. Maybe further developments to link your cars Engine Control Unit to the Internet as well (currently better engaged by Phil Windley’s FUSE project). Apple appear to have done a stick and twist on connecting an iPhone to a cars audio system only, where the cars electronics use Blackberry’s QNX embedded Linux software; Android implementations from Google are more ambitious but (given long car model cycle times) likely to take longer to hit volume deployments. Unless we get an unexpected announcement at Google I/O next week.

My one surprise is that my previous blog post on Apples HomeKit got an order of magnitude more readers than my two posts on the Health app and the HealthKit API (posts here and here). I’d never expected that using your iPhone as a universal, voice controlled home lock/light/door remote would be so interesting to people. I also hear that Nest (now a Google subsidiary) are about to formally announce shipment of their 500,000th room temperature control. Not sure about their Smoke Alarm volumes to date though.

That apart, I noticed today that the US Food and Drug Administration had, in March, issued some clarifications on what type of mobile connected devices would not warrant regulatory classification as a medical device in the USA. They were:

  1. Mobile apps for providers that help track or manage patient immunizations by assessing the need for immunization, consent form, and immunization lot number

  2. Mobile apps that provide drug-drug interactions and relevant safety information (side effects, drug interactions, active ingredient) as a report based on demographic data (age, gender), clinical information (current diagnosis), and current medications

  3. Mobile apps that enable, during an encounter, a health care provider to access their patient’s personal health record (health information) that is either hosted on a web-based or other platform

So, it looks like Apple Health application and their HealthKit API have already skipped past the need for regulatory approvals there already. The only thing i’ve not managed to suss is how they measure blood pressure and glucose levels on a wearable device without being invasive. I’ve seen someone mention that a hi res camera is normally sufficient to detect pulse rates by seeing image changes on a picture of a patients wrist. I’ve also seen an inference that suitably equipped glasses can suss basic blood composition looking at what is exposed visibly in the iris of an eye. But if Apple’s iWatch – as commonly rumoured – can detect Glucose levels for Diabetes patients, i’m still agonising how they’d do it. Short of eating or attaching another (probably disposable) Low Energy Bluetooth sensor for the phone handset to collect data from.

That looks like it’ll be Q4 before we’ll all know the story. All I know right now is that Apple produce an iWatch, and indeed return the iPhone design to being more rounded like the 3S was, that my wife will expect me to be in the queue on release date to buy them both for her.

Start with the needs of the end user, and work back from there…

Great Customer Service

A bit of a random day. I learnt something about the scale of construction taking place in China; not just the factoid that they’re building 70 airports at the moment, but a much more stunning one. That, in the last 3 years, the Chinese have used more cement than the USA did in the 100 years between 1900 and 2000. The very time when all the Interstate and Road networks were built, in addition to construction in virtually every major city.

5 of the top 10 mobile phone vendors are Chinese (it’s not just an Apple vs Samsung battle now), and one appears to be breaking from the pack in emerging markets – Xiaomi (pronounced show – as in shower – and me). Their business model is to offer Apple-class high end phones at around cost, target them at 18-30 year “fans” in direct sales (normally flash sales after a several 100,000 unit production run), and to make money from ROM customisations and add-on cloud services. I’ve started hearing discussions with Silicon Valley based market watchers who are starting to cite Xiaomi’s presence in their analyses, not least as in China, they are taking market share from Samsung – the first alternative Android vendor to consistently do so. I know their handsets, and their new tablet, do look very nice and very cost effective.

That apart, I have tonight read a fantastic blog post from Neelie Kroes, Vice President of the European Commission and responsible for the Digital Agenda for Europe – talking specifically about Uber and this weeks strikes by Taxi drivers in major cities across Europe. Well worth a read in full here.

Summarised:

  • Let me respond to the news of widespread strikes and numerous attempts to limit or ban taxi app services across Europe. The debate about taxi apps is really a debate about the wider sharing economy.
  • It is right that we feel sympathy for people who face big changes in their lives.
  • Whether it is about cabs, accommodation, music, flights, the news or whatever.  The fact is that digital technology is changing many aspects of our lives. We cannot address these challenges by ignoring them, by going on strike, or by trying to ban these innovations out of existence
  • a strike won’t work: rather than “downing tools” what we need is a real dialogue
  • We also need services that are designed around consumers.
  • People in the sharing economy like drivers, accommodation hosts, equipment owners and artisans – these people all need to pay their taxes and play by the rules.  And it’s the job of national and local authorities to make sure that happens.
  • But the rest of us cannot hide in a cave. 
  • Taxis can take advantage of these new innovations in ways consumers like – they can arrive more quickly, they could serve big events better, there could be more of them, their working hours could be more flexible and suited to driver needs – and apps can help achieve that.
  • More generally, the job of the law is not to lie to you and tell you that everything will always be comfortable or that tomorrow will be the same as today.  It won’t. Not only that, it will be worse for you and your children if we pretend we don’t have to change. If we don’t think together about how to benefit from these changes and these new technologies, we will all suffer.
  • If I have learnt anything from the recent European elections it is that we get nowhere in Europe by running away from hard truths. It’s time to face facts:  digital innovations like taxi apps are here to stay. We need to work with them not against them.

It is absolutely refreshing to have elected representatives working for us all and who “get it”. Focus on consumers, being respectful of those afflicted by changes, but driving for the collective common good that Digital innovations provide to society. Kudos to Neelie Kroes; a focus on users, not entrenched producers – a stance i’ve only really heard with absolute clarity before from Jeff Bezos, CEO of Amazon. It does really work.

 

What if Quality Journalism isn’t?

Read all about it

Carrying on with the same theme as yesterdays post – the fact that content is becoming disaggregated from a web sites home page – I read an excellent blog post today: What if Quality Journalism isn’t? In this, the author looks at the seemingly divergent claims from the New York Times, who claim:

  • They are “winning” at Journalism
  • Readership is falling, both on web and mobile platforms
  • therefore they need to pursue strategies to grow their audience

The author asks “If its product is ‘the world’s best journalism‘, why does it have a problem growing its audience?”. You can’t be the world’s best and fail at the same time. Indeed. And then goes into a deeper analysis.

I like the analogue of the supermarket of intent (Amazon) versus a supermarket of interest (social) versus Niche. The central issue is how to curate articles of interest to a specific subscriber, without filling their delivery with superfluous (to the reader) content. This where Newspapers (in the authors case) typically contain 70% or more of wasted content to a typical specific user.

One comment under the article suggests one approach: existence of an open source aggregation model for the municipal bond market on Twitter via #muniland… journos from 20+ pubs, think tanks, govts, law firms, market commentators hash their story and all share.

Deep linking to useful, pertinent and interesting content is probably a big potential area if alternative approaches can crack it. Until then, i’m having to rely on RSS feeds of known authors I respect, or from common watering holes, or from the occasional flash of brilliance that crosses my twitter stream at times i’m watching it.

Just need to update Aaron Swartz’s code to spot water-cooler conversations on Twitter among specific people or sources I respect. That would probably do most of the leg work to enlighten me more productively, and without subjecting myself to pages of search engine discovery.

Death of the Web Home Page. What replaces it??

Go Back You Are Going Wrong Way Sign

One of the gold nuggets on the “This week in Google” podcast this week was that some US News sites historically had 20% of their web traffic coming in through their front door home page. 80% of their traffic arrived from links elsewhere that landed on individual articles deep inside their site. More recently, that has dropped to 10%.

If they’re anything like my site, only a small proportion of these “deep links” will come from search engine traffic (for me, search sources account for around 20% of traffic most days). Of those that do, many arrive searching for something more basic than what I have for them here. By far my most popular “accident” is my post about “Google: where did I park my car?”. This is a feature of Google Now on my Nexus 5 handset, but I guess many folks are just tapping that query into Google’s search box absolutely raw (and raw Google will be clueless – you need a handset reporting your GPS location and the fact it sensed your transition from driving to walking for this to work). My second common one is people trying to see if Tesco sell the Google Chromecast, which invariably lands on me giving a demo of Chromecast working with a Tesco Hudl tablet.

My major boosts in traffic come when someone famous spots a suitably tagged Twitter or LinkedIn article that appears topical. My biggest surge ever was when Geoffrey Moore, author of “Crossing the Chasm”, mentioned my one page PDF that summarised his whole book on LinkedIn. The second largest when my post that congratulated Apple for the security depth in their CloudKit API, as a fresh change to the sort of shenanigans that several UK public sector data releases violate, appeared on the O’Reilly Radar blog. Outside of those two, I bump along at between 50-200 reads per day, driven primarily by my (in)ability to tag posts on social networks well enough to get flashes of attention.

10% coming through home pages though; that haunts me a bit. Is that indicative of a sea change to single, simple task completion by a mobile app? Or that content is being littered around in small, single article chunks, much like the music industry is seeing a transition from Album Compilations to Singles? I guess one example is this weeks purchase of Songza by Google – and indeed Beats by Apple – giving both companies access to curated playlists. Medium is one literary equivalent, as is Longreads. However, I can’t imagine their existence explains the delta between searches and targeted landing directly into your web site.

So, if a home page is no longer a valid thing to have, what takes it’s place? Ideas or answers on a postcard (or comment here) please!

Explaining Distributed Data Consistency to IT novices? Well, …

Greek Shepherd

it’s all greek to me. Bruce Stidston cited a post on Google+ where Yonatan Zunger, Chief Architect of Google+, tried to explain Data Consistency by way of Greeks enacting laws onto statute books on disparate islands. Very long post here. It highlights the challenges of maintaining data consistency when pieces of your data are distributed over many locations, and the logistics of trying to keep them all in sync – in a way that should be understandable to the lay – albeit patient – reader.

The treatise missed out the concept of two-phased commit, which is a way of doing handshakes between two (identical copies) of a database to ensure a transaction gets played successfully on both the master and the replica sited elsewhere on a network. So, if you get some sort of failure mid transaction, both sides get returned to a consistent state without anything going down the cracks. Important if that data is monetary balance transfers between bank accounts for example.

The thing that impressed me most – and which i’d largely taken for granted – is how MongoDB (the most popular Open Source NoSQL Database in the world) can handle virtually all the use cases cited in the article out of the box, with no add-ons. You can specify “happy go lucky”, majority or all replicas consistent before confirming write completion. And if a definitive “Tyrant” fails, there’s an automatic vote among the surviving instances for which secondary copy becomes the new primary (and on rejoining, the changes are journaled back to consistency). And those instances can be distributed in different locations on the internet.

Bruce contended that Google may not like it’s blocking mechanics (which will slow down access while data is written) to retain consistency on it’s own search database. However, I think Google will be very read heavy, and it won’t usually be a disaster if changes are journaled onto new Google search results to its readers. No money to go between the cracks in their case, any changes just appear the next time you enact the same search; one very big moving target.

Ensuring money doesn’t go down the cracks is what Blockchains design out (majority votes, then change declines to update attempts after that’s achieved). That’s why it can take up to 10 minutes for a Bitcoin transaction to get verified. I wrote introductory pieces about Bitcoin and potential Blockchain applications some time back if those are of interest.

So, i’m sure there must be a more pithy summary someone could draw, but it would add blockchains to the discussion, and probably relate some of the artistry behind hashes and Git/Github to manage large, multiuser, multiple location code, data and writing projects. However, that’s for the IT guys. They should know this stuff, and know what to apply in any given business context.

Footnote: I’ve related MongoDB as that is the one NoSQL database I have accreditations in, having completed two excellent online courses with them (while i’m typically a senior manager, I like to dip into new technologies to understand their capabilities – and to act as a bullshit repellent!). Details of said courses here. The same functionality may well be available with other NoSQL databases.

Uber in London: The Streisand Effect keeps on giving

Uber Logo

With the same overall theme as yesterday, if you’re looking at your future, step one is to look at what your customers would value, then to work back to the service components to deliver it.

I’ve followed Uber since I first discovered them in San Francisco, and it looks a simple model – to the user. You want to go from where you are to another local destination. You typically see where the closest driver is to you on your smartphone. You ask your handset for a price to go to a specific destination. It tells you. If you accept, the car is ordered and comes to pick you up. When you get dropped off, your credit card is charged, and both you and the taxi driver get the opportunity to rate each other. Job done.

Behind that facade is a model of supply and demand. Taxi drivers that can clock on and off at will. At times of high demand and dwindling available ride capacity, prices are ramped up (to “surge” pricing) to encourage more drivers onto the road. Drivers and customers with voluminous bad ratings removed. Drivers paid well enough to make more money than those in most taxi firms ($80-90,000/year in New York), or the freedom to work part time – even down to a level where your reward is to pay for your car for a few hours per week of work, and have free use of it at other times.

The service is simple and compelling enough that i’d have thought tax firms would have cottoned onto how the service works, and to replicate it before Uber ever appeared on these shores. But, with a wasted five years, they’ve appeared – and Taxi drivers all over Europe decided to run the most effective advertising campaign for an upstart competitor in their history. A one-day 850% subscriber growth; that really takes some doing, even if you were on the same side.

I’m just surprised that whoever called the go-slows all over Europe didn’t take the time out to study what we in the tech industry know as “The Streisand Effect” – Wikipedia reference here. BBC Radio 2 even ran a segment on Uber at lunchtime today, followed by every TV News Bulletin i’ve heard since. I downloaded the app as a result of hearing it on that lunchtime slot, as I guess many others did too (albeit no coverage in my area 50 miles West of London – yet). Given the five years of missed prep time, I think they’ve now lost – or find themselves in fast follower mode to incorporate similar technology into their service before they have a mass exodus to Uber (of customers, then drivers).

London Cabbies do know all the practical use of rat runs that SatNav systems are still learning, but even that is a matter of time now. I suspect appealing for regulation will, at best, only delay the inevitable.

The safest option – given users love the simplicity and lack of surprises in the service – is to get busy quickly. Plenty of mobile phone app prototyping help available on the very patch that London Black Cab drivers serve.

Starting with the end in mind: IT Management Heat vs Light

A very good place to startOne source of constant bemusement to me is the habit of intelligent people to pee in the industry market research bathwater, and then to pay handsomely to drink a hybrid mix of the result collected across their peers.

Perhaps betrayed by an early experience of one research company coming in to present to the management of the vendor I was working at, and finding in the rehearsal their conjecture that sales of specific machine sizes had badly dipped in the preceding quarter. Except they hadn’t; we’d had the biggest growth in sales of the highlighted machines in our history in that timeframe. When I mentioned my concern, the appropriate slides were corrected in short order, and no doubt the receiving audience impressed with the skill in their analysis that built a forecast starting with an amazingly accurate, perceptive (and otherwise publicly unreported) recent history.

I’ve been doubly nervous ever since – always relating back to the old “Deep Throat” hints given in “All the Presidents Men” – that of, in every case, “to follow the money”.

Earlier today, I was having some banter on one of the boards of “The Motley Fool” which referenced the ways certain institutions were imposing measures on staff – well away from a useful business use that positively supported better results for their customers. Well, except of providing sound bites to politicians. I can sense that in Education, in some elements of Health provision, and rather fundamentally in the Police service. I’ve even done a drains-up some time ago that reflected on the way UK Police are measured, and tried trace the rationale back to source – which was a senior politician imploring them to reduce crime; blog post here. The subtlety of this was rather lost; the only control placed in their hands was that of compiling the associated statistics, and to make their behaviours on the ground align supporting that data collection, rather than going back to core principles of why they were there, and what their customers wanted of them.

Jeff Bezos (CEO of Amazon) has the right idea; everything they do aligns with the ultimate end customer, and everything else works back from there. Competition is something to be conscious of, but only to the extent of understanding how you can serve your own customers better. Something that’s also the central model that W. Edwards Deming used to help transform Japanese Industry, and in being disciplined to methodically improve “the system” without unnecessary distractions. Distractions which are extremely apparent to anyone who’s been subjected to his “Red Beads” experiment. But the central task is always “To start with the end in mind”.

With that, I saw a post by Simon Wardley today where Gartner released the results of a survey on “Top 10 Challenges for I&O Leaders”, which I guess is some analogue of what used to be referred to as “CIOs”. Most of which felt to me like a herd mentality – and divorced from the sort of issues i’d have expected to be present. In fact a complete reenactment of this sort of dialogue Simon had mentioned before.

Simon then cited the first 5 things he thought they should be focussed on (around Corrective Action), leaving the remainder “Positive Action” points to be mapped based on that appeared upon that foundation. This in the assumption that those actions would likely be unique to each organisation performing the initial framing exercise.

Simon’s excellent blog post is: My list vs Gartner, shortly followed by On Capabilities. I think it’s a great read. My only regret is that, while I understand his model (I think!), i’ve not had to work on the final piece between his final strategic map (for any business i’m active in) and articulating a pithy & prioritised list of actions based on the diagram created. And I wish he’d get the bandwidth to turn his Wardley Maps into a Book.

Until then, I recommend his Bits & Pieces Blog; it’s a quality read that deserves good prominence on every IT Manager’s (and IT vendors!) RSS feed.

CloudKit – now that’s how to do a secure Database for users

Data Breach Hand Brick Wall Computer

One of the big controversies here relates to the appetite of the current UK government to release personal data with the most basic understanding of what constitutes personal identifiable information. The lessons are there in history, but I fear without knowing the context of the infamous AOL Data Leak, that we are destined to repeat it. With it goes personal information that we typically hold close to our chests, which may otherwise cause personal, social or (in the final analysis) financial prejudice.

When plans were first announced to release NHS records to third parties, and in the absence of what I thought were appropriate controls, I sought (with a heavy heart) to opt out of sharing my medical history with any third party – and instructed my GP accordingly. I’d gladly share everything with satisfactory controls in place (medical research is really important and should be encouraged), but I felt that insufficient care was being exercised. That said, we’re more than happy for my wife’s Genome to be stored in the USA by 23andMe – a company that demonstrably satisfied our privacy concerns.

It therefore came as quite a shock to find that a report, highlighting which third parties had already been granted access to health data with Government mandated approval, ran to a total 459 data releases to 160 organisations (last time I looked, that was 47 pages of PDF). See this and the associated PDFs on that page. Given the level of controls, I felt this was outrageous. Likewise the plans to release HMRC related personal financial data, again with soothing words from ministers in whom, given the NHS data implications, appear to have no empathy for the gross injustices likely to result from their actions.

The simple fact is that what constitutes individual identifiable information needs to be framed not only with what data fields are shared with a third party, but to know the resulting application of that data by the processing party. Not least if there is any suggestion that data is to be combined with other data sources, which could in turn triangulate back to make seemingly “anonymous” records traceable back to a specific individual.Which is precisely what happened in the AOL Data Leak example cited.

With that, and on a somewhat unrelated technical/programmer orientated journey, I set out to learn how Apple had architected it’s new CloudKit API announced this last week. This articulates the way in which applications running on your iPhone handset, iPad or Mac had a trusted way of accessing personal data stored (and synchronised between all of a users Apple devices) “in the Cloud”.

The central identifier that Apple associate with you, as a customer, is your Apple ID – typically an email address. In the Cloud, they give you access to two databases on their cloud infrastructure; one a public one, the other private. However, the second you try to create or access a table in either, the API accepts your iCloud identity and spits back a hash unique to your identity and the application on the iPhone asking to process that data. Different application, different hash. And everyone’s data is in there, so it’s immediately unable to permit any triangulation of disparate data that can trace back to uniquely identify a single user.

Apple take this one stage further, in that any application that asks for any personal identifiable data (like an email address, age, postcode, etc) from any table has to have access to that information specifically approved by the handset owners end user; no explicit permission (on a per application basis), no data.

The data maintained by Apple, besides holding personal information, health data (with HealthKit), details of home automation kit in your house (with HomeKit), and not least your credit card data stored to buy Music, Books and Apps, makes full use of this security model. And they’ve dogfooded it so that third party application providers use exactly the same model, and the same back end infrastructure. Which is also very, very inexpensive (data volumes go into Petabytes before you spend much money).

There are still some nuances I need to work. I’m used to SQL databases and to some NoSQL database structures (i’m MongoDB certified), but it’s not clear, based on looking at the way the database works, which engine is being used behind the scenes. It appears to be a key:value store with some garbage collection mechanics that look like a hybrid file system. It also has the capability to store “subscriptions”, so if specific criteria appear in the data store, specific messages can be dispatched to the users devices over the network automatically. Hence things like new diary appointments in a calendar can be synced across a users iPhone, iPad and Mac transparently, without the need for each to waste battery power polling the large database on the server waiting for events that are likely to arrive infrequently.

The final piece of the puzzle i’ve not worked out yet is, if you have a large database already (say of the calories, carbs, protein, fat and weights of thousands of foods in a nutrition database), how you’d get that loaded into an instance of the public database in Apple’s Cloud. Other that writing custom loading code of course!

That apart, really impressed how Apple have designed the datastore to ensure the security of users personal data, and to ensure an inability to triangulate data between information stored by different applications. And that if any personal identifiable data is requested by an application, that the user of the handset has to specifically authorise it’s disclosure for that application only. And without the app being able to sense if the data is actually present at all ahead of that release permission (so, for example, if a Health App wants to gain access to your blood sampling data, it doesn’t know if that data is even present or not before the permission is given – so the app can’t draw inferences on your probably having diabetes, which would be possible if it could deduce if it knew that you were recording glucose readings at all).

In summary, impressive design and a model that deserves our total respect. The more difficult job will be to get the same mindset in the folks looking to release our most personal data that we shared privately with our public sector servants. They owe us nothing less.

A first look at Apple HomeKit

Apple HomeKit Logo

Today’s video from Apple’s Worldwide Developers Conference viewing concerned HomeKit, which is the integration platform to control household appliances from your iPhone. Apple have defined a common set of Accessory Profiles, which are configured into a Home > Zone > Room hierarchy (you can define several ‘home’ locations, but one of them is normally selected as the primary one). Native devices include:

  • Garage Door Openers (with associated lighting)
  • Lights
  • Door locks
  • Thermostats
  • IP (Internet Protocol) Cameras
  • Switches

Currently, there are a myriad of different per vendor standards to control home automation products, but Apple are providing functionality to enable hardware (or software) bridges between disparate protocols and their own. Once a bridge has been discovered, the iPhone sees all the devices sitting the other side of the bridge as if they were directly connected to the iPhone and using the Apple provided interface protocols.

Every device type has a set of characteristics, such as:

  • Power State
  • Lock State
  • Target State
  • Brightness
  • Model Number
  • Current Temperature
  • etc

When devices are first defined, each has a compulsory “identify me” action. Hence if you’re sitting on the floor, trying to work out which of twelve identical-looking lightbulbs in the room to give an appropriate name, the “identify me” action on the iPhone pick list will result in the matching bulb blinking twice; for a security camera, blinking a colour LED, and so forth.

Each device, it’s room name, zone (like “upstairs”, “back garden”) and home name, plus the common characteristic actions, are encoded and enacted using Siri – Apple’s voice control on the iPhone. “Switch on all downstairs lights”, “Set the room temperature to 20 degrees C” and so forth are spoken into your iPhone handset. That is the default User Interface for the whole Home Automation Setup. The HomeKit resident database is in turn also available for use by vendor specific products via the HomeKit API, should a custom application be desirable.

There are of course extensive security controls to frustrate any attempt for anyone to be able to do “man in the middle” attacks, or to subvert the security of your device connections. For developers, Apple provide a software simulator so that you can test your software against a wide range of device types, even before the hardware is made available to you.

Most of the supporting detail to build compliant devices is found in the MFI (Made for iDevices) Guidelines, which are only available the other side of a license agreement with Apple here. The full WWDC presentation on HomeKit (just under an hour long) is called “Introduction to HomeKit” and present in the list of video sessions from WWDC here.

Overall, very impressive. That’s the home stuff largely queued up, just awaiting news of a bridge I think. Knowing how simple the voice setup is on Android JellyBean for a programmer (voice enabling an app is circa 20 lines of JavaScript), i’m sure a Google equivalent is eminently possible; if Google haven’t done their own API, then a bridge to Apple’s ecosystem (if the licensing allows it) should not be a major endeavour.

So, the only missing thing was talk of iBeacon support. However, that is a different use case. There are already pilots that sense presence of a low energy bluetooth beacon, and bring specific applications onto the lock screen. Examples include the Starbucks payment card app coming forward to make itself immediately available when you’re close to a Starbucks counter, or the Virgin Atlantic app making your boarding card available when you approach the check-in desk at an airport. Both are features of Apple’s PassBook loyalty card app – which is already used by hundreds of retailers, supermarkets and airlines.

The one thing about iBeacon is that you can enable your iPhone 5S to be a low energy beacon in it’s own right. You have full control over this and your presence is not made available to anything but applications on your own iPhone handset – over which, in the final analysis, you have total control. One use case already is pairing your Pebble Smartwatch with your iPhone 5S handset, so that if your phone leaves your immediate location by a specified short distance (say, 2 meters), you’re aggressively told immediately.

So, lots to look forward to in the Autumn. Quite a measured approach compared to the “Internet of Things” which other vendors are hyping with impunity (and quoting staggering revenue numbers which I find difficult to map onto any reality – starting with what folks seem to suggest is even a current huge market size already).

My next piece of homework will be to look at CloudKit, now that Apple are dogfooding it’s use in their own products while releasing it to third party developers. Hopefully, a good sign that Apple are now providing cloud services that match the resilience of competitive offerings for the first time – even if they are specific to Apple’s own platforms. But that’s all the other side of finishing my company’s end of year tax return prep work first!