Officially Certified: AWS Business Professional

AWS Business Professional Certification

That’s added another badge, albeit the primary reason was to understand AWS’s products and services in order to suss how to build volumes via resellers for them – just in case I can get the opportunity to be asked how i’d do it. However, looking over the fence at some of the technical accreditation exams, I appear to know around half of the answers there already – but need to do those properly and take notes before attempting those.

(One of my old party tricks used to be that I could make it past the entrance exam required for entry into technical streams at Linux related conferences – a rare thing for a senior manager running large Software Business Operations or Product Marketing teams. Being an ex programmer who occasionally fiddles under the bonnet on modern development tools is a useful thing – not least to feed an ability to be able to spot bullshit from quite a distance).

The only AWS module I had any difficulty with was the pricing. One of the things most managers value is simplicity and predictability, but a lot of the pricing of core services have pricing dependencies where you need to know data sizes, I/O rates or the way your demand goes through peaks and troughs in order to arrive at an approximate monthly price. While most of the case studies amply demonstrate that you do make significant savings compared to running workloads on your own in-house infrastructure, I guess typical values for common use cases may be useful. For example, if i’m running a SAP installation of specific data and access dimensions, what operationally are typically running costs – without needing to insert probes all over a running example to estimate it using the provided calculator?

I’d come back from a 7am gym session fairly tired and made the mistake of stepping through the pricing slides without making copious notes. I duly did all that module again and did things properly the next time around – and passed it to complete my certification.

The lego bricks you snap together to design an application infrastructure are simple in principle, loosely connected and what Amazon have built is very impressive. The only thing not provided out of the box is the sort of simple developer bundle of an EC2 instance, some S3 and MySQL based EBD, plus some open source AMIs preconfigured to run WordPress, Joomla, Node.js, LAMP or similar – with a simple weekly automatic backup. That’s what Digital Ocean provide for a virtual machine instance, with specific storage and high Internet Transfer Out limits for a fixed price/month. In the case of the WordPress network on which my customers and this blog runs, that’s a 2-CPU server instance, 40GB of disk space and 4TB/month data traffic for $20/month all in. That sort of simplicity is why many startup developers have done an exit stage left from Rackspace and their ilk, and moved to Digital Ocean in their thousands; it’s predictable and good enough as an experimental sandpit.

The ceiling at AWS is much higher when the application slips into production – which is probably reason enough to put the development work there in the first place.

I have deployed an Amazon Workspace to complete my 12 years of Nutrition Data Analytics work using the Windows-only Tableau Desktop Professional – in an environment where I have no Windows PCs available to me. Just used it on my MacBook Air and on my iPad Mini to good effect. That will cost be just north of £21 ($35) for the month.

I think there’s a lot that can be done to accelerate adoption rates of AWS services in Enterprise IT shops, both in terms of direct engagement and with channels to market properly engaged. My real challenge is getting air time with anyone to show them how – and in the interim, getting some examples ready in case I can make it in to do so.

That said, I recommend the AWS training to anyone. There is some training made available the other side of applying to be a member of the Amazon Partner Network, but there are equally some great technical courses that anyone can take online. See http://aws.amazon.com/training/ for further details.

Help available to keep malicious users away from your good work

Picture of a Stack of Tins of Spam Meat

One thing that still routinely shocks me is the shear quantity of malicious activity that goes on behind the scenes of any web site i’ve put up. When we were building Internet Vulnerability Testing Services at BT, around 7 new exploits or attack vectors were emerging every 24 hours. Fortunately, for those of us who use Open Source software, the protections have usually been inherent in the good design of the code, and most (OpenSSL heartbleed excepted) have had no real impact with good planning. All starting with closing off ports, and restricting access to some key ones from only known fixed IP addresses (that’s the first thing I did when I first provisioned our servers in Digital Ocean Amsterdam – just surprised they don’t give a template for you to work from – fortunately I keep my own default rules to apply immediately).

With WordPress, it’s required an investment in a number of plugins to stem the tide. Basic ones like Comment Control, that  can lock down pages, posts, images and attachments from having comments added to them (by default, spammers paradise). Where you do allow comments, you install the WordPress provided Akismet, which at least classifies 99% of the SPAM attempts and sticks them in the spam folder straight away. For me, I choose to moderate any comment from someone i’ve not approved content from before, and am totally ruthless with any attempt at social engineering; the latter because if they post something successfully with approval a couple of times, their later comment spam with unwanted links get onto the web site immediately until I later notice and take them down. I prefer to never let them get to that stage in the first place.

I’ve been setting up a web site in our network for my daughter in law to allow her to blog abound Mental Health issues for Children, including ADHD, Aspergers and related afflictions. For that, I installed BuddyPress to give her user community a discussion forum, and went to bed knowing I hadn’t even put her domain name up – it was just another set of deep links into my WordPress network at the time.

By the morning, 4 user registrations, 3 of them with spoof addresses. Duly removed, and the ability to register usernames then turned off completely while I fix things. I’m going into install WP-FB-Connect to allow Facebook users to work on the site based on their Facebook login credentials, and to install WangGuard to stop the “Splogger” bots. That is free for us for the volume of usage we expect (and the commercial dimensions of the site – namely non-profit and charitable), and appears to do a great job  sharing data on who and where these attempts come from. Just got to check that turning these on doesn’t throw up a request to login if users touch any of the other sites in the WordPress network we run on our servers, whose user communities don’t need to logon at any time, at all.

Unfortunately, progress was rather slowed down over the weekend by a reviewer from Kenya who published a list of best 10 add-ins to BuddyPress, #1 of which was a Social Network login product that could authenticate with Facebook or Twitter. Lots of “Great Article, thanks” replies. In reality, it didn’t work with BuddyPress at all! Duly posted back to warn others, if indeed he lets that news of his incompetence in that instance back to his readers.

As it is, a lot of WordPress Plugins (there are circa 157 of them to do social site authentication alone) are of variable quality. I tend to judge them by the number of support requests received that have been resolved quickly in the previous few weeks – one nice feature of the plugin listings provided. I also have formal support contracts in with Cyberchimps (for some of their themes) and with WPMU Dev (for some of their excellent Multisite add-ons).

That aside, we now have the network running with all the right tools and things seem to be working reliably. I’ve just added all the page hooks for Google Analytics and Bing Web Tools to feed from, and all is okay at this stage. The only thing i’d like to invest in is something to watch all the various log files on the server and to give me notifications if anything awry is happening (like MySQL claiming an inability to connect to the WordPress database, or Apache spawning multiple instances and running out of memory – something I had in the early days when the Google bot was touching specific web pages, since fixed).

Just a shame that there are still so many malicious link spammers out there; they waste 30 minutes of my day every day just clearing their useless gunk out. But thank god that Google are now penalising these very effectively; long may that continue, and hopefully the realisation of the error of their ways will lead to being a more useful member of the worldwide community going forward.

a16z brilliance vs the Leaking Bucket

Digital Ocean Logo

When I worked for DEC, I used to have a brass plaque on the wall in front of me that reminded us in the Software Services Division of our three priorities. It said, in order of importance:

  1. Warranty Customers
  2. Presales
  3. Consultancy

Paraphrased, this says: look after your customers before you go anywhere near trying to get new ones. Next, support the rest of the company selling the whole toolbox to solve customer needs (software was typically only 10% of a project sale). Finally, if we’d done those first, only then did we try to make profit for our own unit alone.

The other euphemism we knew was an old American Football one, which describes a now illegal play; that of “smacking the helmet”. That’s the crash helmet of a 30 stone athlete running at you, as anywhere his head ends up heading, the full weight of the body will follow. So, a well aimed deflection early in a move causes a disportionate effect when the rest of what’s behind it follows. And in the IT market, that front end constituency is the software development community – aka “Developers”. Hit that effectively, and you’re in great shape.

In theory, it’s a great time to be a software developer. Hardware, Storage and Network capacity is fairly inexpensive. Tools to build everything from Mobile to Enterprise applications are predominantly open source and available to all. So, a lot of the early decision making for where to site your applications is where you find a cost effective on-ramp – and more often than not, you’ll stick to where you first deploy as your business scales.

When you are a developer, you get to hear about Amazon Web Services (AWS) and their fantastic growth. This a result of their CEO Jeff Bezos telling his staff that they would deploy all their businesses as APIs, and allow other companies to use their spare compute/storage capacity. Spikes in demand necessitate massive over “just in case” provisioning, even though those spikes are few every year and very seasonal. That said, the amount of options on there is now wide and complex, and hence a learning curve before you can price your development hosting cost out. Examples here, but for my needs, it would be circa £80/month.

You also get to hear about Google Compute Engine, which open up Google’s capacity to developers who can write to their own specific APIs; that said, they appear to favour apps that can take advantage of their own unique database and auto scaling features. If you want a price, then there is a web site where you can enter a number of parameters, and it will articulate a dollar cost – which in my case, were not inexpensive. Or you can have a crack at this.

Likewise for Rackspace, who do a lot of work among the startup community, but again who have pricing suited to paying for their excellent support services. Most of which developers don’t actually need while starting to build their systems. Examples here.

Early in my own work, I saw a report from Mike Prettejohn‘s company (Netcraft) about a small New York company called Digital Ocean who were growing like topsy. From 137 machines in Dec 2012 to (at the time of writing this) 54,142 in Feb 2014:

Digital Ocean Server Growth

The main appeal to me (like a lot of developers) is that you can provision a server instance with one of a range of prebuilt Linux configs within 5 minutes. And once it’s up, it’s $10/month for a virtual server instance with 30GB of Flash Storage and 3TB of Network Bandwidth per month. Add a further $2/month to get weekly backups and the ability to take as many snapshots of your system(s) as you feel comfortable. Very simple, predictable and does the job. The words you’re reading here are being served off a Ubuntu Linux Server in Digital Ocean Amsterdam, using a WordPress network I built using one of their available images. DIY, not for everyone, but if you know what you’re doing and you can secure your site, it’s about as cost effective as you can get.

Besides seeing the volume of site growth, I look at the Netcraft Hosting Provider Switching Analysis, which gives an indication of how each provider was both growing or churning its customer base – and if there was churn, where it was going. The thing that struck me were the number of sites that were relocating from AWS and in particular Rackspace over to Digital Ocean. At a time when Rackspace have been giving profit warnings, the numbers were over 4 figures of customer sites per month – some 31,279 sites in a year.

Mention Digital Ocean to Rackspace staff (I know two of them), and the best positioning I have from them is that they have many competitors that keep them up at night. That said, I shake my head and wonder if they’re spending all their time looking at new customer acquisition (bath taps at full bore) while leaving that very large plug out of their bath.

With that, Andreessen Horowitz yesterday put some major VC funding into Digital Ocean. Given they are gobbling market share – and that my gut says it’s heavily developer focussed – I think they are a fantastic bet. I wonder when AWS, Azure and Google will have a comparable offer, and until then, i’m sure progress will continue on the current relentless path. Digital Ocean have been a revelation to me so far.

Well done, a16z. You’ve picked a great team. Again.

Enterprise IT meets the Hall of Marbles

hallofmarbles

Every time I hear a vendor utter words such as “private cloud” or “hybrid cloud”, I see visions of brakes being applied. Usually by vendors or IT departments, who sell (or buy) the concept that they can evolve gradually into the future as prices plummet.

I had my first taste of a likely future state a few years back. The London Organising Committee of the Olympic Games needed to provision 500,000 email accounts for a 2 year period, lasting 12 months before and 12 months after the 2012 games. I recall that several prospective suppliers configured branded hardware, software and management using their own hosting plus services, and came out in pricing around £7-10/user/year. Using their existing scale, Google came in at 14p/user/year.

While a prestigious win, I could find no-one that thought Google were selling at a loss. So, what happens when those sort of scale advantages hit Enterprise IT? Do brand vendors not see the tidal wave of low cost, immediately available computing and storage coming their way, a tidal wave due to hit this side of 2017? I think Simon Wardley describes the reality very eloquently in one of his blog posts, relating the story of “The Hall of Marbles”:

It’s the exponential growth part that catches most past suppliers out and that’s due to this expectation of gradual change due to the previous competitive stage (i.e. product vs product). To explain this, I’ll use an analogy from a good friend of mine, Tony Fish.

Consider a big hall that can contain a million marbles. If we start with one marble and double the number of marbles each second, then the entire hall will be filled in 20 seconds. At 19 seconds, the hall will be half full. At 15 seconds only 3% of the hall, a small corner will be full. Despite 15 seconds having passed, only a small corner of the hall is full and we could be forgiven for thinking we have plenty more time to go, certainly vastly more than the fifteen seconds it has taken to fill the small corner. We haven’t. We’ve got five seconds.

Hence for a hardware manufacturer who has sold computer products and experienced gradual change for thirty years, it is understandable how they might consider this change to utility services will also happen slowly. They will have huge inertia to the change because of past success, they may view it as just an economic blip due to a recession and their customers will often try to reinforce the past asking for more “enterprise” like services. Worst of all, they will believe they have time to transition, to help customers gradually change, to spend the years building and planning new services and to migrate the organization over to the new models.

Alas, Amazon alone is estimated at $2bn in cloud revenue for 2012 and predicted for almost $4bn in 2013. If that growth rate continues then by 2016 they will be in excess of $30 billion in revenue. They also have rapidly growing competitors such as Google.

The cold hard reality that many existing suppliers probably don’t comprehend is that the battle will be over in three to four years and for many the time to act has already passed. Like the rapid change in climate temperature in Greenland, our past experience of change does not necessarily represent the future.

In industry, we have a long history of such rapid cycles of change and inertia is key to this. These cycles we call “revolutions” as in industrial, mechanical and the revolution of electricity. During these times, change is rapid not gradual and disruption is widespread.

I’m typing this post into a WordPress network site I provisioned on a Linux instance in Digital Ocean Amsterdam. It is costing me $10/month for my server instance, 30GB storage and 3TB/month network bandwidth, plus another $2/month for regular backups and and ability to store as many snapshots i’m comfortable with. Digital Ocean, like AWS, are gobbling up capacity at a rate of several thousand new server instances every month, and i’m sure Google and Microsoft Azure aren’t standing still either. Just a relentless tide of predominantly Linux servers that are simple to provision and build applications upon.

We are in a time when mobile, the cloud, a deluge of data and collaboration tools – based on open source software – are brewing a perfect storm. Google, Amazon, MongoDB and GitHub will, I’m sure, be there when the dust settles. Who else?

Footnote:

Simon delves deeper into the subject of vendor inertia in his excellent “Bits and Pieces” blog at http://blog.gardeviance.org/2013/01/intertia.html