Seeing the big picture of Technology change – The virtuous circle

The Virtuous Circle

ACCELERATING PACE OF CHANGE

BY JOHN MATHON ON AUGUST 13, 2014

My previous blog talked about the specifics of how some change is being socialized and spread using APIs and open source but the virtuous circle is more than just open source and APIs.  Open source and APIs are how programmers can bring the underlying technology change to developers but the entire circle is the reason this is all happening.

The idea of the virtuous circle is that each component of the circle has been critical to the success of the other parts of the circle.  It is hard to imagine the success of any of these elements in isolation.  The massive adoption of key aspects of the circle is clearly needed but each element of the circle has been key to the success and adoption of the other parts.    The constant improvement in each aspect of the circle drives improvements in other parts of the circle driving more adoption and more innovation.  The process feeds itself and grows faster as each element grows.   It is impossible to say which element is less – critical so they have to be considered as one.

These components of the virtuous circle imply that to leverage the benefits of the circle you have to be using components of the circle in your processes.   The more components you use the more benefit you gain from the circle and the faster the change you will experience.   The combination of these technologies is being called Platform 3.0.

 

The benefits of each part of the virtuous circle

  1. APIs

APIs are the honeybee of software (apis is latin for honeybee)

RESTful APIs became popular after the Cloud and Mobile became big.  APIs were the basic building blocks of Mobile Applications and most Mobile Applications were hosted in the cloud.

After Apple introduced the App Store over the next 5 years some 600,000 applications were created.  These Apps went through a massive increase in power, capability and they needed services (APIs) in the cloud more and more to support the mobile application growth.

Increasing API services in the Cloud drive new Mobile Apps but also drove new revenues to companies that were able to sell those APIs for billions in new revenue further powering the API revolution.   As Enterprises and developers became more and more enamored with the simple RESTful API paradigm it begins to replace in Enterprises the SOAP / SOA mantra.

Increasing success of APIs fosters a parallel development to the App Stores with the proliferation of social API Stores which let you find APIs, subscribe to APIs and publish APIs to communities of developers.   The number of APIs has ballooned to 10s of thousands, many earning billions for their creators.  The success of APIs is changing the role of the CIO into having a role in the top line of a company not just the costs of a company making them more important player.

The growth and success of APIs fosters Enterprises to refactor their Enterprises using RESTful APIs, implement an API Store, deploy API Managers for internal development as well as to publish for external consumption leading to vast changes in Enterprise Architecture.   The selling of services as APIs requires a new way to manage APIs and applications built on the APIs whether mobile or conventional for internal or external consumption.  The success of APIs inside organizations forces vendors to support RESTful APIs in Enterprise Software of all types.    Most open source software is changed to use RESTful APIs as standard ways of interfacing.

  1. Mobile

Mobile took off after the introduction of the Apple iPhone in 2006.  Less than 10 years ago there are now over 1 BILLION smartphones in use and the number of expected to reach ubiquity fairly fast.

Even more amazing the adoption of powerful expensive smartphones which can support powerful mobile apps is keeping pace.  Mobile App usage grows dramatically so that users now spend 84% of their time on average in Apps on their phones not doing calls or other traditional phone things.

The social aspect of the mobile devices themselves lend themselves to proliferation of mobile apps and socialization via the stores and other social apps leads to massive adoption and growth of both.

The growing adoption of Mobile was accelerated by the introduction of the iPad and subsequently other tablet devices.   The mass adoption in Enterprises and consumer space of the mobile interface has led many to believe that within a few years all interaction with computers will be via mobile devices like tablets and smartphones transforming how everyone believes applications will be built and delivered in the future.

Mobile devices support literally hundreds of Apps because of the successful model of the App Store and the integration of Apps in the mobile devices.   This has resulted in changes to Desktop software to become more and more like a mobile device.  The trend is unstoppable that all interactions will eventually be via a mobile like interface and applications managed on any device with App Stores and Social capabilities.

Lest one think this is simply fad and dependent on the smartphone success one has to realize that companies such as Uber have a valuation of $17 billion and all they have is an App.  The Uber app delivers a disruptive capability that empowers many people to make money in new ways and for people to find services faster than ever.

Other examples of transformative power of the combination of mobile, applications and social are ubiquitous.  Companies in the retail sector frequently hold meetings once a week to review their feedback on Yelp to see if they can improve their service.  New ways of sharing documents and pictures transform how we view privacy and Enterprise data is distributed.  Mobile has a major impacts on security strategies of companies.

This unstoppable force of mobile is driving all other aspects of the virtuous circle as well.   For instance, the requirements to deliver, update and improve mobile apps frequently has put pressure on DevOps automation and Cloud scalability.

  1. Social

The growth of social on the desktop started before the Smartphone but really took off in the last 8 years with the smartphone.

Most users now do their social activity on the smartphone in order to interact in the moment wherever they are doing something.   Pictures, texting, various forms of interconnection apps are being created constantly.

Facebook and other pioneers have more than 1 billion users and the use of social has fed a tremendous change in the way Enterprises think of connecting with customers.  They want more and more to learn from social interactions and be part of them so they can influence buying behavior.

Enterprises realize they have to have social capabilities, to be able to capture detailed usage information, detailed interactions and then to mine that information for actionable knowledge like the social companies, Google, other cloud companies have.   Such capability helps them improve their services, make their services more intelligent as well as increasing opportunities to sell to customers.  This requires big-data in most cases.  This new way of storing and analyzing data helps improve applications and services rapidly and is being adopted by Enterprises en masse.

The growth of social applications, big data and the need to scale to billions of users has driven collaboration in open source like never before.  The success of social becomes a key aspect of the success of mobile, the success of APIs and services, Applications so that most companies must deal with this new way of interacting and learning from customers and users.

4.Cloud

Cloud started at a time close to the start of Mobile.  One could look at Cloud as simply the extension of the Internet era but the real start of the Cloud is really about Amazon and the creation of public services for infrastructure it started.

Today, this business is at close to $25 billion and Amazon has a 50% market share.  Amazon’s business is growing at 137% annually and the cloud is becoming an unstoppable force as much as any of these other components in the virtuous circle.

Cloud is the underpinning of most of the other elements of the virtuous circle as the way that companies deliver services.   The way most startups get going is by leveraging the disruptive power of the Cloud.   The Cloud enables a company (big or small) to acquire hardware for offering a service instantly instead of the months required before.  More important for small companies the ability to build and deliver a service in a fraction of the time it used to take and with almost zero capital cost and variable expenses that grow as they grow makes many more companies viable as startups.

The Cloud disruption means most companies no longer need as much venture capital putting more of the benefit in entrepreneurs hands fostering increasing numbers of startups.  The cloud and social also promulgates a new way of funding companies with Kickstarter campaigns able to raise millions for entrepreneurs.   This drives massive innovation and the creation of new devices and applications, services.

Larger Enterprises are realizing that the cloud has benefits for them too.  Many are adopting more and more cloud services.  Numerous SaaS companies started in the internet era are now based on cloud services.   Companies can’t avoid the Cloud adoption as Personal Cloud use explodes and more and more skunk works usage of the cloud happens.

SaaS has grown to over $130 billion industry.  SaaS applications are combined with IaaS and now PaaS (DevOps) is changing the infrastructure and how Enterprises are built.

The transformation of Enterprise infrastructure to Cloud will take decades but is a multi-trillion dollar business eventually.

The economics of the Cloud are unstoppable.  Most Enterprises are simply not in the technology business and have no reason or basis for running, hosting, buying technology infrastructure and basic applications.

Open source projects have driven massive adoption of cloud technology and cloud technology is dependent on the open source technology that underlies much of it.

  1. DevOps

The Cloud by itself allowed you to speed the acquisition of hardware but the management of this hardware was still largely manual, laborious and expensive.  DevOps is the process of taking applications from development into production.  This was a significant cost and time sink for new services, applications and technology.

DevOps automates the acquisition, operation, upgrade, deployment, customer support of services and applications.  Without DevOps automation the ability to upgrade daily, the cost to maintain and reliability of Cloud based services would have faltered.   DevOps started with the growth of the open source projects Puppet and Chef but quickly went beyond that with the growth of PaaS.  PaaS is expected to be a $6-14 billion market in 3 or 4 years.  Heroku demonstrated within several years of its founding that they had 70,000 applications developed, built and deployed in the cloud, demonstrating the power of PaaS to reduce costs and make it easier for small companies to do development.

The ability to deliver applications fast, to develop applications faster and faster, easier and easier is because of the automation and capability of PaaS’s and DevOps to rapidly allow people to dream up applications and implement them, deploy them and scale them almost effortlessly.    This has allowed so many people to offer new mobile applications, new services, for new companies to be formed and succeed faster than ever before.  It has allowed applications to grow to billions of users.

Numerous open source projects provide the basic building blocks of DevOps and PaaS technology which drive the industry forward.   The success of the DevOps / PaaS technology is also changing the way Enterprises build and deploy software for inside or outside consumption.

  1. Open Source

Underlying all these other elements of the virtuous circle has been a force of collaboration allowing companies to share technology and ideas that has enabled the rapid development and proliferation of new technologies.

The number of open source projects is doubling every 13 months according to surveys.   Enterprises now consider Open Source software the best quality software available.  In many cases it is the ONLY way to build and use some technologies.

The open source movement is powering Cloud technology, Big-data, analytics for big-data, social, APIs, Mobile technology with so many projects and useful components it is beyond elaboration.  It is an essential piece of building any application today.

The growth of open source has fostered increasing innovation dramatically.  Initially HBase was one of the only BigData open source projects but Cassandra, MongoDB and numerous others popped out soon.   The NSA itself having built its own big-data technology open sourced its technology as well.  In every area of the virtuous circle we see open source companies critical to the growth and innovation taking place.

Companies form around the open source projects to support the companies using the projects which is critical to the success of the open source project.  Some of these companies are now approaching the valuation and sales of traditional Proprietary software companies.    There is no doubt that many of these companies will eventually become as big as traditional closed source companies and we may see the disruption of the closed source model more and more as companies realize there is no advantage to the closed source model.

 

The Impact of the Virtuous Circle

The virtuous circle of technology has been in operation like this for the last 8 years or so.   Its existence cannot be denied so the questions are:

1. To what extent do these technologies change the underlying costs and operation of my business?

2. To what extent do these technologies change the way I sell my services to the world?

3. To what extent do I need to adopt these technologies or become a victim of disruption?

These questions should be critical to any company, to its business leaders as well as the CIO, CTO and software technologists.   An example of this would be Uber I refer to every now and then.   The cab industries in NYC and Paris and other cities weren’t looking at the Cloud, Mobile apps, Social.  They didn’t see that they could offer dramatically better service to customers by integrating their cabs with mobile devices, the cloud and social.  So, they have uniformly been surprised by the growth of Uber and now competitors like Lyft etc…    I don’t know how this will resolve in that case but we can see how the music industry hasn’t had a smooth transition to the new technologies.

Some businesses such as advertising are undergoing a radical transformation.  Advertising was one of the least technology savvy industries for many years.  The growth of digital advertising has changed this business to one of the most technology intensive businesses.  One advertising business I talked to is contemplating 70 billion transactions/day.

Everyone who faces consumers is feeling the effects of the virtuous circle.  The need to adopt mobile apps, to adopt social technology, big-data, APIs and consequently to adopt Cloud, DevOps, Open Source is unmistakable.

However, the impact of these technology improvements affect the way everyone develops software, how everyone in any business can learn to operate their business cheaper, to be more agile and adapt faster.    Some are calling this change in the basic building blocks of software Platform 3.0.   I will explain Platform 3.0 and what it is in later blogs but it is a critical change in Enterprise and software development that every organization needs to look at.

Therefore the impact of the virtuous circle has become virtually ubiquitous.  The scale of the businesses of mobile with billions of users, social with billions of users, APIs with billions in revenue and 10s of thousands of APIs, Cloud now a $160 billion/year business growing at a very high rate and other changes that have spun off from these in terms of how everyone operates makes this technology and circle a critical understanding in todays world.

 

Changes to the Virtuous Circle

As we move forward there are some things we can see that are happening.  I will be blogging on all these topics below more.

  1. Internet of Things is real and growing very fast

The Internet of Things (IoT) is expected to be somewhere in the 7-19 trillion business in very few years.  This is counted by looking at all the hardware that will have IoT capability.  In business we will see IoT everywhere.  The underlying technology of IoTs will undergo massive change like all the previously described areas so I see IoT is integrated into the Virtuous Circle already.   Some IoT technologies are open source already and there is more and more movement to standards and collaborative development.

  1. The Network Effect

Having a thermostat that can learn and be managed remotely is cool and somewhat useful but when you combine it with other IoTs the value grows substantially.  Being able to monitor your body work and consumption is useful to the individual but nobody knows what we could do if we had this information over many millions of people.  The effect on health could be dramatic essentially allowing us to reduce the cost of medical trials and affecting health care costs and outcomes dramatically.  The same with all IoTs.   The same with all API’s.  Each API by itself has some utility.  However, when one combines APIs, IoTs, mobile apps and billions of people with billions of devices we don’t know where all this is going but I believe this means the virtuous circle will continue to dominate the change we see in the world for the foreseeable future.

  1. Privacy and Security

So far in this evolution of the virtuous circle we have mostly sidestepped issues of privacy and security.  It is only in 2013 and now in 2014 that we have seen the Cloud start to pull ahead dealing with some security issues.  All of these trends have had a negative effect on privacy.  I find this personally difficult to understand but people seem to be waiting for the first scandal or to see where this will go before they make any decisions about how we will adjust our ideas of privacy.   I believe at some point in the next decade we will see a tremendous change in technology to support privacy but that remains to be seen.

Other resources you can read on this topic:

The virtuous Circle 

The Nexus of Forces: Social, Mobile, Cloud and Information

The “Big Five” IT trends of the next half decade: Mobile, social, cloud, consumerization, and big data

Cloud computing empowers Mobile, Social, Big Data

Nexus of New Forces – Big Data, Cloud, Mobile and Social

The technology “disruption” occurring in today’s business world is driven by open source and APIs and a new paradigm of enterprise collaboration

IoT 7 TRILLION, 14 TRILLION or 19 TRILLION DOLLARS!

 

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s