Showing posts with label Microservices. Show all posts
Showing posts with label Microservices. Show all posts

Sunday, December 8, 2019

Arch Unit

If Software Architecture is done to a reasonable standard, we should expect to see:
  • Well designed patterns that can fulfill both functional requirements and non-functional requirements
  • No crazy crazy coupling, concerns are properly separated and everything is testable.
If we get that, we should have confidence that as the software evolves it is maintainable. So the tricky part is all too often Architectural rules start off great on a whiteboard (or a powerpoint slide) but just get lost in code, because they are too difficult to enforce.

Arch Unit is a super mechanism to impose Architectural rules and patterns on your code base.  It has been around a few years but something I only discovered this year.  I came across it when I was trying to think of ways to ensure the proverbial "utils" package did not turn into the proverbial "dumping ground". Ideally, we'd have no utils packages ever. But, in the real world they nearly always exist. The utils package shouldn't have many efferent dependencies.   So for example, suppose you have a package called shoppingcart.   Then you have need some sort of utility function to add the total of two carts, remove special offers, add loyalty discounts, blah blah blah.  The last thing you want to see is someone checking that into the utils package with dependencies towards the shoppingcart package.  Because, if it is so shoppingcart focused, it should really just be in the shoppingcart package. If this happens, very soon your utils package will have dependencies to everything and everything will have dependencies to it.  Disaster. What is the point in packages if anything can just depend on anything? They will cease to provide any name-spacing or encapsulation benefits.

So, how can Arch Unit help?  Well very simple you define Architectural rules like a JUnit test.  Wait a sec...  It is a JUnit test.    The efferent (outward) and afferent (inward) for you utils package are very simple expressed as:

@ArchTest
public static final ArchRule utilPackageDependenciesRules = classes().that().resideInAPackage("com.company.application.util")
           .should().onlyDependOnClassesThat().resideInAnyPackage(getAllowedDependencies("com.company.application.exception"))
           .andShould().onlyHaveDependentClassesThat().resideInAnyPackage("com.company.application.shoppingcart"
 "com.company.application.payment);
So that's it. Now repeat for every package and you now have code control that runs like any other JUnit test. So therefore it will run easily as part of your CI, CD etc. Now, if you have architected your packages well, you don't  have to bring up at code reviews. Instead the rules are part of your CI. As your software evolves and new packages come along and dependencies rules change, simply just change the rules that are expressed in nice fluent Java APIs. Someone new joins the teams and wants to get up to speed on the Architectural package rules? Simple, just direct them Architectural tests.

Not only does ArchUnit give you the ability to express package rules, you can also define your own rules aka conditions and then apply them to whatever code you want. For example, suppose you want a condition that an object is immutable. You naturally therefore want no setters. That could be expressed by this condition.
    static ArchCondition noPublicSettersCondition =
         new ArchCondition("class has no public setters") {
             @Override
             public void check(JavaClass item, ConditionEvents events) {
                 for (JavaMethod javaMethod: item.getMethods()) {
                     if (javaMethod.getName().startsWith("set") && 
                       javaMethod.getModifiers().contains(JavaModifier.PUBLIC)) {
                         String message = String.format(
                             "Public method %s is not allowed begin with setter", javaMethod.getName());
                         events.add(SimpleConditionEvent.violated(item, message));
                     }
                 }
             }
         };
You could then apply the noSetter condition to any custom Exception a developer may write. It wouldn't be good if an Exception had a setter would it?
    @ArchTest
    public static final ArchRule noExceptionsHaveSetters = classes().that()
      .areAssignableTo(RuntimeException.class).should(noSettersCondition);
    
Suppose you keep noticing that Loggers defined in classes either aren't private, aren't static or aren't final. Don't waste time talking about it code reviews. ArchUnit it!
    @ArchTest
    public final ArchRule loggers_should_be_private_static_final =
            fields().that().haveRawType(TaLogger.class)
                    .should().bePrivate()
                    .andShould().beStatic()
                    .andShould().beFinal()
                    .because("we agreed on this convention");
So the goal here is to conceptualise good rules that will help your to remain testable and maintainable  and then enforce them in a way that is easy to check and understand. ArchUnit really is a great library tool.

Sunday, June 9, 2019

Defining a Resource

Fielding's dissertation  describes a Resource as:

"Any information that can be named" ... "a document or image, a temporal service (e.g. “today’s weather in Los Angeles”), a collection of other resources, a non-virtual object (e.g. a person), and
so on. In other words, any concept that might be the target of an author’s hypertext
reference must fit within the definition of a resource. A resource is a conceptual mapping
to a set of entities, not the entity that corresponds to the mapping at any particular point in
time."

Defining a Resource is both a Science and an Art. It requires both Domain knowledge and API Architectural skills.   The following points detailed below serve as a checklist which may help you determine the shape of your Resource, what data it should contain and how it should be presented to consumers of your API.

The Resource must contain a Business Description

  • The business description should be 3 - 4 sentences in simple prose which explain what the Resource is. 
  • A developer with a moderate knowledge of  your system should be able to understand the description
  • Any caveats of the Resource should be made clear

The Resource should be useful on its own


This is similar to the maxim of defining the boundary of a micro-service, where a micro-service should be considered to be useful on its own.  Similarly, a Resource should be useful on its own.

For example, instead of:
/street-address/{id}

RESPONSE

{
    "street1": "String",
    "street2": "String"
}
and
/address-extra/{id}

RESPONSE 

{
    "city": "String",
    "country": "String"
}
It should be:
/address/{id}

RESPONSE

{
    "street1": "String",
    "street2": "String",
    "city": "String",
    "country": "String"
}
If a Resource on its own is not useful and always necessitates a subsequent request, it means code will inevitably become more complex as well as there being a performance impact incurred from the second request

Use an Appropriate Noun


Use of a simple noun over a compound noun is preferred.  For example, Address is better than AddressInfo or AddressDetail.  This is a general rule, there will always be exceptions.

If using multiple Resources to represent different views of the same data, for example: Address and AddressDetail, use the simple noun e.g Address first.  Then if the second representation is more detailed use ResourceNameDetail or if it is less detailed use ResourceNameSummary.  For example, suppose there is a requirement to introduce an Address type Resource:
  1. Address is introduced first
  2. If a subsequent view of Address is needed that is more detailed, the new Resource should be called AddressDetail
  3. If a subsequent view of Address is needed that is less detailed, the new Resource should be called AddressSummary

If it is only used in a READ does it need to be a Resource?


If a Resource is only ever used in a Read request and never a Write (Create, Partial Update, Full Update, Delete, ...) request it is questionable if it needs to be defined as a Resource with its own URI.  It could just be added to the parent payload and if there is a concern that payload then becomes too complex, the parent could just provide a sparse query - where the client can decide per API request what it wants returned.

Resources should conform to the uniform interface


The uniform interface is a very important part of good API design.  It is not just about using special verbs for different requests but also ensuring the data shape is consistent.

If creates, reads, updates, deletes etc are done in a consistent way, it means code is more consistent, reusable and more maintainable.

This means:
GET /addresses/{id}
and
GET /addresses
must return the same address data structure to represent an Address.
GET /addresses/{id}
RESPONSE
{
    "id":"546",
    "street1": "String",
    "street2": "String",
    "city": "String",
    "country": "String"
}
and
GET /addresses

RESPONSE
{
    "elements": [
         {
              "id":"546",
              "street1": "String",
              "street2": "String",
              "city": "String",
              "country": "String"
         },
         ...
     ]
}
Similarly, for write payloads, the Data Structure should be the same.  So, a partial update to change street1 would be:

PATCH /addresses/{id}
REQUEST

{
    "street1": "Walkview"
}

RESPONSE
{
    "id":"546",
    "street1": "Walkview",
    "street2": "Meadowbrook",
    "city": "Dublin",
    "country": "Ireland"
}
and not something like
PATCH /addresses/{id}
REQUEST

{
    "newStreet1Value": "Walkview"
}

From a Resource perspective, the data structure must be consistent. A different data structure means a different Resource, which should be named differently and have its own path.

Don't expose everything


If your DB model is quite sophisticated, you can be sure not all attributes need to be exposed at an API level. Some fields may only be getting persisted for back office processing and should never presented make it to any UI.

When adding an attribute to a Resource, consider:

  • to only include fields that you are sure the client is interested in 
  • if you are not sure, leave the attribute out. It is much smaller problem to add an attribute later on, then to remove an attribute that has already been exposed.

API Models shouldn't blindly mirror DB Relational model or OO Models


In database modelling approaches such as normalizing data or collapsing inheritance hierarchies are used.  In Object Orientated design, techniques such as polymorphism, inheritance hierarchies etc are used to promote things like code reuse and to reduce coupling.

Resource modelling does not have to follow theses techniques. The consumer of an API doesn't care if the data is all in one table, or normalized over multiple tables.  In general, the API returns data in a format that is easy to use and does not require much additional mapping by the client before it can become useful.

Use Hierarchical data to Avoid repetition


One of the advantages of hierarchical data over flat formats such as CSV is that it provides a mechanism to avoid repetition.  For example, consider a flat data structure which contains a list of persons and what team they are in.  In CSV this is:

team, firstname, lastname
Liverpool, Mo, Salah
Liverpool, Andy, Roberston

In JSON this could be:
{
    "team": "Liverpool",
    "players": [
        {
            "firstName":"Mo",
            "lastName":"Salah"
        },
        {
            "firstName":"Andy",
            "lastName":"Roberston"
        },
         ...
     ]
}

Use Hierarchical Data to Make context clear


Another advantage of hierarchical data is that it helps provide context. To understand a flat data structure you need to know what the query was that generated the data to understand the meaning of it.  For example, consider a bunch of rows that contain a date range.

name, fromDate, toDate, holidays
Tony, 2018-01-01, 2018-02-02, true
Tony, 2018-02-03, 2018-03-01, false

You could make assumptions that there is a new row when there is a change in Tony being on holidays.  But, what if there is another column?

name, fromDate, toDate, holidays, sick
Tony, 2018-01-01, 2018-02-02, true, false
Tony, 2018-02-03, 2018-03-01, false, true

Are the date ranges corresponding to holidays, sickness or both?

If we get more data back maybe it might be clearer...
name, fromDate, toDate, holidays, sick,
Tony, 2018-01-01, 2018-02-02, true, false
Tony, 2018-02-03, 2018-03-01, false, true
Tony, 2018-03-02, 2018-04-01, false, false
Now it looks like it's sickness that the date range corresponds to and its only a coincidence it lines up with a holiday period. However, when we get more data back this theory also fails.
name, fromDate, toDate, holidays, sick,
Tony, 2018-01-01, 2018-02-02, true, false
Tony, 2018-02-03, 2018-03-01, false, true
Tony, 2018-03-02, 2018-04-01, false, false
Tony, 2018-04-02, 2018-05-01, true, false

It gets even more complicated when just don't have some information.  For example:
name, fromDate, toDate, holidays, sick,
Tony, 2018-01-01, 2018-02-02, true, false
Tony, 2018-02-03, 2018-03-01, false, true
Tony, 2018-03-02, 2018-04-01, false, false
Tony, 2018-04-02, 2018-05-01, true, false
Tony, 2018-05-02, 2018-06-01, null, false
Tony, 2018-06-02, 2018-07-01, null, false
Tony, 2018-07-02, 2018-07-08, true, false
Tony, 2018-07-08, 2018-07-09, true, null

The limitation with flat data structures is not only lack of normalisation but that they can only go so far in making the data self-describing.

When it isn't clear what data means, it is inevitable processing the data will be buggy.

We could represent the same person data in hierarchical format as:
{
    "name":"tony",
    "holidays": [
         {
            "fromDate":"2018-01-01",
            "toDate":"2018-02-02"
         },
         {
             "fromDate":"2018-04-02",
             "toDate":"2018-05-01"
         }, 
         {
             "fromDate":"2018-07-02",
             "toDate":"2018-07-09"
         }
     ],
     "sick": [ 
         {
             "fromDate":"2018-02-03",
             "toDate":"2018-03-01"
         }
     ]
}
Now, the data is much more self describing and it is clear when a date range is for a holiday and when it is for a sick period.

Resource Relationships

Resources on their own only describe themselves. A Resource model describes relationships between Resources.  This will give an indication of:
  • dependencies between Resources. What Resources are needed for a particular Resource to exist or what is impacted when a particular Resource changes: updated or deleted.
  • Data navigation - in a large domain model, it is much easier to understand and follow if navigational and directional sense is provided to consumers of the model.  Especially, when to navigation across (Resources loosely connected) can be be differentiated from navigation down (Resources strongly connected)
Hypermedia links aren't only used to achieve HATEOAS.  Resources that describe what they are linked to using hypermedia links demonstrate a very powerful mechanism to express the Resource model. Advantages include:
  • A large domain model is split into more manageable pieces.  Typically users are only interested in a particular part of the model.  When Resources self describe their own relationships, it means a large complex model is split up into more digestible chunks and users get the information they need quicker. 
  • The Resource model is self-describing and kept in sync with code. Everything is co-located.
Make clear Parent - Child relationships
A Child Resource describes its Parent URL hierarchical name spacing. A Parent Resource has children of one or many types should make this clear by providing links to the children.  For example, if a Team Resource has Players child Resources.  The Team payload should make this clear.

REQUEST
https://api.server.com/teams/4676
RESPONSE

{
    "id":"34533",
    ...,
    "_links": {
          "self":"https://api.server.com/teams/4676",
          "players":"https://api.server.com/teams/4676/players"
    }
}

Make clear Peer relationships

This is similar to above except it is for Resources that exist in a different hierarchical name space. So for example, suppose the team is in division 1.  A link should be included in the team's division attribute.
REQUEST
https://api.server.com/teams/4676

RESPONSE
{
    "id":"34533",
    "division": {
        "name":"Division 1",
        "_links": {
              "self":"https://api.server.com/divisions/1"
        }
     },
     ..., 
    "_links": {
        "self":"https://api.server.com/teams/4676",
        "players":"https://api.server.com/teams/4676/players"
    }
}

Make clear Links to Other Representations

If data is modeled to have multiple Resources which represent different representations of the data, the Resources should also include links to each other.
REQUEST
https://api.server.com/teams/4676

RESPONSE
{
    "id":"34533",
    "division": {
        "name":"Division 1",
        "_links": {
              "self":"https://api.server.com/divisions/1"
        }
     },
     ..., 
    "_links": {
        "self":"https://api.server.com/teams/4676",
        "players":"https://api.server.com/teams/4676/players",
        "teamDetails":"https://api.server.com/teamDetails/4676"
    }
}

Thursday, May 19, 2016

Immutable pointers - a pattern for modular design to help achieve microservices

Modular design has a lot of benefits, including:
  • making it easier to predict the impacts from a change
  • helping developers work in parallel
But, it is much much much harder than people think.  For a start, the abstractions have to be at the right level.  Too high and they can become meaningless and too low and they cease to be abstractions, as they will end up having way too many dependencies (efferent and afferent)

In a recent green field project, which was in the area of digital commerce, I was intent on achieving good modular design in the back end for the reasons outlined above and because the project had potential to grow into a platform (more than likely microservices) that would be used by multiple teams. To achieve the modular design, after some white boarding the back end was split up into a bunch of conceptual components to achieve shopping functionality.  The core components are listed below.


  • Shopping component - Core shopping functionality e.g. Shopping Cart management
  • User component - user management, names, addresses etc
  • Purchases - details about previous transactions
  • Merchant - Gateway to merchant API to get information about merchant products etc. 
  • Favourites - Similar to an Amazon wishlist
Now, every software project uses the word "component" differently. In this project, a component was strictly defined as something that did something useful and contained:
  • domain classes 
  • services
  • a database schema  (could be its own database, but the idea was to isolate it from any other components persistence) 
  • its own configuration
  • its own dedicated tests
  • its own exception codes domain 
The outside world could access a component via a bunch of ReSTful endpoints. 

Any component could be individually packaged, deployed etc.  Now, the astute out there will 
be thinking,  "this sounds like microservices", well they almost were.  For this project, 
some of them were co-located but they were architected so that deploying them out into 
individual  deployed artefacts (and hence a microservices approach would be easy). 

Ok, to reiterate, the goal was achieve a very clean modular design.  This meant, that I didn't want 
any dependencies from one component's database scheme to another and for this blog post we are only going to focus on how that aspect of the modularity was achieved. 

Now, looking at the above components, it doesn't take long to see that isn't going to be so easy.  For example: 
  • A shopping cart (in the Shopping component) will have a reference to a User (in the User component)
  • A shopping cart item (Shopping component) will have a reference to a Product (Merchant component)
  • A Shopping cart (Shopping component) will have a reference to a shipping Address (User component

So the challenge of achieving modularity in the persistence tier should now be becoming more clear. References of some sort across components need to be persisted.  Immediately, any developer will ask, 
"Wait a sec, if we just use foreign keys for these inter schema references we get ACID and referential 
integrity for free!"  True. But then you are loosing modularity and introducing coupling.  Say you want to move your products (in the Merchant Component) away from a relational database and use something like Elastic or Mongo DB instead - to leverage their searching capabilities.  That foreign key isn't so useful now is it?  

Ok, so first of all in looking for a solution here, I thought about all the references that were across 
components to see if there was anything in common with them.  One thing that was obvious was that 
they were generally all immutable in nature.  For example:
  • When a Cart Item (Shopping component) points to a Product (Merchant component) it points to that product only.  It never changes to point to another product.  
  • When a Shopping Cart (Shopping component) points to a User (User component), it is also immutable.  My shopping cart is always mine, it never changes to be someone else's.
So I was now starting to think about preferences:
  1. Avoid cross component dependencies if you can (this should be kinda obvious)
  2. If you have to have them, strive for immutable references.
So, next up was to have a name for this type of relationship - which I was calling "Immutable pointer" in design and architecture documents. But, for actual code I needed something more succint. The database schema was already using "id" for primary keys and "{name_other_relationship}_id" for foreign keys. So I decided all cross component relationships were named as the the name of the entity being pointed to and "ref".  

So, some concrete examples:
  • userRef   (ShoppingCart pointing to the user)
  • productRef (CartItem pointing to the product)
  • shippingAddressRef (ShoppingCart pointing to the ShippingAddress)
This meant anytime anyone saw something like "xyzRef" in code, schema or logfiles they knew it was a cross component reference. In case it wasn't obvious, Ref was an abbreviation for Reference.

Next up was to decide on the format for the actual Refs.  I took a bit of inspirational from that thing call the internet, which of course has similar concepts where abstractions: web sites of web pages contain immutable pointers to other abstractions: hyperlinks to web pages in other web sites.


So similar to hyperlinks and URLs, the refs would follow a hierarchical name spacing format.
Some good team input from senior technical member suggested to continue the inspiration from the Web and start the hierarchical names with cp://.  CP for Commerce Platform the name of the project. This was analogous to http://  I thought this was good idea as it indicates that our platform generated the reference.  Again, this meant they stood out in logfiles etc and could be differentiated from any downstream components also using hierarchical type references but in a different context. 

The key point of the ref was that when generated it should of course be unique.  To achieve this 
a mixture of database primary keys or something unique about the data (e.g. product skus were used)

So some examples: 
  • userRef -> cp://user/{uuid of user}
  • productRef -> cp://merchant/{uuid of merchant}/product/{sku of product}
  • cardRef -> cp://user/{uuid of user}/card/{uuid of card}

Always immutable?

As stated, the first preference was to avoid the cross component reference. The second preference
was to use the immutable pointer (ref pattern). However, what about an edge case where the cross component reference could be mutable.  Could this happen?  Well it could.  Easiest to explain by an example.   

Every shopping cart doesn't just have a User, it also has a selected shipping address where the contents purchased will be shipped to. In the domain model, the user's address lived in the User component.  But unlike the other cross component references the shipping address could change. Consider your Amazon shopping cart.  Imagine you are on the checkout screen with your selected card and your selected address, but before you proceed to checkout you go into your user preferences and delete your card and address.  This project had to facilitate similar scenarios.  So the User deletes the address that a shopping card id pointing to what should happen?

So for inspiration for solution here, we can look to the various NoSQL patterns and in particular 
one of the most popular is eventually consistency. What this says is that unlike ACID you don't 
always need consistency straight away, all the time.  In certain cases it is okay to allow inconsistency
on the basis that the system is able to reconcile itself. 

So in this case:
  1. The shopping cart is pointing to a specific shopping address using an addressRef. 
  2. The user deletes that address by hitting a ReST endpoint in the user component.
  3. This means the shopping cart will point to an address that doesn't exist. The system is inconsistent.
  4. The next time the user reads the shopping cart, in the request handling the shopping component asks the user component if the address with this address ref sill exists and if it doesn't removes the pointer.  
  5. This system is now consistent
So with the architect hat on it is really important we get this all right. Otherwise the goal 
of modular design falls apart. 

In this case, it is worth reiterating the strategy one more time:
  1. Avoid cross component references.  Doesn't matter how great your pattern is, if you have a lot of cross component references,  it is more than likely you have got the component abstractions at too low a level.
  2. Favour immutability.  In general immutability means less code paths, less edge cases, less complexity in code. 
  3. Eventual consistency.  
Software Architecture is about trade offs, and finding the right balance.  In this case, it was the balance between clean modular design but not going over board with it so that it become impossible to achieve. 

For anyone trying to do microservices, I would strongly recommend trying to master how you would do modular design in your architecture first.  If you can't do this, when you add in the complexity of the network things get very complicated. 

  

Sunday, February 7, 2016

Book Review: Building Microservices

The architectural idea of microservices was inspired - in part - by Unix's philosophy of code being short, simple, clear, modular, extendable and that it could be repurposed by developers. The term is currently up there with Internet of Things, Big Data and the Cloud, in contemporary technical lexicon.

Author Sam Newman is an industry expert on the subject. He has:
written for Infoq, presented at Javazone and various other events regarding microservices.

His book 'Building Microservices' does an excellent job introducing the key concepts of microservices:
  • services are autonomous, live on their machines
  • resilience - one service fails, it doesn't impact other services
  • scaling - because services are independent they can be independently scaled
  • deployment - deployment should be easy. A change to a service means that's all that's deployed.
It is worth pointing out it isn't just a book about microservices. Many of the ideas and best practises detailed (for example HATOES in your ReST approach and to check OWASP for security references) are equally applicable to non - microservices architectures. But, that said, what I really like about this book is anytime a general architectural or software engineer concept (and there are a lot) is explained it is explained very well.  Some examples:
There are - of course - many tips regarding how you approach microservices. For example:
  • Don't get too hung up on DRY. It may make it harder to keep your services independent.
  • Consider using CDC's for your testing approach
  • Canary releasing / Blue, Green testing for releasing.
  • Using bulkhead and circuit breaker patterns to make services ore resilient.
As an overview to microservices, it's a great book. However, the reader must bear in mind there is no one size fits all approach to microservices. The finer details will depend on your project, your team and even a bit of trial and error. It's difficult to critique this book. So instead, I'll just flag some concerns not about the actual book but about microservices in general and how the industry and developers are reacting to them.

My first concern with microservices is more a practical than a technical one. Doing modular design for everything from your schema, service layer, end points, configuration isn't as easy as some people might think. Especially in teams of varied skill level, varied backgrounds and inevitable commercial pressure that happens in every project. It requires a lot of technical aptitude, leadership and discipline. If a project is not good at achieving modular design in a monolith - for whatever reason - I think it will really struggle at modular design in microservices when the complexity of the network has to be also considered.

The second concern I'd have is that when explaining the principle ideas of microservices, it is often compared with a monolith to the point the word monolith is a pejorative term.  The two approaches: monolith and microservices are presented as if it's either one or the other.  I think this a false dichotomy fallacy. There are other approaches available.

Thirdly, microservices are no doubt,  a clever idea.  But that doesn't mean there are a panacea. In some projects they will be a good fit, in others they will not be worth it. One obvious factor to bear in mind is your non-functional requirements.  One useful point of reference,  that is worth considering is the project James Lewis (another Thoughtworks guru) described in his talk about microservices in 2012.  In this presentation, three non-functional requirements for the project caught my attention:
  • One component had to handle 1,000 TPS
  • Another had to support a user base of 100 million users
  • Another had to support batch loads of 30 - 90 million records
While I am not saying you should be in this ballpark before considering microservices, I am just trying to suggest that most projects don't have these sort of demands and there's a lot of merit of considering something like a very well structured modular monolith first using DDD principles and then to migrate towards microservices should the benefits justify it. This strategy is well explained by Martin Fowler in his Monolith first article.

Some other references:


Until the next time enjoy yourselves.


Monday, August 3, 2015

Book Review: Cloud Based Architectures

Migrating to Cloud-Native Application Architectures, Matt Stine.

In the last few years, it can be very easy to believe marketing departments simply aren’t paid unless they use the word "Cloud" as often as possible.  Its usage has become so ubiquitous that is has inevitably become ambiguous so one very astute thing author Matt Stine does is outline what he means by "Cloud" (in particular cloud architectures) in the opening pages of this book.  Essentially, cloud architectures can be summarised by the three S’s. 

  • Speed: ability to provision and release resources (computing, networking and storage) with ease thereby achieving faster project development time
  • Safety: making software architectures more resilient (fault tolerance, fault isolation etc) 
  • Scalability: One the many non-functional characteristics that has become even more important with proliferation of mobile computing.  More mobile devices mean more application usage mean more back ends APIs getting hit.  Architectures must be able to respond to this demand otherwise applications fail. 

Twelve Factor App

Throughout the book approaches and patterns for achieving cloud based architectures are introduced. Now, if you have worked on anything distributed which was well architected, you will definitely experience a little bit of deja vu.  For example, the Twelve Factor App is a collection of patterns developed by engineers at Heroku containing twelve (you guessed it) principles to guide a good architecture fit for cloud.  You are sure to have used codebase (make deployable units have their own code base), dependencies (use proper tooling for managing dependencies), config (externalise configuration), backing services (databases, caches etc should be consumed identically by all resources) before but patterns are supposed to be common solutions to common problems and good architecture is good architecture, so it is not unexpected some of the described approaches won’t always sound completely new. 

Microservices

How do Microservices help enable the 3 S’s?
  • Speed: Deployment is much simpler and faster with independent code chunks.
  • Security: Not too much here, but it is fairly intuitive that if you break something large into small chunks it is much easier to access control sensitive parts. 
  • Scale:  Much easier to only have to scale the parts of an architecture you need to scale as opposed to having to scale the entire architecture - which is what happens in a Monolithic application.

Cloud Migration

For organisations to move to Cloud based architectures, several changes are needed:
  • DevOps rather than Silos: this is to facilitate speed of delivery.
  • Continuous Delivery:  one way to be to sure the problems of long release cycles are guaranteed to be avoided 
  • Decentralise autonomy:  yes, it is a nice idea to give everyone more power and influence. It will certainly speed things up. But, in my own view some things (core APIs, major technical architectural decisions) should only be handled by people of a certain technical background who are prepared to be fully accountable for them.  Would you let every person working on the design of a bridge or in a heart transplant have the exact same say? 
  • Inverse conway manoeuvre: Software companies should do their architecture first and then re-align their organisations to fit that so that they avoid the anti-pattern of making architectures to resemble an existing organisational structure.   This sounds nice in theory but it may not be so easy to achieve (possibly easier to do in very small companies that never have a strict organisational structure anyway).

Break up the Monolith

So if you have bought into the cloud based approach and since it’s unlikely your project is green field, you need to figure out how to break up the Monolith.  Some ideas include:
  • Bounded Contexts: This allows inconsistent definitions of concepts as long as they are consistent inside a well defined context which makes it much easier to decompose data models. For example, in the Security context a User may always refer to something that can be authenticated and has particular roles but in a Management context it would be something that has an image, an address and some application capabilities (checking out a shopping cart).
  • Identifying bounded contexts in the monolith and making them micro services.
  • Containerisation: Using something like Docker can provide many advantages to using VM's
  • Writing new features as micro services
  • Making the Monolith look like micro services by using anti-corruption layers

Spring cloud and Netflix

By the sounds of it Spring cloud's and the Netflix OSS project sound like an absolute must to checkout. Amongst other things, they help:
  • Achieve dynamic configuration
  • Implement sophisticated service discovery
  • provide alternative to load balancing which involve client side more.  
In addition, Netflix's Hystrix library (which provides some very useful metrices) employs some clever fault tolerance patterns:
  • circuit breakers - if a system notices that another system is broken it stops calling it.
  • bulkheads -  failure is limited by strict partitioning

Any negatives?

So lots of positives about this book.  Plenty of well explained information in a short number of pages. That is a major achievement.   I have alluded to a few minor points above, but they are very minor.  It would be very difficult to criticise this book in any substantial manner especially when it is free.    

However, it is important for the reader to bear a few considerations.
  • Business realities: Approaches such as Continuous Delivery just may not fit your business plan.  Why would you want to provide a feature someone hasn’t paid for?  (In fairness, the author does make this very point, but it really is worth re-iterating)
  • Technical realities:  One problem I have with many software approaches is that they assume that everyone is at a given skill level. In an ideal world the only people who work in software engineering would be very good engineers but the reality is there is always a very wide range of skill levels on any project.  From super meticulous to complete spoofers.  This means that some things that can sound very elegant in theory but a lot more difficult to achieve in practise.  When considering moving to things like DevOps, there will definitely be benefits. But are organisations going to find it easier or harder to make engineers accountable? I would imagine there would be serious risks of it being latter if it is not properly thought through.
  • Some concepts e.g. Client Side load balancing sound very interesting. But, they are only introduced rather than critically analysed.  Much more research and thought would be required (for me anyway) before considering adopting.