Scala Snippet: How to filter a list in Scala

In Scala, filtering and processing collections is easy and elegant. There are many filtermethods available, but the most used will probably the basic filter method.

Here’s a code example of some filtering on my (ex)camera collection.
The filter method will not only work on Lists, but on any Scala collection.

 
object MyCameraCollection02 {
 
  case class Camera(brand: String, model: String, sensorType: String, yearBought: Int) {
    override def toString: String =
      s"$brand $model \t\t $sensorType \t($yearBought)"
  }
 
  def main(args: Array[String]) {
    val canon5dmarkIII = new Camera("Canon", "5D MkIII", "FF", 2013)
    val canon5dmarkII = new Camera("Canon", "5D MkII", "FF", 2009)
    val canon6d = new Camera("Canon", "6D", "FF", 2014)
    val canon550d = new Camera("Canon", "550D", "APS-C", 2010)
    val canon40d = new Camera("Canon", "40D", "APS-C", 2008)
    val canonIXUS330 = new Camera("Canon", "IXUS 330", "1/2.7", 2001)
    val canonIXUSZ90 = new Camera("Canon", "IXUS Z90", "APS-C", 1999)
    val panasonicGM1 = new Camera("Panasonic", "GM1", "M43", 2014)
    val panasonicFZ20 = new Camera("Panasonic", "FZ20", "1/2.5", 2005)
    val sonyrx100 = new Camera("Sony", "DSC-RX100", "1\"", 2013)
    val sonynex5 = new Camera("Sony", "NEX-5", "APS-C", 2011)
    val sonyr1 = new Camera("Sony", "DSC-R1", "APS-C", 2005)
 
    val myCameras = List(canon5dmarkIII, canon5dmarkII, canon6d, canon550d, canon40d, canonIXUS330, canonIXUSZ90, panasonicGM1, panasonicFZ20, sonyrx100, sonynex5, sonyr1)
    val canonCameras = myCameras filter (_.brand == "Canon") // Every Canon camera I ever owned
    val sonyCameras = myCameras filter (_.brand == "Sony") // Every Sony camera I ever owned
    val pansonicCameras = myCameras filter (_.brand == "Panasonic") // Every Panasonic camera I ever owned
 
 
    // apscCamera's only
    val apscCameras = myCameras filter (_.sensorType == "APS-C")
    println("==APS-C camera's owned==")
    apscCameras foreach println
    println()
 
 
    // Canon camera's which are not full frame. You can filter, filtered lists.
    val canonNonFF = myCameras filter (_.brand == "Canon") filter (_.sensorType != "FF")
    println("==Non-FF camera's owned==")
    canonNonFF foreach println
    println()
 
    // Filter by boolean expressions on class properties
    val apsBefore2012 = apscCameras filter (_.yearBought < 2012)
    println("==APS-C camera's bought before 2012 owned==")
    apsBefore2012 foreach println
    println()
 
    // Filter by combining boolean expressions.
    val ffcamerasBefore2012 = myCameras filter (cam => cam.yearBought < 2012 && cam.sensorType == "FF")
    println("==Every FF Camera I ever owned before 2012==")
    ffcamerasBefore2012 foreach println
    println()
  }
}

Running this example will give you the following result:

==APS-C camera's owned==
Canon 550D 		 APS-C 	(2010)
Canon 40D 		 APS-C 	(2008)
Canon IXUS Z90 		 APS-C 	(1999)
Sony NEX-5 		 APS-C 	(2011)
Sony DSC-R1 		 APS-C 	(2005)

==Non-FF camera's owned==
Canon 550D 		 APS-C 	(2010)
Canon 40D 		 APS-C 	(2008)
Canon IXUS 330 		 1/2.7 	(2001)
Canon IXUS Z90 		 APS-C 	(1999)

==APS-C camera's bought before 2012 owned==
Canon 550D 		 APS-C 	(2010)
Canon 40D 		 APS-C 	(2008)
Canon IXUS Z90 		 APS-C 	(1999)
Sony NEX-5 		 APS-C 	(2011)
Sony DSC-R1 		 APS-C 	(2005)

==Every FF Camera I ever owned before 2012==
Canon 5D MkII 		 FF 	(2009)

Bruce Lee’s Top 5 Agile Coaching Tips

When I was a kid, I was a big Bruce Lee fan. I walked around the playground rubbing my nose with my thumb pretending I was the man. When I had a piece of rope, I had to do my version of the nunchaku routine from Way of the Dragon and made cat-like noises. Looking back at Lee, I find it quite striking how many of the principles of his fighting style Jeet Kun Do apply to agile practices.

Check out these descriptions of the fighting style:

  • “Jeet Kune Do is not fixed or patterned, and is a philosophy with guiding thoughts.”
  • “Jeet Kune Do practitioners believe in minimal movements with maximum effects and extreme speed.”
  • “The system works by using different “tools” for different situations, where the situations are divided into ranges, which is kicking, punching, trapping, and grappling, where martial artists use techniques to flow smoothly between them. “
  • “Through his studies Lee came to believe that styles had become too rigid and unrealistic. He called martial art competitions of the day “dry land swimming”. He believed that combat was spontaneous, and that a martial artist cannot predict it, only react to it, and that a good martial artist should “be like water” and move fluidly without hesitation.”

JKD is a reactive style that responds to changes in in situations and applies the best “tool” for getting the job done in the fastest way possible in that situation. JKD does not care about styles or forms for the sake of it.

Ofcourse this philosophy resonates highly with an agile mindset.

So just for fun, here are my Top 5 Bruce Lee Agile Coaching Tips
Continue reading “Bruce Lee’s Top 5 Agile Coaching Tips”

Scala Snippet: multiline statements in the REPL console

One of the coolest things a standard Scala install will give you, is the Scala interpreter.
Technically speaking, this is not an interpreter. In the background, each statement is quickly compiled into bytecode and executed on the jvm.
Therefore, most people refer to it as the REPL: Read-Evaluate-Print-Loop.

You can access it by starting a command shell on your system and typing in ‘scala‘.
Do make sure your either run it from the place where scala is installed or have scala on your environment PATH.

By using the repl, you can quickly experiment and test out different statements. Once you press ENTER it will evaluate the statement and display the result.

Frequently you want to execute multi-line statements and luckily the repl has a solution for that.

Simply type in :paste and the repl will accept multiline statements.
To exit this mode and evaluate your code, simply type CTRL+D.

Example:

$ scala
Welcome to Scala version 2.11.4 (Java HotSpot(TM) Client VM, Java 1.7.0_75).
Type in expressions to have them evaluated.
Type :help for more information.

scala> :paste
// Entering paste mode (ctrl-D to finish)

class Camera {

//this is a multiline example in the REPL

 val brand:String = "Canon"
 val model:String = "5D mark III"
}

// Exiting paste mode, now interpreting.

defined class Camera

scala> val cam = new Camera
cam: Camera = Camera@bfd117

scala> cam.brand
res0: String = Canon

Scala Snippet: Object, Companion Object and Static Methods

If you are a Java developer moving to Scala, one notable difference in terminology that can cause confusion is the term ‘object’.
In Java an object is always an instance of a class, created by calling a constructor.

Object

In Scala an object is used for defining a single instance of a class with the features you want.

In practice this means you will use it:

  • to keep utility/helper methods and constants in one place
  • to have a single immutable instance to share across a system
  • to implement the singleton design pattern to coordinate actions across a system

An object can have the same features as a class. You can extend other classes or traits. The only notable difference is that an object cannot have constructor parameters.

Companion Object

In Java there might be occassions where you want to use static methods.

In Scala you define these methods inside a ‘companion object’, which has to be defined in the same source file as the class.
The class and object can access each others private methods and variables, but you have to do it as you would call a out of scope static method in Java.

Checkout this example in the repl:

$ scala
Welcome to Scala version 2.11.4 (Java HotSpot(TM) Client VM, Java 1.7.0_75).
Type in expressions to have them evaluated.
Type :help for more information.

scala> :paste
// Entering paste mode (ctrl-D to finish)

  object Camera {
    def updateShutterCount(shutterCount:Long): Long = {
      var updatedShutter = shutterCount + 1;
      updatedShutter
    }
  }

  class Camera(brandName:String, modelName:String) {
    val brand = brandName;
    val model = modelName;
    var shutterCount: Long = 0;

    def makePicture {
      shutterCount = Camera.updateShutterCount(shutterCount)
    }
  }

// Exiting paste mode, now interpreting.

defined object Camera
defined class Camera

scala> val canon = new Camera("Canon", "5D mark III")
canon: Camera = Camera@17403a7

scala> canon.shutterCount
res0: Long = 0

scala> canon.makePicture

scala> canon.shutterCount
res2: Long = 1

It’s a trivial example, but as you can see, I’ve used the “static” method from the companion object to update the shuttercount when calling the makePicture method on my canon Camera.

Finally, a singleton object that does not share the same name with a companion class is called a standalone object.

Scala Snippet: Case Class vs plain ordinary Class

In Scala there exist the construct of a ‘case class’. According to Martin Odersky this supports you to write a “regular, non-encapsulated data structure”. It always seems to be associated with pattern matching.

So when to use a case class and when to use a ‘plain’ class?

I found this nice explanation stating:

“Case classes can be seen as plain and immutable data-holding objects that should exclusively depend on their constructor arguments.

This functional concept allows us to

  • use a compact initialisation syntax (Node(1, Leaf(2), None)))
  • decompose them using pattern matching
  • have equality comparisons implicitly defined

In combination with inheritance, case classes are used to mimic algebraic datatypes.

If an object performs stateful computations on the inside or exhibits other kinds of complex behaviour, it should be an ordinary class.“

Defining a case class gives you a lot of boilerplate code for free:

  • Getters are generated for the constructor parameters. Setters are only generated when the parameters are declared as var. They are val by default.
  • A nice toString method is generated.
  • An equals and hashCode methods are generated.
  • A copy method is generated to clone an object.
  • An apply method is generated, removing the need to use the new keyword when creating a new instance of the class.
  • An unapply method is generated.

Test it out by starting the Scala repl in a terminal:

$ scala
Welcome to Scala version 2.11.4 (Java HotSpot(TM) Client VM, Java 1.7.0_75).
Type in expressions to have them evaluated.
Type :help for more information.

scala> case class Camera(brand: String, model: String)
defined class Camera

scala> val myCamera = Camera("Canon", "5D Mark III")
myCamera: Camera = Camera(Canon,5D Mark III)

scala>

If you type in the name of your object with a ‘.’ and press tab, the repl will display some of the available methods that are generated.

scala> myCamera.
asInstanceOf   brand canEqual  copy  isInstanceOf   model   productArity   productElement   productIterator   productPrefix   toString

Final note
Alvin Alexander writes that case classes are primarily intended to create “immutable records” that you can easily use in pattern matching expressions.
So while you can define the constructor parameters as var, they are intended to be val.

Scrum Master Tip: Make it transparent!

I am frequently asked by colleagues for advice on how to be a good Scrum Master. I will discuss some of the tips I share in a couple of blog posts.

First of all I do like to state that I believe it’s best to have a Scrum Master that is able to get his hands dirty in the activities of the team (i.e. coding, analyzing, designing, testing etc.). It will enable him/her to engage and coach at more levels than just overall process.

In my opinion one of the most important things a Scrum Master has to do is to make things transparant for the whole team.

Now this seems like very simple advice, and it is. However, when you are in the middle of a sprint and all kinds of (potential) impediments are making successfully reaching the sprint goal harder and harder, the danger of losing transparency always pops up.

Here are three practical tips:

1. Make it a habit to ask yourself these questions after every standup:

  • Is it clear for everybody what we as a team should  focus on right now?
  • Is everybody focussing on the things that we should focus on right now?
  • Are we going to reach our sprint goal?

In an ideal world the answers to these questions should always be positive and clear after every standup, but in practice I have seen many standups just becoming a reporting meeting, instead of an inspect & adapt meeting. If you feel these questions have not been answered, make sure they do get answered.

2. Don’t be afraid to adjust your scrum board

Sometimes in the heat of the action, the Scrum board doesn’t give as good a representation of the current, actual status of the sprint, as it did a few days back. This can be due to many reasons. One of the reasons can be that a task/problem turns out to be more complicated, than first thought, but the sticky note representing the task still has the more general, original description.

Don’t be afraid to decompose a sticky note into more notes, so it becomes more transparent what we are currently dealing with. What I sometimes do, is get together behind a whiteboard, forget the orignal sticky notes and make a small to do list on how to finish the story. After that, I throw away the original sticky notes and replace them with new notes representing the new insights and current status.

3. Point out the gut feeling everybody has but nobody talks about

bellSometimes, in the heat of the action, you can have this feeling that things don’t go well. Maybe you feel that although everybody is working and busy, you aren’t working effectively and in-tune with each other. Your job is to make it transparent, because chances are more people have this same undergut feeling if you would directly ask them.

In my team we have a small bell and we have a rule that if you have these kinds of feelings that we don’t completely work in tune and have a common consensus, you push the bell and we have a short standup.

I have found these practices very useful in my daily job as Scrum Master in our DevOps team.

Maximizing the work NOT done

One of the principles of the Agile Manifesto makes the seemingly paradoxical statement: “the art of maximizing the amount of work not done”. This statement is part of the 10th principle of this Agile Manifesto.

The complete 10th principle reads as follows:

“Simplicity–the art of maximizing the amount of work not done–is essential”

When I first heard this statement it puzzled me and I couldn’t really grasp the meaning of it. I would think that you would want to maximize the things done. According to this statement simplicity is equal tothe art of maximizing the amount of work not done.

An example

To clarify this statement imagine the following:

You have a to-do list with more than 100 items on it. Besides that you have a NOT-to-do list with 0 items on it.To simplify the to-do list we have to inspect it and search for:

  • Duplicates
  • Unnecessary complexities
  • Dependencies

When we have done this we should also look at external factors like

  • time
  • budget
  • demand
 To-do list

Furthermore we have to declutter and prioritize our to-do list.

Based on all these factors we can move as much items from our to-do list to our NOT-to-do list as possible. According to the Agile Manifesto this is the art of maximizing the work not done a.k.a. Simplicity.

Why not everything?

You might say, “Why not just place everything on the NOT-to-do list?”

To qualify our example, note that we should make our to-do list as simple as possible, without losing any of its usefulness! In other words we can’t take so much off our to-do list that it is no longer useful.

Eliminating waste is key to this principle and as Albert Einstein once said:

Everything should be made as simple as possible, but not simpler.

 

 

 

Some tips

  • Eliminate low importance and low urgency issues. I.e. put them on the NOT-to-do list.
  • Always try to keep your processes as simple as possible, without the end-product losing its desired functionality.
  • Ask yourself if you are producing something that is most useful with the least amount of time.

Highly effective…

The interesting part of this principle is that it is generally applicable. This principle is true and important for a wide range of processes outside the software industry and can even be applied on our own personal lifes. It is a principle of time management and of work optimization, and many different expressions of this same principle are out there.

Stephen Covey’s third habit: “First things first”, is a slightly different expression of the same thing.

time_management2

 

Maximize the number of items in the fourth quadrant (the NOT-to-do list), so you can dump it!

I’ll conclude with another expression of this same principle, most of us already know as KISS 😉 (no pun intended).

Keep it Simple, Stupid!

Till skillnad från svaga charmig flicka bilder mode enkel festklänningar , men också en hel del människor gillar att klä sig. Vi välja rätt klänning för deras bröllop. Du kan också välja en festklänningar

 

Tasty Test Tip: Matching generic typed classes in Mockito

Say you have a arbitrary class under test, which is dependent on a class DataProcessor which has a method with the following signature:

 String processData((String data, MultivaluedMap  params, Token token)

you might want to stub it with Mockito 1.9.5 in the following way to match the generic typed class:

when(dataProcessor.processData(anyString(), any(MultivaluedMap.class), any(Token.class))).thenReturn(processedData);

However running this, gives the following error:

org.mockito.exceptions.misusing.InvalidUseOfMatchersException: 
Invalid use of argument matchers!
3 matchers expected, 4 recorded:
[ ....]

This exception may occur if matchers are combined with raw values:
    //incorrect:
    someMethod(anyObject(), "raw String");
When using matchers, all arguments have to be provided by matchers.
For example:
    //correct:
    someMethod(anyObject(), eq("String by matcher"));

This is caused by the generic typed MulitvaluedMap.

Solution:

 MultivaluedMap  map = Mockito.any();
 when(dataHandlerMock.getResult(anyString(), map, any(TokenDataBean.class))).thenReturn(jsonString);

Run the test and it will pass (these lines at least :-)!

Mock a superclass method with Mockito

Say you want to test a method from class which extends some kind of superclass. Sometimes you can be dependent on code in the superclass, which is undesirable in a test.

Now actually, the first thing you should consider is to refactor your code, because it’s violating the Single Responsibility design principle:
there is more than one reason why your class is subject to change. Another advice is to favor composition over inheritence. In this way you can mock the code you are collaborating with.

Having said that, sometimes you run into legacy code you just have to work with and aren’t able to refactor due to circumstances. Here’s a trick I found on Stackoverflow to “mock” a superclass method to do nothing with mockito.


public class BaseController {
     public void method() {
          validate(); // I don't want to run this!
     }
}
public class JDrivenController extends BaseController {
    public void method(){
        super.method()
        load(); // I only want to test this!
    }
}

@Test
public void testSave() {
    JDrivenController spy = Mockito.spy(new JDrivenController ());

    // Prevent/stub logic in super.method()
    Mockito.doNothing().when((BaseController)spy).validate();

    // When
    spy.method();

    // Then
    verify(spy).load();
}

Scrum your Scrum: Deviating from ‘standard Scrum’

“Scrum by the book”

I often come across teams that are ‘doing Scrum’, but fail to realize the core principles that form the foundation of it: transparency, inspect & adapt. A lot of teams I’ve met, are starting out by doing Scrum ‘by the book’. They are doing the standard events as talked about in the Scrum Guide in the obvious way. Doing 1 daily standup answering three questions,  1 sprint retrospective, 1 demo and 1 sprint planning. The obvious artifacts like a burn-down and standard sprint backlog are used.

Now to me there is nothing wrong with this approach. You have to start somewhere and if you have no previous experience to rely on, by all means, do Scrum as it comes ‘out of the box’. If you come from a waterfall background or other heavy project management approaches that try to do pre-deterministic risk control, you’ll probably find your productivity going up. [bra_blockquote align=”right”]”If after a year of Scrum, you are still doing ‘Scrum’, you don’t do Scrum'[/bra_blockquote]

However, if you keep sticking to your standard routine and blindly apply your learned techniques in new projects, you’ll sooner or later find out that the magic wears off. If you are in this situation, it’s time to Scrum your Scrum!

When I heard my first presentation on Scrum back in 2007, the guy who presented said  “If after a year of Scrum, you are still doing ‘Scrum’, you don’t do Scrum”.
I always found this expression a bit awkward, but what he probably meant was that if you keep on doing Scrum in the same manner after a substantial period of time, you are missing the core values of Scrum. You should adapt your way of working to your specific environment and circumstances.

Shu-Ha-Ri

In Aikodo there is a learning technique called “Shu-ha-ri”. This literally means “embrace”, “diverge” and “discard” and it says something like first learning from a master and copying his exact techniques and forms. After that the student learns the underlying principles and starts learning from other masters and applies that into his own practice. Finally the student learns from his own practice, he creates his own way of thinking and adapts his techniques to his own specific circumstances.

Alistair Cockburn came up with the idea to apply this way of learning to software development techniques and methodologies.

Taking it home

So applying this way of growing to your Scrum adoption, here are some tips:

  • Don’t be dogmatic about your burn-down: it’s about the transparency of seeing progress.
  • Don’t be dogmatic about your daily standup: if you are working with 2 or 3 people and you really are communicating well, you might want to skip it sometimes
  • Don’t be dogmatic about your ‘daily’ standup: if there is a lot of pressure and a tight deadline, you might want to do a standup every two hours.
  • Don’t be dogmatic about your three standard questions to answer during the standup. If there are other questions relevant for your situations progress- and impediments tracking, ask them (but remember to keep the short timebox!).
  • Don’t be dogmatic about having your user-stories in a specific ‘as a.. I want.. so that’ format. It’s about getting clear communication and the team knowing enough to build the software.
  • Don’t be dogmatic about having your user-stories to have an all inclusive list of acceptance criteria, remember it’s not a contract. Prefer face to face communication, together with the whole team.  The user stories should be a record of that communication.
  • Don’t be dogmatic about having your Scrum Master fixing all impediments. It’s about team effort. If your Scrum Master is sick or so busy fixing some impediments and you can do something about it yourself, do it yourself. The Scrum Master’s role is to make the team run smooth and if you can help him/her it’s better than waiting passively.

How are you doing Scrum in your team? If you are stuck on improving your way of working, remember Shu-Ha-Ri.

 

Tasty Test Tip: Using ArgumentCaptor for generic collections with Mockito

Mockito has a very nice feature that allows you to verify what parameters were used when a method was executed.

For example:

ArgumentCaptor argument = ArgumentCaptor.forClass(Person.class);
verify(mock).doSomething(argument.capture());
assertEquals("John", argument.getValue().getName());

However, when using generic typed objects, some problems rise up.
For example, the following won’t work:

ArgumentCaptor> argument = ArgumentCaptor.forClass(List.class);

This is obviously not a Mockito problem, but a generics problem.

To solve this, follow these two steps:

1. use the @Captor annotation.

    
@Captor 
private ArgumentCaptor> argumentCaptor;

2. initialize the Mockito annotations in your initialization method (add one if you don’t have one).

    
@Before
    public void init(){
       MockitoAnnotations.initMocks(this);
    }

And presto! You can now capture the parameters that were used when a to be verified method was executed.

 
verify(someMock).someMethod(argumentCaptor.capture());

The Quick & Dirty Fraud

If you have a car, then every once in a while, you probably have your vehicle checked to see if it’s still up to safety and environmental standards.

So you take your car to the garage and have it checked. Now, the garage will do some tests and eventually you’ll get a nice paper showing what kind of maintenance they have done.

Nowadays, cars are complex, computerized machines. (The days of dad lying under the car to do some fixing with some elemental tools are all but gone.) This means that as a customer, you will have to rely on the professional capabilities and integrity of the garage. You’ll have to trust that if the garage says the car is fixed and okay, it really is fixed and okay. [bra_blockquote align=”right”]How would we label deliberate quick ‘n dirty fixes by developers, just to get the project ‘done’, leaving the customer with piles of technical debt?[/bra_blockquote]

Now imagine that you went to the garage, received the paper that your car is okay, go on the road, and your car breaks down. What would be your reaction? You’d probably hold the garage responsible for this, as they are the experts and you paid them to do a good job. What would your reaction be if they told you that they didn’t have time to correctly solve your cars problems and did a ‘quick fix’, without them telling you?

In software development I frequently come across similar situations.  Companies hire in IT solution providers, to help them solve their IT related problems and deliver high quality solutions for their business. Now software products and projects tend to be very complex and therefore, customers will, one way or another, have to rely on the professional capabilities and integrity of the solutions provider (or the solutions provider, checking the solutions provider).

How then would we label deliberate quick ‘n dirty fixes by developers, just to get the project ‘done’, leaving the customer with piles of technical debt?

It is the responsibility of solutions providers and professional developers, to point out the consequences of quick ‘n dirt fixes. A good solutions provider therefore will, out of professional honor and integrity, have the courage to point out the severe consequences and even refuse quick ‘n dirty fixes, knowing that in the end, low quality won’t pay off.

Quality isn’t negotiable and long after the ‘quick’ has been forgotten, the ‘dirty’ will probably still be there.

The Lean-Agile Connection

Most people working in professional IT-driven companies, have heard about Agile, most people in professional companies have heard about Lean. Those who are interested in the subject, have found out that these two very popular phrases are actually closely related. So what is the relation between lean and agile? I’ll try to briefly answer this question and will start with a little history.

Dr. W. Edwards Deming

Contrary to popular belief, the origins of Lean aren’t in Japan, but derive from the Ford company in America and the work of the American statistician W. Edwards Deming. The following excerpt states the heart of his philosophy:

“by adopting appropriate principles of management, organizations can increase quality and simultaneously reduce costs (by reducing waste, rework, staff attrition and litigation while increasing customer loyalty). The key is to practice continual improvement and think of manufacturing as a system, not as bits and pieces.”

Eliminating waste and continuous improvement are very famous elements in Lean Thinking and are applied in Agile methodologies.

Edwards also made the widely-used PDCA-cycle famous (Plan-Do-Check-Act).

Two of the fundamental pilars of Scrum are Inspect & Adapt and one can easily see the close relation to PDCA’s Check & Act.

Toyota Production System

The work of Deming formed the base of the Toyota Production System, invented by Taiichi Ohno. Ohno says that everything he knows he first learned at Ford. Then all he did was go back to Japan and remove waste (The Machine That Changed the World).

The main objectives of the TPS are to design out overburden (muri) and inconsistency (mura), and to eliminate waste (muda) through the process of continuous improvement — kaizen.  The principles & practices of the Toyota Production System are the base of what we now know as “Lean Manufacturing“, used in many industries.

 

The core principles and techniques used in TPS, are at the heart of many Agile techniques (techniques that allow you to respond and adapt to change, and take advantage of opportunities while controlling risks).

Scrum

For example, they form the foundation of Scrum. Scrum for software was directly modeled after “The New New Product Development Game“ by Hirotaka Takeuchi and Ikujiro Nonaka published in the Harvard Business Review in 1986. This is the article in which the contrast is made between a traditional sequential or “relay race” approach and a holistic or “rugby” approach. Hence  the name Scrum was derived.

Takeuchi teaches Scrum in his classes at Harvard Business School. To him, Scrum is only indirectly related to software development. It is directly related to leadership and running the top companies in the world. It’s about  cross-functional teams working intensely together generating continuous improvement.

As Jeff Sutherland, co-inventor of Scrum states: “The “lean techniques” touted by Western observers are side effects of what Takeuchi and Nonaka see as the root cause of performance. And the root cause is Scrum, which is teams engaged in continuous improvement.”

and:
“The lean techniques can all be seen in a good Scrum if you look for them. I was the Chief Engineer for the first Scrum team, hired to deliver a new product in six months that would replace all legacy products. We implemented the ScrumMaster who worked on the line just as Taiicho Ohno set up at Toyota to improve on what he saw at Ford. One of our biggest customers was Ford Motor Company. The speed of the software market means the Chief Engineer needs a product owner to keep up. The product owner is embedded in the team and drives all priorities based on going to the market and then going to the gemba.

Poka yoke is embedded in all Scrums that I am responsible for through a systematic prioritization of automated testing to make deployment fail safe.

If you do not see these things in a Scrum, the people have not studied Takeuchi and Nonaka and have not implemented it properly.” 

Lean Software Development

The phrase Lean Software Development originated in a book by the same name, written by Mary Poppendieck and Tom Poppendieck.

Lean development can be summarized by seven principles, very close in concept to TPS principles:

  1. Eliminate waste
  2. Amplify learning
  3. Decide as late as possible
  4. Deliver as fast as possible
  5. Empower the team
  6. Build integrity in
  7. See the whole

Kanban

Kanban is a method for developing software products and processes with an emphasis on just-in-time delivery while not overloading the software developers. It emphasizes that developers pull work from a queue, and the process, from definition of a task to its delivery to the customer, is displayed for participants to see. The Kanban method as developed by David Anderson, is rooted in these basic principles:

Start with what you do now
The Kanban method does not prescribe a specific set of roles or process steps. There is no such thing as the Kanban software development process or the Kanban project management method. The Kanban method starts with the roles and processes you have and stimulates continuous, incremental and evolutionary changes to your system.

Agree to pursue incremental, evolutionary change
The organization (or team) must agree that continuous, incremental and evolutionary change is the way to make system improvements and make them stick. Sweeping changes may seem more effective but more often than not fail due to resistance and fear in the organization. The Kanban method encourages continuous small incremental and evolutionary changes to your current system.

Respect the current process, roles, responsibilities & titles
It is likely that the organization currently has some elements that work acceptably and are worth preserving. We must also seek to drive out fear in order to facilitate future change. By agreeing to respect current roles, responsibilities and job titles we eliminate initial fears. This should enable us to gain broader support for our Kanban initiative. Perhaps presenting Kanban against an alternative more sweeping approach that would lead to changes in titles, roles, responsibilities and perhaps the wholesale removal of certain positions will help individuals to realize the benefits.

Leadership at all levels
Acts of leadership at all levels in the organisation from individual contributors to senior management should be encouraged.

Concluding

As one can see, Agile Methodologies, Scrum, Lean Software Development, Kanban etc. all have very strong ties and often overlap each other. They all have in common that they are highly influenced by and are based on The Toyota Production System.

I often see development teams going “through the motions” of specific agile techniques or frameworks, without actually realizing the fundamentals that are at the core of these frameworks. True Agile /Lean software development simply does not exist without the principles behind TPS like continuous improvement (Hansei / Kaizen), eliminating waste (Muda) and empowered teams.

Why Time Is No Measure For Progress

One of the questions that Project Managers always pop up during projects is: “how many days will it take to finish this work?”.

This is a very natural and sensible question. You want to know when a project will be finished, so people can have an expectancy of the needed budget, what functionality is in it and when it will be finished.

However, answering the question is more difficult than traditionally is assumed.

An illustration

Imagine you’re expecting a friend from a distant city, to come over to your house for a visit. The city is 150 miles away.

He’s been traveling for a while and you want to know how long it will take to arrive to your house. You expected him to make the trip in three hours.

You give him a call and pop him the question. “How long will it take for you to get here?”.

Imagine him answering “I’ve been traveling for two hours”

What kind of conclusion can you get from this?

  • Maybe it would be:“great, normally it takes three hours to travel, so he should be here in one hour. Everything is going according to my plan”.
  • Another conclusion could be ‘I traveled the same distance last Saturday and it took me three hours an so he should be here in one hour.’

Hopefully however, it is obvious you cannot make any conclusions at all. You have no idea of the progress that has been made and you have no idea of the speed at which your friend is traveling. Any conclusion you’ll make based on this answer, will have an assumption in it, and probably a bad one.

It assumes that the situation you’re friend is in, is comparable to some hypothetical situation you have in your mind.

[bra_blockquote align=”right”]Basically, the only reliable data, is historical data.[/bra_blockquote]

Continuing the story, your friend eventually arrived. It took him five hours to complete the 150 miles, due to a big traffic jam just a few miles of your city. And by the way, your friend has an old car that doesn’t travel faster than 62 miles per hour.

So what could your friend have said for you so you could have the best base for a guesstimation of the ETA?

Basically, the only reliable data, is historical data.

Obviously, if your friend has never traveled the course before, there is no historical data.
His most reliable answer would then be:

“I’ve traveled 90 miles so far and my average speed has been 45 miles per hour. I cannot give any guarantee at what time I will arrive, since there might pop up something unexpected. However based on my current progress and velocity, I expect to arrive in one and a half hours.”

Notice that the elapsed time is of no relevance to make a prediction, once you know the speed and progress that has been made so far.

Bringing it home

So back to the project manager’s situation. What can we learn from this? “When will we finish?”

  • First of all, software development projects tend to be complex and unpredictable. There’s is no way you can beforehand guarantee at which point in time, you’re going to deliver x functionality for y budget.
  • Second, to measure progress, you need to know at which speed the development team is making progress. You’ll have to find a way to make that transparent. And here’s a clue:  time is in no way a measure for progress. The only measure for progress is delivered value.
  • Third, you cannot project your hypothetical situation or even your own experience with teams to the situation of other people, let alone a whole different team. Situations differ, because people differ, teams differ and organizations differ. This alone creates a unique and beforehand unpredictable dynamic. In the hypothetical assumptions that are usually the base of ungrounded guesstimations, the team is working without interference, independent of surroundings and with everything available and no unexpected things taken into account.

So how to do it?

  • To measure progress, make beforehand a rough estimation of your to be implemented functionality in story points. Story points are an indication of the amount of effort that needs to be performed to finish a piece of functionality. This is comparable to the number of miles to be traveled and has no element of time in it.
  • Next, keep track of the finished story points in time during your project. This is an indication of the team’s velocity. It doesn’t give you any guarantees about the future, but it’s the best data you can have, since it’s data extracted from the real world.
  • Third, as you go through the sprints, regularly do a re-estimation of the backlog that still needs to be done.
    This, in combination with the teams velocity, will give an indication of when your team will have finished the work.
  • Finally, don’t ever do fixed time, fixed budget, fixed functionality. No really, don’t do it. My personal preference goes to fixed time, fixed budget, flexible scope. Most of the time, people won’t remember if you’ve missed a piece of functionality. However, you will be remembered for a missed deadline.

Ignore history at your own peril.

If You Don’t Learn, You Might As Well Not Test

This is a guest post by UX designer Richard de Vries.

The most popular way of optimizing a website is by A/B testing it. For some reason the bigger the website gets, the harder it becomes to test. If you are dealing with optimization in a big company there is a fair chance you will recognize this. Putting up with rules and regulations, trying to get everyone involved and not getting people to agree upon what to test are just a few of the many things you will run into. The easiest way to solve this is to avoid working for big companies and only work for startups. However if you change your approach to testing, you might be able to create some magic, even in a big company.

The difference between a startup and a big company

According to the book The Lean startup by Eric Ries the difference between a startup and a mature company is the ability to learn. The key to success for a startup lies in how fast and how effective it can learn. For some reason, the more mature a company gets, the less effective it becomes at learning. I think this alone is the reason why bigger companies seem so stupid at times.
Why learning is so important

“On Monday Jack goes to the pub and drinks 5 whiskey and a glass of water. On Tuesday he wakes up with a huge headache. At night he goes to the pub again and drinks 5 whiskey and glass of sprite. On Wednesday he wakes up… again a huge headache, so that night he drinks 5 whiskey and a glass of coke. When Jack wakes up on Thursday with yet another hangover, he says to himself ‘I really shouldn’t drink that last drink anymore!’.”

This is a story told by Paul Hughes on the design for conversion conference. With this story Paul tried to explain the danger of testing the wrong thing in AB testing. I think this is the absolute best way to explain why you should learn from your tests.
Build – Measure – Learn

Once you have established that learning is a vital part of optimisation, you need to add this to your ‘process’. I put process in quotes because it can be anything from a strict company process to a bunch of people more or less doing things in the same order.

I’m sure you have built and measured things before (probably even through some sort of process). So basically, all we do is adding a learning phase to that process. In the lean startup build and measure does not have to differ very much from the way you already work, it’s just all agile.

Build

If you are building or designing websites you are very familiar with this phase. Build is a little bit confusing as it is both the design and development phase. What should be important to remember in this phase is that you want to build something that is measurable, not something that is perfect.

Measure

Once you have established that something built and designed to a level that can be tested you can measure it. This does not have to be an A/B test by definition. You can also measure in some (guerilla) usability testing. What is important from the object of the measure phase is that you receive much data. In my experience, the more data the better, but one should not jump to conclusions. Most of the times, conclusions that are drawn in this phase are rushed and therefore simply not true.

Learn

Conclusions in the measure phase are not true, simply because data is not a conclusion and a conclusion is not data. In order to get to a conclusion you must ask why. In fact you must ask why five times

Say we test a piece of copy: “Best value” vs. “Most purchased” and “Most purchased” won. Apart from relying on existing knowledge such as the psychology of persuasion, you can also go back to your case. Apparently your users were triggered more by the fact that most people bought this versus it being well priced. So we ask “Why are users triggered more by most purchased than best value?”

(possible) answer “Because people are more concerned about getting the right package than getting the best deal for their money?” – “Why?”

(possible) answer “Because they fear that when buying the wrong package they can no longer switch” – “Why?”

(possible) answer “Because it is not clear how to switch between packages on the conversion page” – “Why?”

(possible) answer “Because there is no text, link or graphic explaining this”

Exactly this answer is a hypothesis to test again. The smart reader here notices that we only needed 4 why’s to get to this. Excellent!

Use what you learn

Every big organisation stepped in the pitfall of not learning anything at all because they got stuck in finding a way to capture that knowledge. Wiki’s, templates, books, sessions, presentation are all great ways of sharing knowledge and they should all definitely be used in some way. However the main purpose of learning in our context is using it again. By defining a new hypothesis from our test is the ultimate way to do this.

And now it all begins

Optimizing in this cycle is very much like an engine (or any other cyclic process) the first cycle will be tough and clunky. And again much like an engine, the first cycle will be clunkier and tougher the bigger your website or company is. However once you get it started and you manage to get through the first cycle you will find that it will go smoother and smoother and you might have created your own startup magic in a big company.

 

[bra_team_member name=”Richard de Vries” position=”UX Designer” last=”no” contact_icon_0=”wp-content/themes/zigzagwp/images/socialize/socialize-twitter.png” contact_icon_url_0=”http://www.twitter.com/richie_”]Richard de Vries works as a freelance UX designer at architecto.nl and makes an idea into a website every week with 400minutes.com[/bra_team_member]