Manoj Nair's experience of adding great value as a tester

Manoj Nair is the co-founder of Weekend Testing and works for an organization that has a worldwide presence. The core business of the organization he works for, is to help people shop, online or at their stores. He functions as a Senior Tester for projects that are to be deployed in United Kingdom and Europe. 


An year ago, they decided to expand its presence in the United States of America and in the North American region. A lot of software had to be remodeled and developed to be able to deploy in USA. This was a strategic move by the organization to show its dominance in a market that their competitors had claimed to have conquered. Manoj Nair was chosen to play a role of Test Lead for the project despite being a newbie to the organization. It wasn't a lucky dip that helped the management choose Manoj to be the Test Lead for this important project but the passion and skills he demonstrated in the short period. They were convinced that bringing in Manoj would help them feel safe. Oh, by the way, he wasn't the domain expert.


Right to what they thought, Manoj ended up doing something so important for the project that literally saved the organization at least $6 million US dollars. Let me type that again, its Six Million US Dollars. If you think that was great, wait. He also prevented the company from being embarrassed in the global market and amongst its competitors.


So, what do you think? He found a bug and got it fixed that would have otherwise cost so many million dollars and embarrassment? That's what comes to mind when we talk about testers doing great work but here is a story that is different and demonstrates the great value testers like Manoj can bring in. I think he did things that most testers in that place might not have.


There's a large group of testers who have come to believe that functional testing is all what they need to do. If you don't believe that, you just might believe this - There's a large group of testers who know that other kinds of testing is equally important but they aren't skilled enough and powerful enough to convince the management and customers about it. That's where Manoj Nair differed and won.


In this new project that had to be deployed in the USA, he proposed quality criteria beyond just functionality and was seeking the time and budget approvals from the management to test for Usability, Accessibility, Reliability, Performance, Security and more. At this point, Manoj performed just like any other tester. Most of the test plan documents I saw during my first few years as a tester had all sorts of buzz words but people ended up just testing for functionality and I used to wonder why.


Manoj's test plan had an objection from someone in the senior executive team. The objection was, "About 70% of the code we are going to use is already deployed in other countries and hey its been working. Why do we need to test for all sorts of things again?" and then there could have been a chatter, "This guy is a newbie, so he probably doesn't know" and "Ah, he is relatively new to the domain".  What we are about to witness is that those domain experts were blinded by the fact that moving code from one country to another is like treating that if a software works on one machine, it would work everywhere else.


Manoj wanted to counter the objection. So, he decided to learn the impact of not doing certain things when moving code from one country to another. He found this link during his research : Target Settles Accessibility Lawsuit for $6 Million


Manoj drafted a response showing his respect to the objection and why he still continues to think that moving code from one country to another has to be treated like a fresh project in itself. He also shared with his management the link to how one of the organization had to suffer from lawsuits and a fine as big as $6 million.


The management was partially convinced but they needed more evidence before they could listen to this young chap advising them on what they should budget on. They consulted an attorney in the United States of America who was considered to be a specialist in laws of the US government pertaining to software quality and compliances. The attorney came back advising them to treat Accessibility and Usability as important as the payment and transactions through their website and did acknowledge that the information on Target was true and they had suffered $6 million and embarrassment.




Look at what happened after that. The senior management did listen to Manoj and treated this as a new project giving it the importance it needs. They came out of the bias that it was just re-use of code and started to treat this as a new application all together.


What did Manoj gain from it? As a tester he benefited a lot. He won so much credibility in the team that the next time he proposes something or reports a bug, a lot of people are eager to learn what he has to say. He has made himself influential in a short period of time in an organization. He is at least as equal as what the domain experts of the organization are. He has made people to believe that having a team with diversity is more important than having all domain experts agreeing to each other in what the other one says.


That's awesome!

Now you'd like to know that I wasn't surprised by what Manoj achieved because I know his background and education. He was constantly building skills that are important for a tester. I first met Manoj in Pune in 2008 although he had been reading my blog before that. We found a good synchronous between us. He relocated from Pune to Bangalore to work closely with me. That's when I introduced him to Parimala, Ajay, Sharath and Santhosh or should I say that I couldn't resist myself from getting these people to meet. This team craving for practice and learning set out to found Weekend Testing that today is considered as the revolution from India in the testing space. Manoj has attended my exploratory testing workshops (even the most recent ones) and is a BBST Foundations graduate. He has taught bug advocacy, exploratory testing, heuristics and oracles to wannabe and experienced testers in the past. He is well read (at least to the extent that he wouldn't feel intimidated when people are talking books) and networks with testers.

So, I am hoping that after you went through his background, it doesn't surprise that Manoj is adding great value to the project he is in. He practiced and was prepared for facing challenges even when his job in the past didn't demand all the skills he was learning. Today, he is skilled enough to handle most tough challenges in testing. He is inspiring his team members to surpass the boundaries.

If you find it tough to explain people what other kinds of value does a tester add to the project than just finding bugs, just get them to read this story. Also, if you find testers cursing themselves that they are not being treated well, get them to read this story and most importantly Manoj's background which is all about the quest for constant education and practice.

Kudos Manoj! You just did something that makes the country proud. Not just our country but people from United States of America, for the care you showed towards accessibility and usability for people with low vision and maybe other challenges.

On behalf of all testers who would love what you did, my little gift to you is this blog post. 

Are there any software testing blogs from Indians that are really Indian?

हम क्यू नहीं अपने बाषा में टस्टिंग ब्लॉग नहीं लिक्थे? में जब रशियंस की ब्लॉग दुन्द्था हूँ तब मेरे को रशियंस बाषा की टेस्टिंग ब्लोग्स मिलता हैं. वैसे ही चिनेसे या जपनेसे या गेर्मान टेस्टिंग ब्लोग्स बी हैं. अबी थो जीमेल की गेनेराल सेत्तिंग्स में त्रन्स्लितेरतिओन आप्शन एनाबले किया थो सब भाषा. ( एक इंडियन की स्तिथि ऐसे हैं देखो, मुझे कोंफुसे हो रहा हैं "भाषा" कोर्रेक्ट हैं या "भाशा")

आप इंग्लिश में टाइप करो और औतोमटिक हिंदी त्रन्स्लतिओन मिल जाता हैं. उसमे आपको इन्तेरेस्तिंग बुग्स मिलेगा. अगर आपका मत्हरू भाशा हिंदी हैं, थो आप ही इसको टेस्ट कर सकते हैं. क्या आप सोच रहे हैं की चिनेसे वाले हिंदी भाशा को टेस्ट कर सकते हैं? 

मेरे इंडियन भायों और बहनों, मत भूलो की आपकी भाशा में सोचना और लिकना भूलना नहीं. शायद आप मेरा हिंदी में थोडा गलती होगा, थो क्या? में थो आज सें हमारा हिन्दुस्तानी भाशा में टेस्टिंग ब्लॉग ज़रूर लिकने वाला हूँ.  

ನನ್ನ ಪ್ರೀತಿಯ ಕನ್ನಡ ಬಂದುಗಳಿಗೆ, ನನ್ನ ಹೆಸರು ಪ್ರದೀಪ್ ಸೌಂದರ ರಾಜನ್ ಅಂದ ಮಾತ್ರಕ್ಕೆ ನಾನು ತಮಿಳಿಯನ್ ಅಲ್ಲ. ನಂಗೆ ಇಂಡಿಯಾ ಮುಖ್ಯ ಆದರೂ ನನ್ನ ಮಾತೃ ಭಾಷೆ ಕನ್ನಡ. ನಂಗೆ ಕನ್ನಡದಲ್ಲಿ ಮಾತನಾಡುವುದು ಬಹಳ ಇಷ್ಟ. ಆದರು ನೋಡಿ, ಇಂಗ್ಲಿಷ್ ನಲ್ಲಿ ಬರದು ಬರದು ನನ್ನ ಮಾತೃ ಭಾಷೇನೆ ಮರತ್ ಬಿಟ್ಟೆ. ಸಕ್ಕತ್ ಅವಮಾನ ಅಗುತ್ತೆ ನನಗೆ.  ಆದ್ರೆ ಇನ್ ಮುಂದೆ ನಿಮ್ಮ ಪ್ರದೀಪ್ ಕನ್ನಡದಲ್ಲಿ ಟೆಸ್ಟಿಂಗ್ ಪೋಸ್ಟ್ ಬರಿತಾನೆ. ಹಾಗೆ ನಿಮ್ಮನ್ನು ಕನ್ನಡದಲ್ಲಿ ಟೆಸ್ಟಿಂಗ್ ಬಗ್ಗೆ ಬರ್ಯೋಕೆ ಕೇಳುತಾ ಇದೀನಿ.

ಇನ್ನು ಕೆಲವೇ ವರ್ಷದಲ್ಲಿ, ನಾನು ಬಹಳ ಚೆನ್ನಾಗಿ ಕನ್ನಡದಲ್ಲಿ ಟೆಸ್ಟಿಂಗ್ ಪೋಸ್ಟನ್ನು ಬರಿತೀನಿ. ನೀವು ದಯವಿಟ್ಟು ಟ್ರೈ ಮಾಡಿ. ತಪ್ಪಾಗಿದ್ರೆ ಪರವಾಗಿಲ್ಲ ರಿ, ಬರಿಯಕ್ಕೆ  ಶುರು ಮಡುದ್ವಲ್ಲ ಅದೇ ಸಾಕು. ಬನ್ನಿ ನಮ್ಮ ಭಾಷಯಲ್ಲಿ ಟೆಸ್ಟಿಂಗ್ ಪೋಸ್ಟ್ ಬರಿಯೋಣ.

என் இனிய தமிழ் மக்களே, உங்கள் பாசத்துக்குரிய  பிரதீப் சௌந்தரராஜன் பேசுகிறேன் :) நான் பிறந்தது  ஒரு மாதவா கன்னட குடும்பத்தில். படித்ததெல்லாம் பெங்களூர் பள்ளி கூடத்தில். கன்னடம் தான்  படித்தேன் ஆனாலும் தமிழ் மீது ஒரு பற்று உள்ளது. பொறியியல் படித்து திருச்சியில் உள்ள மூகாம்பிகை கலூரியில். தமிழ் அப்பொழுது தான்  கற்றேன். சன் டிவி பார்த்து, சொல்லுவதை  கேட்டு  கேட்டு  படிக்க கற்றேன். இரண்டு சுழி "ண" மற்றும் ஒரு சுழி "ன" எங்கு  வரவேண்டும் என்று இன்னும் தெரியாது ஆனாலும், தமிழில் நான்  டெஸ்டிங் ப்ளாக் போஸ்ட் எழுதுவேன் . 

அது தமிழில் உள்ள பற்றுக்காவும் மற்றும் உங்க மாதிரி தமிழை நன்றாக கற்ற ஆட்களை தமிழில் டெஸ்டிங் ப்ளாக் எழுத ஊக்குவிப்பதற்காக. தமிழில் டெஸ்டிங் ப்ளாக் எழுதுங்க, தமிழையும், தமிழையே நம்பி இருக்கும் இளைஞர் பட்டாளத்துக்கு  வழி காட்டியாக இருங்க.  வாழ்க வளமுடன் .

For those who don't understand any of the above written languages (Hindi, Kannada & Tamil) and you want to know the meaning, just read the title and you'd get it.

I am asking, why aren't Indian testers writing blog posts in their own languages? I see people from Germany blogging about testing in German. I have come across Russian blogs ( Take both Alexei's for example), I also saw a few Chinese & Japanese testing blogs. Not to forget, Polish, Dutch and even more.

I haven't come across one testing blog that's truly Indian in the language. I have decided to occasionally try writing in the languages I know. Its shame but true that I have forgotten how to write well in Indian languages. I learnt Sanskrit, Hindi, Kannada, Tamil and a little bit of Telugu but can't write in any of them as fluent as I can do in English.  

If someone can write in any of the Indian languages and about testing, it would truly be the Indian testing blog. What I have been writing so far is an Indian's testing blog written in a foreign language. See the difference?

Who is making software testers, dumb and bad?

Not so long ago, I thought there existed a set of testers called, "bad testers". I hated them. I wanted to punch them on their face and get their face to bleed. I wanted to become a powerful politician and kill them all and escape without being charged for genocide. I wanted to become a superhero and get people to fire them from their jobs. I wanted them to beg for jobs, money and survival. I thought that is the way to get them to open their minds for learning. All this should have shot my blood pressure up while those bad testers remained cool. They were untouched by my criticism and continued to think that I was an asshole.


Whenever I found them, I insulted them as much as I could till I realized that they needed more care from me than those whom I was already caring about. I started caring for them. My world changed and so did theirs.


I was always wondering how these bad testers are happy. Needless to say I thought I am a great tester and still continue to think that way. By "great", I mean, "just what is required". Today, you can be a great tester by being just what is required. Tomorrow, the case might change.


I tried shifting the question from "Why are bad testers happy about themselves?" to "Who is making these bad testers happy?" and "Who is preventing the bad testers to learn that they are doing bad testing?"

That's when I said to myself, "There are no bad testers. There are some who are forced to practice bad testing. The force is either internal or external or a combination of them".

That was an important shift in the strategy that helped me in my exploration of identifying what factors cause a tester to appear bad or practice bad testing.


Internal forces 
I mean, ones that the testers themselves are responsible for or have control over.

• Money more important than anything else: For some testers who are sole breadwinners of the family, they might internalize the idea that what works for others is a safer route to traverse than exploring new paths and risking their cash flow. They spend their life traveling those peoples route who themselves have followed someone else's route. All finding it to be safe and hence not wanting to change.

• Fear of losing the job: For some testers, losing a job means unbearable social pressure. These testers don't ever try to speak against anything to protect their jobs. Their whole life is spent on running just one test case - Is this the right time to shut my mouth? - to which the result always remains - Pass.

• Shallow ambitions in life: For some testers, their ambition is to never do something fascinating but just run the rat race, build a house, buy a car, get married & have kids. They also try to ensure that their kids continue to run the rat race. I am not speaking against taking care of the family but taking care of the family should be balanced with building high ambitions in life and working towards it.

• Victim of Rutherford-Bohr's experiment: Some testers, no matter what exciting stuff they are presented with, try to return to their most stable state of ignoring all the exciting stuff because their life is already happy (grounded). 

• Living someone else's dream: Some testers, don't have dreams of their own. They just pretend to have their own while they are living other's dream. Some live the dream of their parents and rest their manager's. Living others dream makes their life boring and they give up on almost everything, forget testing.

• Taste of early success causing a drift from continuing to learn - Mostly a very dangerous one. These kind of testers think they are on the right path and there is no reason for them to change. 

• Having learned that good testing is hard - Some testers acknowledge what good testing is but they also learn it is very hard to test well. Out of that, some of them make up their mind saying they are not in for such hard work because they think life is bigger than doing good testing. Nothing wrong but they don't seem to be doing to the big part well, either.

External forces
I mean, the ones who are responsible or has a power or influence to get good testing done.

• Head or Tails of testing - I have talked to at least slightly less than a hundred Heads of Testing of big, medium and small scale organizations. They have so much power to change things and yet they don't seem to be doing anything about it. I must also admit that some people are doing very well while most don't appear to be. Why don't these people take a break from their work, sit along testers on one of the project and test for just a couple of days to realize how hard it is and what can they do to help these testers do a great job. 

• The interviewers - At least people in India, when they are out of college, want to just learn enough to crack an interview. When interviewers emphasize on demonstration of memorization than skills, its easy for a billion plus population to crack them. Fakers get in, Genuine people might not.

• Testing institutes - Business demands scale, I agree. Scaling at the cost of quality of education is in my opinion, spoiling your own country's chances. Please read the book Outliers by Malcom Gladwell and more specifically Chapter Five - The Three Lessons from Joe Flom. You'd know what your business needs for future if it has to remain scalable.

• The experts - If you have great ideas, please price them a little lower for the first few years or based on the geography. You won't be considered cheap, trust me. Don't make money an entry barrier to someone who wants to get excellent at testing. 

• Commercial conferences - If you have have had good deals of sponsorship and paid delegates for a specific year, consider giving 80% discount to 10 people who cant afford it but want to attend it.

Combination of internal & external forces

When the external forces & internal forces combine, its a killer combo for bad testing

• Lack of speed in firing poor performers - If the people responsible to get good testing done are delaying in firing poor performers then the hope in the poor performer rises that he or she is doing well and should continue doing that. In at least half the organizations I consult, I get the opportunity to consult because they haven't fired the poor performers for a long time and something went kaput.

• Not paying good testers well - I have been to a few conferences in India where Head of IT or Head of Dev or Head of Testing are keynote speakers. Their speech is usually, "We have come to realize that testing is of great importance" but then they don't match the pay of the good testers they have to their claims. People call that "keynote". Can you walk the talk?

• Waiting till the year end to spend on training budget - Wondering why many organizations keep their training budget till the year end and not organize a training when the team needs it sometime mid year? As a side note, I wish, in India, the Learning & Development department, which is a separate entity in the organization is eliminated and every department becomes Learning & Development apart from what they do. 

• The book writers - When you write books that are not different from any other books that are available, you are re-iterating the point that the industry isn't changing. Many testers who accidentally pick up a book and skim through it read stuff that they have read a couple of years ago feel they are on track (and also end up not buying your book). Is that a message your book wanted to communicate?

I am 30 now. I am more curious about my age of 50 and waiting to get there, because I hope, I would have seen many changes - lots of positive ones. Mostly because the generation to which I belong or the generations junior to that of mine would have solved the problems I have listed and might have gone beyond that. I am not discarding the fact that the older generations have not solved it. There are dozens of them out of a population of millions.

When I tried punching just one bad tester I met, blood oozed out. Not on the face but in my hands for it was a mirror that I saw. 

Help Chandru to live his testing dreams

I give myself opportunities to meet many different software testers from Bangalore or to the places I travel. One of the ways that's been successful for me is "Coffee with Pradeep". This has brought many testers to my lives who have had a good influence on me. Sharath Byregowda contacted me for a coffee and then we work closely. He is one of the good testers and thinkers I have met. He recently moved to UK and I am already missing him so much.

Meeting Chandrashekar 

On February 10th, this year, Chandrashekar B.N (Chandru) got in touch with me and asked if I would come to meet him over a coffee. I instantly agreed because his emails were showing the passion he had to test. He seemed to give importance to improving his skills. I met him around the third week of March and this guy was silent, taking notes. He and his friend Sunil had come to meet me. Both these guys were mostly silent but took notes of our conversation. Wouldn't be wrong if I say, I liked their seriousness.

When Chandru spoke

At last Chandru spoke. He narrated his story and it was an emotional moment for me. His father died while he was young. His mother, with two kids (Chandru and his younger brother) struggled very hard to get them a basic education. Chandru didn't grow up with any luxury. They lived in a small room for the last 20 years. Chandru got a job as a software tester after his Bachelors and was running test cases. He wasn't happy. He learnt that he has a passion to test and wanted to get skilled at it. He was earning just enough to get 3 meals a day for 3 of his family.

In his situation, someone would have said, "I want to get more money. How do I do that?" but instead he said, "I want to test better. How do I do that?". That was an amazing moment for me. I shall continue to be in India just to meet testers like Chandru who amidst several troubles in life want to test better.

I started to coach him. I got him to network with testers in Bangalore who are as serious as him. He came to Bangalore Workshop on Software Testing. He was enjoying this new found life. His friend Sunil was no less passionate. Both of them didn't stop there. They made a list of books they ought to be reading and went on a book hunt in Bangalore. Chandru quit his job and took up another job with no hike in pay, just to gain more freedom in testing.

When God decided to test Chandru


About 40 days back, I received a call from Sunil and I was shocked to learn that Chandru was diagnosed for Blood Cancer. To be specific it was Acute Lymphoblastic Leukemia - with Ph+ve - B cell - Blood Cancer. I was in Gurgaon when I got the call. I tried calling his mobile but couldn't speak to him as he was undergoing some tests. That night, I just couldn't sleep. I felt too bad for Chandru. Just when he was about to rise, he got this bloody cancer for no fault of his. I got up in the morning with the little sleep I had and then said to myself, "Should the people who care for him lose hope. Is that the way I should be? Not at all."

The next day, I was fortunate to be able to speak to him. He knew he had cancer. I didn't know how he would talk to me. I didn't allow him to speak much. I was shouting at the top of my voice, "You are a warrior and you will fight this out. You have a dream to be a good tester and you are not letting down yourself or others. That's the only thing that should be running in your mind"

When Chandru decided to take up the challenge


A few days back, I returned from Gurgaon and directly went to to the hospital. The confidence, the smile, the charm that Chandru had when he met me is one of the greatest things I have seen.

Today


He is fighting well. Doctors say he is doing well, except to the part of the bone marrow which he doesn't have a control over. His confidence is amazing. Its one of the most inspiring things for me. I love this guy and he is my hero. I have started to strongly believe that the mind can control the way body accepts and reacts to treatment. He makes me think of Andy Dufrense of Shawshank Redemption.

Where do you come in this story?


Chandru is the only earning member of his family. He being hospitalized and his medical insurance money long depleted, he has no source of income for his treatment. With his treatment cost estimated about 20 lakhs for Chemotherapy / 50 lakhs if bone marrow transplant has to be done to cure him, he has ran out of options.
So what, he has the testing community to help him. I hope I was right in making the statement.

Help, So far

• Parimala has donated 25,000 INR (I am so proud of her)
• Weekend Testing funds of 12,000 INR has been donated
• Mohan Panguluri has given a standing instruction to the bank to transfer a certain amount every month from his income to help Chandru other than sending a mailer to everybody in Test Republic.
• Couple of people have tweeted seeking support to help Chandru.
• Chandru's office colleagues have donated a part of their salary to Chandru
• Some testers who saw the mailer of Test Republic have donated a few thousands.
• Ajoy has featured this in his Testing Circus magazine
• STC 2010 conference has confirmed that they would be announcing it during the conference and make more testers aware of an opportunity to save a testers life.
• Dhanasekar S has come forward to help Chandru. 
• I am trying to do my bit to such a wonderful man who is brimming with confidence when others might have almost given up. Letting him down would be our biggest sin.
• Rahul Verma and I are going to be doing fund raising public workshops and donate the fund to Chandru's treatment. If you are from Bangalore / Chennai / Hyderabad interested to attend our workshops whose funds collected would be given to Chandru's treatment email me at my mail id which is pretty public.

Yet, we have fallen short of funds for the bone marrow transplant. So, I kindly request you to come forward and help Chandru. The best part is - he said he is going to treat every rupee that has flown in as an interest free loan and says, "I will get well, continue to work as a tester and pay back all money". For such a guy, I think, you should.

Go team up with all testers you know, talk about how you can help and get it going. The time is now.

Important links & details

Help Chandru Website : http://helpchandru.com ( Website not fully developed )

His reports: http://helpchandru.com/files/Chandru_Report_Detailed.pdf

Cancer Patients Aid Donation Page: https://donations.cpaaindia.org/?projects=For%20Chandrashekhar or http://www.cpaaindia.org/casefile/index.htm#chandrashekhar
Note: Income Tax concession for this mode of transfer / donation
Important note: While donating through CPAA website, please mention that the donation is for Chandrashekar BN in the projects section

Those of you who want to do a wire transfer from India to his bank account can:
Note: No Income Tax concession on this mode of transfer

Name:Chandrashekhar B N

Account Number:218010015960

Branch:Koramangala, Bangalore

Bank:ING Vysya Bank

IFSC Code No:VYSA0002180

Chandru's Paypal account email id: daysofchandru@gmail.com
Note: Income Tax concession subjected to approval in this mode. Approval pending 

Every little help you do, matters a lot. When you donate, please email sunilkumar56@gmail.com / hariprasad.email@gmail.com and let him know the details, so we could confirm on receiving the same. If you are in Bangalore and have O+ve blood and is willing to donate, please get in touch with Sunil or Hari whose numbers you can find in the website http://helpchandru.com

Please,  Help Chandru

Tour of being an independent test consultant

Hiya! Hope you are doing very well. I am Pradeep Soundararajan, your tour guide for the next few minutes. I am glad you chose to take this tour of being an independent test consultant.





Here are some questions you might have: How does it feel to be an independent test consultant? What is it like to be one such in India? Will I be able to survive? Will I make enough money to run my family? Will I make as much money as the organization I am employed is paying me? Will I get enough paid work? What if I don't get paid work for a long time? Will people want my kind of skills? How do I know someone needs a consultant? How do I get clients? How will my family take this? What do I explain to my spouse? What kind of a pressure does society add when I am not in any paid work for a while? How does it feel to work from home not just for a day but for an entire month, or maybe a year or more? Will I have enough money to pay my home loan EMI?



These are some common questions that have popped up from those who have wanted to take the tour. So, if you have these questions or maybe even more, you won't be disappointed with this tour.




Some people ask me, "Hey, can you give me a few clients of yours and help a fellow Indian to also be an independent consultant?" I want to help people be independent consultants in India. By that, of course, I mean, I'd like to see them do stuff that helps them get credibility, reputation, paid work and clients for themselves.



I'd like to take you through the journey of having been an independent consultant. A journey that is not so often written or spoken about



Tour Point 1: Knowing enough about enough



If you want to be an independent test consultant, there are some prerequisites that you need to fulfill. You need to be bold enough, skilled enough, curious enough, pleasing enough and willing to talk to people or do some work good enough to get good enough people to talk to you.



That's a magical formula right? No one knows what "enough" means and hence it is a problem and an opportunity in disguise. If you knew what "enough" meant, you are kinda through to anything you want to achieve. I think, not knowing how much is "enough" makes you to work hard and get close enough. Oops, close enough?



I have been able to survive so far. I have no clue if my current skills are enough for me to survive for the next year and I am on a constant upgrade of skills and knowledge. I invest money on learning and my investment for July 2010 is on a few books, Ethical Hacking Guide to Corporate Security by Ankit Fadia & Job Interviews - Walter Vierera. Time is a much more important investment than money for me. So, just by spending money on those books wouldn't mean much unless I make a further investment of time on it.



Tour point 2: Love for failures



Many projects today suffer because people working on it aren't willing to try new ideas. They are special people on earth who know things would fail even before trying them out. However, as a consultant, if you try to be like them, you'd be expensive for your clients. You would do what their employees are doing for a price much higher than the employee cost.



Its important to not fail at a client's location but should that stop me and you from loving failures? I have a lab where I can experiment ideas whose results I don't know yet and that lab is the world of my colleagues, community and my gurus.



It's OK to fail, once in a while, at a client's location because even if you do great stuff, there could be things beyond your control that might make it look very bad. However, if you can get that client to call you back for more paid work in future, it boosts your confidence a great deal.



It happened to me. I was black listed in an organization and now they not just white listed me but want to work with me closer. Their CEO is in direct touch with me. If I feared failing, I would have done more mediocre stuff than what they thought I actually did.



Tour point 3: Excellence instead of money



India is a great place for some inspiring movies. I strongly recommend that you watch the movie "3 idiots". No, don't Google and read the story, just watch it. There are several good messages in it and one of them is, "Excellence instead of money". This movie is a super duper hit in India. I just wish people not just like such movies but also bring in necessary changes to their lives.



I want to be rich but I want to be rich while I am excellent. I wouldn't mind money coming on my way but money wouldn't necessarily make me feel rich. I want to be rich in testing skills and knowledge. I want to be rich in knowing many testers and how they work. In the process, if money follows, I am super happy.



Tour point 4: Don't expect people around you to understand what you are trying to do



When an article about me appeared on a few national news papers, my article was published on a few magazines, I was interviewed by CNBC TV18, I was on news for a local TV channel, my parents were so proud of me that I can bet they were flying high. However, whenever they see me sitting in home for more than a week without any paid work, they start to ask me, "Why don't you join some company like Infosys?"



If you expect your parents or spouse to completely understand what independent test consulting means then you'd be inviting disappointment. I was expecting them to understand what I was trying to do but experience teaches that I shouldn't. This has nothing to do with the respect we have for them but a learning of what we can't help them understand.



While at home, I am glued to the computer, trying to learn something, practice testing or support my clients post my onsite engagement or reply to emails. Some people around me take it for granted that I am jobless. Someone calls me and say, "Hey, you are at home only na, so why don't you come pick our luggage and keep it there?" It irritates a lot. I am at home but not jobless. I am trying to generate a paid work, which is a part of my work. People don't understand that. So, be ready for all that.



Tour point 5: No promotions and no designation change



If you were used to being an employee for long and then chose to be a consultant, you must know that there is no one who is going to give you a promotion. It is what you call yourself that matters. I am calling myself a Consulting Tester or just an Independent Test Consultant. If I am bored of it in 2012, I may call myself a Senior Consulting Tester. I give myself fancy title sometimes. I was calling myself a Test Magician and then I am now calling myself a Brainual Tester.




Tour point 6: Being an independent consultant doesn't mean you are the expert



Without saying much, I am not an expert and I am an independent consultant. James Bach, Michael Bolton, Elizabeth Hendrickson, Jerry Weinberg, Scott Barber, Matt Heusser, Jonathan Kohl, Karen Johnson are experts who are independent consultants. People like Ben Simo, Jon Bach, Cem Kaner, Vipul Kocher, Ashok are experts who are employees. There are some good thinkers and future experts like Meeta Prakash, Parimala, Lanette Creamer, Ajay, Santhosh Tuppad, Sajjadul Hakim, Ramit Manohar, Sharath Byregowda, Shmuel Gershon, Issi Hassan, Markus Gartner, who are employees, too.



So, independent consultants don't necessarily mean an expert. Employee don't necessarily mean a non expert. If I had to be an independent consultant only after becoming an expert, I wouldn't have been one till now. If I don't be an independent consultant, I don't know if I would ever get close enough to an expert while being an employee.



So, if you are waiting to be an expert and then be an independent consultant because you thought there was a strong relationship between them, you could be wrong.



Tour point 7: Tackling loneliness



Even in 2nd most populous country in the world, there are a lot of people I have seen who feel lonely. So, loneliness is not about people not being around you but about people whom you want to be around you not being around you. For me, my first wife is my laptop, just like many others I guess. So, despite having two wives (laptop and the one to whom I am married), I get lots of situations where I feel lonely. Loneliness is not always a problem; its a blessing in disguise. Ask our fellow bloggers, they'd tell you that they churned out a cool post during such situations.



However, being an independent consultant and working from home means, I have no colleagues that I meet on a daily basis. I meet a lot of new people every year but meet the same people very few times. Having no colleagues to meet on a daily basis means frustration at times.



When I go through Facebook or Orkut and see some people posting photos of their team member's birthday party celebrations, team outing to a hill station, team lunch, going to movie as a team... it hurts me a lot. I just take it as though I am in a penance of becoming a good tester and I have to bear with all of it. Recently, I was pissed off when I found no one to join me for a movie that I wanted to go. Even if I did find, their timing and my timing was off. Hey, employees are pissed off too. So, I am still fine.



Tour point 8: Getting clients is like sowing seeds and waiting for them to sprout



We think clients come from a specific place or set of places and we are wrong. When I was a rookie consultant looking for opportunities, Michael Bolton, told me that finding business is like sowing seeds. He also told; you never know when the seeds sprout.



I sowed a seed by answering a question in a forum without knowing I was sowing it. The one who asked the question was impressed and help the seed sprout by giving me a business worth thousand dollars.



When I didn't have a public reputation for speaking, I offered 2 hour talk called "Mirchi Test Masala" for free. This attracted at least seven organizations to have my talk at their office. I was gaining experience of speaking to testers and engaging them for at least 2 hours. I was also testing my own testing exercise on them. This was in 2007. One among the 250 people I might have spoken to during my Mirchi Test Masala tour was impressed with my talk. In 2010, he asked his current employer to have my workshop for his team. It happened. As you see the seed sowed in 2007 sprouted in 2010.



So, if you want to be an independent consultant next year, you should have started sowing at least 2 years back unless clients are just waiting for you to becoming a consultant. Also note that there are other consultants sowing seeds in the same place. If you are skilled enough you don't need to be too bothered.



Tour point 9: Freedom at its best



If you are my blog reader then, "Some birds aren't meant to be caged, their feathers are just too bright" isn't a new thing to you. If it is, please note that the quote is from the movie Shawshank Redemption. I struggled for freedom to test. I saw that I could test a lot better when there is freedom. I also see that many testers can perform a lot better with a lot of freedom and responsibility bestowed on them. The only way I could win freedom to test in India was by being an independent consultant. These days, I am coming across a few managers who want to provide freedom to test but they don't have the curious people in place.



Tour point 10: Managing the tough situation of low bank balance



During August 2009 to Dec 2009 there was no paid work. There were enquiries but nothing turning to work. My bank account went low enough that the bank thought it was expensive to send me an SMS about my bank balance. At that moment, everyone around me urged me to look for a job. It isn't a bad thing to go back being an employee. At least I won freedom for a while and I can always win it back. I updated my profile and started to apply for jobs. I didn't even get a single interview opportunity in Bangalore. What? Yeah, so, no matter how much reputed you are, nature teaches you humility at such times. I dropped my plan to look for a job and look for some consulting opportunities. Again, a seed sowed in 2008 sprouted and I found 2 back to back consulting opportunities. Today, as I write, I have certain things lined up, clients asking and competing for my dates. I have no visibility beyond a certain month in this year. I have no problem with that. I am just hoping that some seeds will sprout.



Tour point 11: Live your dream



3 weeks back, my school classmate died in an accident. It just reminded me that death is not certain of a time or situation. Any day it might strike. I was shocked by his death and asked myself; what if I die today? Also, if you have listened to the famous speech of Steve Jobs, he talks about death and suggests that we work as though today is our last day.



Living your dream is very important. So important that you may not have another life to live your dream. My dream is to be a good tester and if possible help others with similar dreams to get there. I know I am living my dream today, at least to some good extent, so I am not afraid of death.



Tour point 12: Earning money is tough but not impossible



Some of my own students hesitate to coach other testers because they think teaching isn't something they want to do. Indirectly they are saying, "This demands me to learn new skills which I am not prepared to". There is money in teaching. If they continue to think that way, they would never be able to earn contacts, different learning, upgrade their skills, learn about different contexts, challenges and hence more money.



Tour point 13: Consulting is not only problem solving but jiggling the right things



I was thinking of myself as a super hero by having solved some problems for my clients. When I interacted with Michael Bolton of what I did to claim success, he helped me understand that I jiggled with things rather than solving problems. I started to learn what exactly I was doing. Although I believed I was solving problems for my clients, it was they who were solving it and I was just helping them do it. At times I did solve problems, too.



If you are serious about being a consultant or being a valuable employee and want to jiggle the right things, I strongly recommend you to read Jerry Weinberg's books on Consulting. I mean, Secrets of Consulting and More secrets of Consulting. They are my best reads.



Tour point 14: Get yourself a guru and a role model



Sachin Tendulkar who is considered to be a God for cricket in India has a guru. Suriya, a rocking Tamil actor has Kamal Hassan has his guru and inspiration. Kamal Hassan has more than just one guru. Rajinikanth has a lot of gurus and role models. Virendar Shewag has a role model (Sachin) that he wants to be like. The great Arjuna of Mahabharatha had a guru teach him archery. Ekalavya had a guru. All these successful people have a guru and a role model. In testing, as we think we are experts or good enough, we don't feel a need to have a guru and role model. We sink our own career that way. I must have avoided thousands of traps by having more than a few gurus.



James Bach helped me be an independent consultant and I often consulted Michael Bolton to shape myself. There are other people who have helped me too and have been my inspiration and role models. I have posters of these people on my wall. It keeps pushing me. Without gurus and role models, I would have been lost by now.



Your tour ends here unless you have a few questions to ask me. Thank you for your patience.

Experience Report : BWST 2

 Experience Report - Bangalore Workshop on Software Testing - 2

3rd April, 2010 @ Shilton Royal, Bangalore

Event Sponsor: Vipul Kocher, President ITB

Photos (and camera) credits: Rahul Mirakhur

 
Bangalore Workshop on Software Testing stepped into its 2nd year on April 3rd, 2010. Why wouldn't it?
This time, it was much more organized than BWST 1. Of course, we learn some lessons, if not all, when we do something.

The groundwork required announcing it on my blog, managing the registrations, speaker invites, cancellations, getting sponsorship, managing the budget, conference and hotel booking. As we had Selim Mia from Bangladesh as a participant, the work also involved helping Selim getting a Visa to India. Santhosh and Parimala were of good help for me in the ground work.

So, the D-day arrived. Some new faces and some old started dropping a couple of minutes earlier to the start time published to them. What? An Indian conference happening on time? That's crazy ain't it? So, till the time arrived some participants started to discuss about Left, Right, Neutral of Politics, Democracy, India and more.

We started off at 9 AM with a check in & self introduction from all participants and Parimala explaining the rules of the day and how to use K cards. She announced herself as Kaali Maata for the day who would facilitate the workshop.

Sharath Byregowda's talk on "How he cleared the trap that prevented him from attending BWST2"

What better way to start than to get a guy who was not likely to attend BWST2 because he had to be in office for someone's estimation being the biggest ever blunder committed. Sharath talked how he cleared the trap to be able to attend BWST2. While he was talking, lots of green cards started popping up. He explained how bugs (that were show stoppers) allowed the test team to buy some time while the development team were fixing them. 

Rahul Verma raised a question, "How can you go and look for show stoppers? Isn't it only when you find a bug you'd know if it is a show stopper?". That triggered some red, blue and green cards to show up. The discussion shifted from that to Severity, Priority and flowed into certifications. Sharath talked about the experience he had in proposing BBST course in his organization.


Ashok's talk on Metrics

So, Ashok T, CEO of Stag Software brought in his wide experience of dealing with metrics and questioned what metrics made sense. He talked about metrics used in various stages of software development and testing and shared his experiences of working with several clients on them. He talked about having helped his clients understand that collecting all these metrics is good but do we really know why we are doing it and what is our goal?


Discussions evolved around metrics and estimation. Not in anyone's experience there, an estimation had gone right. Rahul Mirakhur had to say, "If you get the estimate right, then its not right". There appeared to be a lot of interest from most people to try to get estimation right. I don't think anyone should try getting their estimation right because that is not the only thing that would be helpful to the clients they work for.

Selim Mia

We had a participant and speaker from Bangladesh, Selim Mia. Such passion is great to see and experience. He traveled by train from Bangladesh to Bangalore and back. The moment he stood up, everyone asked him a question unanimously : Please tell us how is testing done in Bangladesh? He had to answer that before he could proceed on his planned talk. I personally think there is lot of potential in Bangladesh but I also hope they avoid falling to traps that we have fallen into.

Selim had a combo for us : An experience report and sought answers to questions he had as a test manager. Based on what he said it occurred to me that the senior management there at Bangladesh were in for results. So, sometimes they gave enough freedom to testers to achieve results - which is good.

Discussions returned to estimation, metrics, scripted and exploratory testing. Vipul had been to Bangladesh, so he shared his experience of interacting with people there and termed it "pleasant". Two names that Indians now know from Bangladesh is "Sajjadul Hakim" and "Selim Mia". I think its time we know more.

Lunch

BWST 1 taught us a few lessons. We went out for lunch and in search of a hotel and spent about 2 hours in BWST 1 on it. This time we organized it in the same hotel and hence we saved a lot of time. We spent munching a good lunch for 50 minutes and then got back to action. Good lunch, I liked the soup & noodles!


Rahul Verma

I was invited to a conference in Germany and I couldn't make it. Instead of me, Rahul Verma did and I think it was a good thing to have happened for the conference. I believe Rahul Verma can deliver the blows that audience needs and that is exactly what he did. His presentation was an exploration into what crap, trap and good means to you versus to those whom you report to or to who reports to you. He delivered some punches to bloggers like me, which I welcome :)

He shared a few experience reports from his office and then led to asking us if we do have an opinion about what we say and how did we arrive at it? He left with a message saying, "Evaluate"

Those who probably did not know Rahul Verma were probably pleasantly shocked by his presentation and I argued over having an opinion and not having an opinion. I feel opinions are in making. Not expressing opinions could be dangerous in some situations while expressing it could be dangerous in a few others.


Vipul Kocher

Testers break rules. Good testers know when it is safe to break rules. So did Vipul. He cut the crap about  crap in the theme and focused on helping us learn the Noun and Verb technique of generating test ideas. The first thing that strikes is his acknowledgment to Elizabeth Hendrickson. It is a strong message to the testing community to owe credits to someone who has helped an idea to be helpful to a larger mass of testing community. Although "Noun and Verb" technique was unheard to many, they did enjoy learning it from someone who had implemented it in the past and shall continue to work on it. I usually don't run out of test ideas and I think through his presentation, the likeliness of me running out of test ideas has further reduced.


Ashok had questions to ask about Noun and Verb technique and I found them interesting. I too have plenty of questions about it but I am going to be working on it for a while before I try to find out answers. There were a few junior level testers who were at BWST this time such as Sai Divya & Shwetha Ghorpade who acknowledged that it will be interesting to implement the technique at their work.

Meeta Prakash


Meeta hit the bell. She took the word crap from the theme and thought about one of the crappiest thing at work - meeting. Her session didnt need any facilitation. She kept polling the audience and questioning things about meeting. It was interesting to know every person except Allmas (lucky Indian) thought there was a lot of time being wasted in meeting. Sometimes people not needed are pulled into a meeting or other times people who are most needed in the meeting are left out because they speak truth. I got reminded of a meeting money burn meter that I saw sometime back which shows how many dollars are burnt in the meeting.

The discussion was interesting. I was reminded of one my ex-manager whose meeting only gets over when his wife calls him. Those who reported to him felt his wife was a savior and a viking.People brought in their experiences of craps and traps of meeting. I once worked for a CMM Level 3 company who was trying hard to achieve CMM Level 5 status. An SEPG team in there ( don't know SEPG : Software Engineering Process Group ) wasted a lot of our testing time. SEPG team ensures that a spelling mistake bug is as expensive as a database corruption by involving the whole team to do a root cause analysis of how the spelling mistake bug went unnoticed. Vasu brought in his experience of how he learned to handle customers diligently  by observing his manager handle it in a meeting. Allmas is way too lucky. She appears to have a team that handles meeting so well. Don't envy her, she is going to be as unlucky as you when she moves out of the organization.

Sukanta Bhatt

Oh this man! The stories he shared on testing medical devices got people doing two things - laughing while thinking. People didn't seem to want him stop. He shared experiences that led to discover new things and realize the value of interacting with customers and being at the customers place watching them how they use the product we develop and test. Was a cool way to end BWST 2 presentations.

Jantha (participants) got curious about medical applications and devices testing space and started to pound him with questions and asking more experience reports on medical devices. Well, that happened over a beer.

Checkout

So, when the bell rang at 5:30 PM, we officially concluded BWST 2. Before we did that, we had a checkout in which each person given less than a minute were asked to talk about one take-away from the whole day. Rahul Verma and Sharath believe that it was cruel of me to ask for "one" takeaway while there were lots. Not that others had just one but these people voiced their opinions about it. The flip side is if someone who is new to this concept of BWST and had been all silent, is easy for them to talk about one take way unlike others who are practiced to speaking a lot.

Ashok & Vipul

The audience weren't exhausted. So, they ignored my call for heading to a pub nearby and asked Ashok and Vipul to talk about experience of running a testing services organization. Questions were posed about China, US, Europe and ANZ regions. The thrust was how do we as Indian testers do better.

Photoshoot

So, we all got shot at the end by one of the hotel staff :)


Beer in Pub

We decided to catch up in Enigma, the Pub @ Koramangala and testers rocked the place. A couple of pitchers and soft drink went in. Girls who had been to BWST 2 also accompanied us to a pub as they knew they are hanging out with some of the gentlemen of the industry. That's it.


Note of thanks

To all participants : Santhosh Tuppad, Sharath Byregowda, Selim Mia, Ashok T, Rahul Verma, Vipul Kocher, Dr.Meeta Prakash, Sukantha Bhat, Swetha Ghorpade, Senthilnathan, Allmas Mullah, Eshwar Kumar, Lakshmi Narasimha, Gokul, Dhanasekar S, Rayanagouda Patil, Vasu Swaminathan, Rahul Mirakhur, Mandeep Singh, Ajay Balamurugadas, Ravisuriya, Yeshwanth Rao, Sai Divya, Chandrasekha, Parimala

To our dear event sponsor: Vipul Kocher
To my co-organizer, Santhosh Tuppad
To the facilitator, Parimala
To you, for reading this.


I shall see at least some of you at BWST 3!

Follow your own style

 This post is India specific.

I have a reputation and credibility in this industry to an extent that some people inside and outside of the country have at least heard my name, if not for my skills. I have a brand of my own. I have a style of my own. I call myself with cool names. I sound like a geek. I speak like an American. I pretend to be good at something. I brag. I blog. I podcast. I publish videos of my testing. I oppose certain things. I do so many other things that increases the visibility about me or my work to the world.

You should know why I do all these:

1. To help myself be a better tester.
2. To help the testing community in whatever way I can. 
3. To earn a living.

The danger

Some testers whom I coach / inspire appear to have mistaken some of the things I do as THE way to go about building their reputation and credibility. Ever since I am discovering that people are falling to a trap of trying to ape me, I feel bad of having a reputation and credibility.



Not that I didnt try doing things the way people who inspired me were doing. I tried aping almost everything they did. However, I did something to go beyond aping what they appeared to be doing.

It is a known fact that my life as a tester took turns when I discovered James Bach and then Michael Bolton. For me, these people were Gods of testing. You should visit my room, I have photos of James, Michael & Jerry enlarged and framed on the wall. That is just a minutest example of how much they mean to me. 


While they didn't make a parrot out of me who repeats after them, there was a little parrot sitting in me, trying to ape the things they do. It is human nature, I guess, to try ape your hero. Be it Rajinikanth or Upendra or Michael Bolton. I felt good about myself whenever I tried aping them. Why do we ape people? To feel good and proud at least for a short while.

Lesson 1:

Mimicry is one of my hobbies. A couple of years back, I was trying to practice the way James Bach & Michael Bolton speak. I did that so much that, I am now caught up in an accent that people think is half American and half Indian. Some people think I am faking my accent. I don't know, it has so happened that whenever I speak English, I start sounding like an American. So, be careful, your nature could be altered if you are trying to ape someone and it sounds or appears plastic to many if your nature is altered.

When you are trying to be like someone as bad as me, you will become a mutation of a mutation. That's not good for you.

Lesson 2:


Just after being done with 2 exercises that James put me on, I claimed to be a Rapid Tester on my blog. Some of my readers to whom all this was the western thing, even appeared to believe it. James had something different to say, "Are you a Rapid tester?" with a big laughter. I think it was the best joke he had heard for that year. He knew I was not yet a Rapid Tester although I thought with just little appreciation from him, I had almost become a Rapid Tester. For someone who was bored of calling himself as a "Test Engineer", Rapid Tester sounded cool and refreshing. Thankfully, I can make the claim today and live up to half their expectations.

So, when you make a claim about yourself or your testing skill, be careful to check and test it out by asking others, especially those who would find bugs in it.

Plus, when your hero appreciates you, don't fly for long and go out of reach. Respond to calls from gravity.

Lesson 3

Inspired by the way Jerry Weinberg writes, I tried aping the style. The easiest thing to ape in anyone's writing is a comma or a full stop. I was successful and here is one of my old post for your evaluation. There are 77 commas in that post. I was a comma addict.

I pestered Michael Bolton more than anyone else to help me better my writing skills. So he used to say, "Pradeep, don't, write, like, this" and I used to cry like a small baby. Michael used to offer me a philosophical explanation to why I cry. Miss those times :)

You could be thinking that you are aping the way your hero does his work but without knowing you are committing a blunder of a lifetime.

Lesson 4:


I used to assume that James was aggressive when he spoke against test cases and tried aping the aggression in similar contexts. I thought if I were to be respected as an exploratory tester, I had to oppose test cases. So, during my first stint as a consultant, I was roaming around the streets of Bangalore seeking business opportunities and knocked several company doors posing to be an expert tester. None of them were bothered about what I said.

One company invited me inside and started interviewing me. Turns out that they had an urgent need of a tester to replace the one who had quit. I stopped the interview process and explained to them that I was an independent consultant and what it means. A test manager there giggled for a while and said, "Even to learn our product it would take you three months and you are suggesting you can work on a  hourly basis / daily basis with us". So, I challenged them that I would find bugs in one hour in their product that is outside of what their test team might have found. So, they gave me a machine, their product loaded on to it and an one hour to produce a report. I fished out 22 bugs in that hour + a spelling mistake free report. I don't know, it wasn't shocking to me about the outcome but they were definitely surprised and excited to see so many bugs in one hour. I was called in to the Vice President and he told me, "You seem to have great test ideas. Can you write us help all these ideas as test cases? We will pay you 1.5 lakhs a month" and I walked away from their office silently but with aggression at heart saying "What? Pradeep & writing test cases? You know what, he is an exploratory tester, a rapid tester, such a unique thing for India and they are insulting him".

Years later, when I think about it, I feel, I lost a potential business opportunity and most importantly, I didn't help someone who needed it. They might have just needed my test ideas and they might have referred to it as "test cases". On just working for an hour, if they wanted to pay me that much, maybe when I worked for a month, they'd allow me to coach their testers and help create some good testers for that organization. I lost all of it.

I still oppose the idea of scripted testing. I teach exploratory testing. I am a Rapid Tester. However, today, if someone wants me to help them solve their testing problems - I forget all my ego and aggression and try to be in best service of the client. My work shall do most of the speaking. I shall help my clients understand why test cases would make their testers bad instead of walking away saying, "Ah! Pradeep & writing test cases?"

Having a guru is very important but it is equally important that you don't ape your guru forever.

I used to dream "When will I become James Bach?", "When will I become like Michael Bolton?" to which they taught me to be myself. That's their beauty!

 My LinkedIn profile reads, "Pradeep Soundararajan, a tester with experience of 7 million 4 hundred mistakes in testing" and it would mean nothing if I cant test as good as you who might not have a cool fancy title for yourself. So, don't get carried away with things I do.

You are what your skills are.

Finding Nemo Answers: Shmuel, William & Bangladesh Testers

So, I posted a challenge for software testers in December through this post : Why testers need to learn to write code that works. It was widely tweeted, circulated and even translated to Russian.

At the end of the post was an exe file with a challenge in it. I was practicing coding and wanted to do it in a way that helps me and others. I probably thought many Indian testers might pounce on it but that didn't happen. Even the testers I mentor didn't seem to take it serious. A part of mentoring is to not get disappointed when your mentees are not excited about a thing you are excited or you want them to be excited. 

However, there were some pleasant surprises. Thanks a lot to Alexei Barantsev, Shmuel Gershon, William Fisher & Sajjadul Hakkim and his team at Bangladesh.

I am going to be presenting a review of Shmuel Gershon's and William Fisher's work. What about Bangladeshi testers work, then? Well, its in encrypted form which I will link in the bottom.

At this point, I'd like to warn you that if you plan to attempt the challenge, you might not want to read further. However, if you don't want to attempt but read the answers and review, please go ahead. Think for a moment, maybe two and see if you really want to read the answer before attempting it. If you don't know what the puzzle is then the following wouldn't make sense to you, so just play with this and come back.

So, you decided to read their answers. No problem, you still made a wise choice :)

If you want to read Shmuel Gershon's experience report and answer first before reading my review on it, please do so. Here is the link to his blog.

The Black Box Tester Approach!

Shmuel, "So I downloaded Pradeep’s application, and got to work! Rather than just trying to solve the puzzle, I looked at it as if the mission was: “This is commercial 'roulette' style game. Stakeholders want to know it the game logic can be broken/learnt, which could mean a substantial loss of money when people start winning every time.”

That's a nice start. He didn't see it just as a puzzle but set a bigger mission to himself and made it more interesting. The way he set the mission didn't contradict a bit to the mission I had provided yet was different from the actual. That's a cool thing to do in such contexts. I would have been more curious to jump in to the puzzle and treat the mission as it is. So, that's a learning for me.

• 1. First, I ordered my environment to allow efficient and organized work:
  
  
  
  
  
  1. Increased the Screen Buffer Size of the Command line I was using to 500 (it turned out that I would do well on less, even half, than that (see point 2.b). But it didn’t hurt).
  2. Renamed the executable to be shorter and without number version: ren fnemo_1.7.6.exe fnemo.exe
  3. Changed the prompt to something shorter, cleaner, and that gave me context about this task against the other Cmd lines windows open: prompt $Cfnemo$F$S$G

Testing Hint: Organizing your testing environment before you start will likely help you during your tests. After you’re in the heat of fight, it will be harder to stop and get organized.

Shmuel provides a testing hint to all of us that is important but not practiced often. I should say that I would jumped into it without bothering to do any setup because

• I tried some basic executions of the program, to grasp the feeling of what it does and how hard it is to find Nemo. This also taught me what are the messages returned by the application when Nemo is found, and when Nemo isn’t found. They’re “Found Nemo this time!, Nemo Gill Bubbles SharkTooth Flow Phamplet Stinger” and “Ah! bad luck, didnt find Nemo this time!” respectively.
1. I also learnt that Pradeep used Perl and Perl2Exe to do the app.
2. Additionally, the application clears the screen at every execution, so the big command line buffer is not so helpful.

While trying to developer test this program, I found that not clearing the command line left over was a little bit irritating to me to perform my next test and hence I added a clear screen in the code. You might be interested to note that I clear screen at start because when my code erred, I didn't want to see it. A clear screen gave me a fresh look at the output and I was happy. It is possible that other programmers do something similar which impacts the usability of the product.

So, programmers aren't wrong unless they don't want their code to be tested.

The next time I write code, I shall keep this in mind and would try giving the user an option if he'd like to clear the screen every time this program is launched. That might solve one problem, probably without creating others.

Sarmila commented on the Challenge page about disassembling the executable. Although by learning all the assembly one can learn about the rules that move the fishes around, it would be extremely difficult and overkill. But this made me think on how Perl2Exe works… Maybe it just wraps the Perl interpreter and the Perl script together? If so, what if the script is stored internally in clear? I tried to open the app in an Hex editor, but no, the Perl script isn’t in clear text there, it is obfuscated.

• I also tried the useful Strings tool by SysInternals. My intention was to see if, maybe, the array of fishes could be seen at the app code, in order to learn the initial order of fishes. It couldn’t.
• That ended my cheating session  , from here on I used only a black box functional approach.

So, Shmuel came to a conclusion that the code might be obfuscated. Well, its not. Watch out for William Fisher's approach! However, Shmuel did try to cheat. When the programmers say, "Don't test this part!", that doesn't mean you don't do any tests there. A little bit of cheating could reveal bugs that remain even after the feature is moved to a testing ready state. Maybe, you might learn something about the software or maybe learn how the programmer stages his or her work. That insight is helpful, too. So, did Shmuel cracked without cheating?

I don't know, why not read further?

By now, I knew that Nemo changed places between plays.

1. Nemo also change places between invalid parameters (which may or not be a bug).
2. Moreover, the other fishes change places too, even their placement related to Nemos’ placement changes.
3. So I tried to see if Nemo will return to the same place in any consistent way or number of times. Running that is easy, you just enter “3″, “3″, “3″… and count where Nemo was found and where he wasn’t.
• I discovered there is regularity (for “3″, its every 2, 3, 17, 5, 10, 5…), but the regularity was irregular enough, and also changed for different positions. This was likely a consequence of the real logic, rather than the logic itself. This correlates to the movement of fishes, but does not cause it.

Shmuel starts with simple tests and that's very important. I don't know why our brains try to move towards complicating things than simplifying it. It looks like "simple thinking" is a skill and a must for tester. Puzzles, especially those which we were unable to solve were the ones which we complicated it so much that we put a puzzle2 over the puzzle1 and tried solving puzzle2 and yet tried getting the answer for puzzle1 through that. Solve it directly!

Another detail that helped me in the game was knowing the developer and the purpose of the challenge. This puts a lot of content in the context. Pradeep is a rather playful  and would put some tricks into it.

1. First I tried to see if Nemo could sometimes be in any placement bigger than 7 (there are 7 fishes, places 1 to 7, but there were no rules as to where the fishes were limited to be).
2. I tried in a toss of up to 220 attempts, and Nemo wasn’t in place 8 even once. So I let this idea on StandBy for now.
3. Then I tried… what if the “Minimum attempts” input at the beginning of the app affected where Nemo appears? 
   
   
   
   
   
   • 10 is the default for the “Minimum Attempts“. Nemo is at place 1 at the beginning there.
   • 11 attempts chosen: Nemo is at place 1 at the beginning too. Maybe it does not affect?
   • 12 attempts chosen: Nemo is at place 4! Bingo!
   • 13, 14, 15, 16 showed that the original places were cycling at ‘3′ intervals. That means that when trying to find Nemo on the default 10 attempts, it will be in the same placements as 10, 13, 16, 19, 22, 25, 28, 31, 34, 37, 40, 43, 46, 49, 52…

Having understood some bit of me, Shmuel made a conjecture that I could be playful and do some tricks to fool you all. That's a kind of thing like trying to understand the psychology of the programmer and model tests around it. 

You also notice the OFAT heuristic? You don't know OFAT, no problem, its One Factor At a Time. Now, I don't need to explain MFAT, right? Shmuel, tried using the OFAT heuristic (although he might call it in a different name) to narrow down or learn about the behavior. It did work well for him. He found something valuable that looks like helped him proceed on solving the puzzle.

You’re seeing that “220 attempts” above, and thinking if I really entered each number manually.

1. By this time, I decided that I was in need of some automation to insert inputs to the application.
2. The original app had no apparent automation capabilities, so I decided to use the internal input redirection of the Command line: 
   
   
   
   
   
   • By using the “<” operator, I could redirect an input file “input.txt” to the app using “fnemo.exe <  input.txt“
   • I extended that to “fnemo.exe < input.txt > output.txt“, which gives us with a complete dump of results that can be searched with Notepad.
3. We had an easy automation now, and this can help our sapient testing. I was exploring, and using automation to help me try different things help me explore my possibilities and get a bigger picture of the situation when needed.

Look at that powerful stuff. A tester shouldn't be classified as a manual only tester or an automation only tester but the industry designations suck. Shmuel provided evidence that it is the context in which either of the approach is used. Well, manual isn't a good work, so I agree with Sapient tester, as James Bach coined. Testing is related to human brain not the hands that type the input. All good testers I come across, don't separate manual and automation testing. They see it as approaches that compliment each other.

However, lets see what Shmuel did after that?

Do you still want to continue reading? 

---

• With the knowledge and tool I had so far, I did a lot of trials in order to find where Nemo is at each iteration. This could give me a hint if he moved in any regular or consistent manner.
• So I made a big file of winning moves (a lot of trial and error went in it).
• It was now time to model the results:
• First attempts at modeling the results:
• I got this: Nemo’s position: 1, 3, 5, 3, 2, 7, 3, 4, 7, 4, 4, 1, 5, 5, 2, 5, 6… 
  
  
  
  
  
  • Nemo was certainly jumping around without any clear regularity, even when I did it twice the times there was no repetition

On the Second modeling attempt, I tried to see if a more visual model could help:

1	2	3	4	5	6	7
X
					X
				X
		X
					X

Even in a table with many more results, it made no sense.

"No answer is also a useful answer". The first time I said that was in one of Elizabeth Hendrickson's exercise that I volunteered to guinea pig. Shmuel did fine. He looked for a pattern in the output but didn't get it and that was a clue for him that there appears to be no pattern. I wish he had done more tests on it but then sometimes critical thinking is a trap.

I didn't want someone to find the pattern of where Nemo is, that easily. I set the trap.

I changed my approach. Looking only at Nemo was not giving enough insight. Plus, I knew the other fishes moved along, so there might be a hint in their placement.

• I got this table:
• 
  
  
  
  
  

  Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger
  Stinger	SharkTooth	Nemo	Phamplet	Gill	Bubbles	Flow
  SharkTooth	Flow	Phamplet	Stinger	Nemo	Gill	Bubbles
  Stinger	SharkTooth	Nemo	Phamplet	Gill	Bubbles	Flow
  Stinger	Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet
  Phamplet	Gill	Bubbles	Flow	Stinger	SharkTooth	Nemo
  Phamplet	Stinger	Nemo	Gill	Bubbles	SharkTooth	Flow
  Flow	Stinger	SharkTooth	Nemo	Phamplet	Gill	Bubbles
  Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger	Nemo
  Flow	Stinger	SharkTooth	Nemo	Phamplet	Gill	Bubbles
  Flow	Phamplet	Stinger	Nemo	Gill	Bubbles	SharkTooth
  Nemo	Phamplet	Gill	Bubbles	Flow	Stinger	SharkTooth
  SharkTooth	Flow	Phamplet	Stinger	Nemo	Gill	Bubbles

• That didn’t look good. Nemo moved spuriously, and the other fishes either jumped aimlessly or stayed at their place. Nothing made much sense.

Shmuel, while he reads might be surprised to know that he didn't have to do this table. The code does the table, I had enabled logging. Shmuel didn't probably pay attention to his C:\ folder :)

Must admit that version 1.4 didn't have the logging but Michael Bolton, yes, the singer and tester, did a Rapid Test and suggested this feature. Maybe Shmueld did extract this from the log file but his report doesn't suggest he did.

I still thought that looking at the other fishes will bring the breakthrough needed.

1. In order to learn the movement of the fishes, I did a table of their placement in relation to Nemo at every run. 
   
   
   
   
   
   1. i. This is what I got:
   2. 
      
      
      
      
      

      Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger
      Nemo	Phamplet	Gill	Bubbles	Flow	Stinger	SharkTooth
      Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger
      Nemo	Phamplet	Gill	Bubbles	Flow	Stinger	SharkTooth
      Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger
      Nemo	Phamplet	Gill	Bubbles	Flow	Stinger	SharkTooth
      Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger

   • See anything interesting? The rows repeat themselves alternatingly! So the fishes were not really moving around, they were following Nemo!
   • My take is that Pradeep has two different arrays: One for Odd rows, the other for Even rows.
   • This may not be true, but it doesn’t matter for us. When testing, we not always know what exactly the programmer wrote in the code, but we infer a mental model. If it suits the needs, it is a good model even when not the real thing.
   • Many times I think these fake mental models are even better than the real thing. It’s the best way for them to act as an intuitive Oracle when analyzing an application.

"Boy, you got closer". That was my expression when I read the above. Good going pal!

So, if you have got curious to see what he did after that, you must visit his blog. I am not going to be revealing it further but to spoil your curiosity, he did crack it.

Kudos Shmuel! I hope other testers congratulate this effort of yours. It was very inspiring. I wish I could say "You could have done this and that" but it doesn't make sense to say that especially after your brilliant effort plus you did crack it! Your excel sheet, shows the way!

Now to William Fisher!

Whitebox Testing Approach

Aren't you curious to see the whitebox approach in action?

William Fisher...

K





Key:









 ! = Bug
* = Action taken
? = Question to investigate
@ = Assumption
Thoughts before executing:
? Size of Array
Unsure if the size of the array
@ At minimum, it must contain 7 elements.
? "All fish change positions"
@ Do all fish REALLY change position?
Wonder if their position oscillate between a set number of actual configurations
Are the ordinal positions simply incremeted? decremented?
Randomization?
Seeded?
? Are their intial positions consistent between session?
! I understand the rules, but how do I USE this?
? Assuming that I'm to enter a number representing an ordinal position
zero-based?
 
 
Notice, he has a different style of reporting than Shmuel, yet it is equally very useful and understandable. Finally, what matters is; is the information provided in a useful and understandable manner to the target audience? If you meet that through any kind of reporting style, good job done. William appears to set the context right to read his report further and when I ask questions to myself if I do that, I don't. However, just because I don't do it and I see other tester doing it is not a plausible reason that I should do it. Knowing different ways is of help when I'd have to switch style from one kind of audience to another.
 
If I have held you till this moment, It is a big achievement. Not just mine but of Shmuel & William, too.Test 1: 
 
 
Track the fish at ordinal position 1 across successive attempts
> Minimum attempts is set to 10 but you may increase it: 0
> 10 attempts left
! I entered a number and received the message "10 attempts left". Was I only supposed to press enter?
---
> Guess the position of Nemo :0
> You probably didn't give any any input or your input was invalid;please provide a valid input
> Guess the position of Nemo :"
! Did that count as an attempt? How many attempts do I have left?
@ If the language uses zero-based, potential exists for off-by-one error if input is not decremented properly
---
 
 
I told you. I did tell you, its hard for a tester to avoid finding a bug :). You also notice he is asking questions that are different from the ones Shmuel asked. He might also have suspected if making an invalid entry or just hitting the enter might cause a movement of Nemo but did it happen? 
 
> Guess the position of Nemo :1
> Ah! bad luck, didnt find Nemo this time!
>
> 9 attempt(s) left"
! There is an extra carriage return (prior to '9 attempt(s) left'. Should that be there?
! Attempts are decrementing from 10. Opening screen says I can increase it. How do I increase the number of attempts?
@ Different code used to display max number of attempts, than what is used to display the remaining number of attempts
@ The use of '(s)' in 'attempt(s)' indicates that the code does not change output based on remaining attempts.
@ Refactoring possiblity: use one function to display the remaining number of attempts. This will accomplish the same thing
---
 
  
So, as and when tests are performed, William is thinking about constructing the code in mind. It is definitely a useful approach, at least to me. I always try to imagine how the code behind this GUI look whenever I am not exposed to it. Modeling the code, no matter how wrong, is a useful approach. I get test ideas different than what I had by modeling the code behind the GUI. Now, if there ain't a fancy name for it, don't worry. It doesn't matter.
 
 


> Guess the position of Nemo :1
> Ah! bad luck, didnt find Nemo this time!
>
> 8 attempt(s) left
End session:
Found Nemo with 1 attempt left
Having found Nemo, I've been given the opportunity to guess one more time.
It closes after 5 seconds; cannot mark the entire session.
? Does this program generate any artifacts?
? When an attempt is not accepted, do the array positions change anyway?
================
Test 2:
Manual run the same inputs in s new session.
Run session with FILEMON attached
End session:
No artifacts generated by the executable
Did not find Nemo in this session
? Is Nemo's position time-based?
================

 
 
This is cool. Suspecting a specific thing to be happening and basing tests on it and not being biased when seeing the output. Appears like a critical thinking approach to me.
 
 

Test 3:
Manual run the same inputs in s new session.
Run session with TSEARCH attached
In TSEARCH, HexDump shows some secrets:
- print "\nHave you reverse engineered the logic? If you haven't go on!\n";
. else. {print "Ah! bad luck, didnt find Nemo this time!\n";.
End session:
Need to dump ASCII Strings from exe file.
 
You expect me to talk about tools now? Right. Tools are to help testers. This is another example of sapient testing, I would say. Until we use a tool, we don't know if its going to be of much help. If you observe what William was doing, he was constantly worried about code and was also concerned about which tool could be of help to him. Tried Filemon and then TSEARCH. I didn't know about TSEARCH till William sent me the report. Wow! You got to do something to get a tester to talk to you. That's all education happens as a side effect.
Every software offers us enough clues to help solve the problem. It ain't an intentional design, I guess. I still like to think that way although I might be wrong. I am not going to say this to my client while testing for them or we wouldn't have time to talk about these.
 
And you'd want to know what happened after William got that clue? Watch out for the video.
 

If you were by any chance curious while reading this, I too am, for a different reason. I want to know what would happen when William Fisher discovers the things that Shmuel did and vice versa. I bet they'd appreciate each others effort and approach and would learn from each other. 
 
Shmuel had almost concluded that there appears to be no way to look into the array by opening the exe in the HEX editor but William's experiments violated that.

 
Both of them almost missed the log file. William discovered the existence of the logfile by looking to the code. The next time I should just pass on the .pl file instead of EXE, I guess. All beans spilled, ah.

 
 
What's the deal with Bangladeshi Testers?

 
 
Ever since I discovered SQABD.com , I am wanting to go there. I want to go talk to those testers there. Not got a chance yet. I am hoping it doesn't remain a dream. Sajjadul is a tester with thoughts that are widely respected within and outside of Bangladesh. He and his team took time out and discussed about solving this puzzle, after experimenting on it for a while. 



Watch the video: (Some part in Bangladeshi and rest in English)

Want to get your hands on Finding Nemo 2? Watch out for it in March.