What is invisible work?

invisble man

Work that is done by someone or a group of people that is not tracked anywhere is invisible work! This could be something that is an interruption to your day, this could be a side project that you are doing with someone else in your organisation, or anything along these lines, that is not covered as part of your daily stand-up or updates.

In fact, no one apart from those involved in the work know that this work is being done and it doesn’t really have many stakeholders apart from the ones doing the work.

What is not invisible work?

Top secret projects or something strategically significant that you run a secret project that is well protected from the rest of the organisation and the outside world to avoid leaking information to competitors. This work is visible to the stakeholders and its progress is quite often tracked very closely.

Hence this type of work is not invisible.

What is wrong with invisible work?

not being able to see progress with work is messed up
There is so much wrong with keeping work invisible.

  • It is only mentioned in passing
  • there is no place to visualise what’s being worked on when looking at it from a team perspective
  • this work might be additional work to the regular commitments of the team
  • it could be unplanned work that someone sneaked in but you may never know as you can’t even see it in progress anywhere
  • a team cannot confidently commit to new work without overloading the engineers working on things that are not tracked

You still wondering “What kind of work is that?”.

Chances are that you have an organisation with healthy practices that you ensure all work is tracked.

It could also be that you don’t realise someone is sneakily being asked to do work that is not the team’s priority.

What are the downsides of this - as a leader?

leader can't see
  • You tend to trivialise complexity of the work that someone is doing as they brushed off the complexity saying - it is just a one-off request. This lack of clarity makes you wonder why the priority work is taking longer than expected.
  • You fail to recognise the accomplishment of the person(s) who just completed this work
  • You are unable to foresee any dependencies on (up/down)stream teams, which you could have unblocked for your team to make progress
  • You are unable to communicate progress on this invisible work because you didn’t know it was being worked on in the first place

Ever experienced any of that?

I have had an instance where there was an incident or rather difficulty that my team caused by changing the version of a core library that forced users to go update dependencies on their underlying modules as they were now migrating to our platform. I don’t want to go into the details as I would have to explain everything about my project. But I guess that gives you a gist of the problem.

The worst part, as an engineering manager, I didn’t even know this happened, until my manager got wind of the issue as feedback from the head of engineering of the team that was affected. I was lost for words! I didn’t know my team was working to help them with this! No sign of this work on our beloved JIRA board!

What are the downsides of this - as a team member?

team can't see
  • You cannot understand how long something has been in progress
    • every standup - you keep repeating - it is almost done
  • When the said work is completed, nobody knows that you worked so hard to finish it. Leading to lack of recognition
  • Your team members do not know that you could have benefited from their help

The story of taming invisible work - inspired by true events

The context

let's set the stage

There was a small insurance managing agency that had an even smaller IT department. It comprised of 8 developers or less, some application support staff, the CTO, a business analyst and a couple of scrum masters. They thought they embraced agile and were proud of it. They knew IT was going to be core for the future of the organisation so hiring was in full swing.

There was no particular team topology.

Every team owned every application in the organisation. Teams were delivery machines. But team topology is not the focus of this story. So ignore this part.

The IT department in this organisation was serving several different departments:

  • finance
  • underwriting
  • claims
  • marketing
  • the Chief Innovation Officer’s next big idea

It might sound like a small list but trust me there were a lot of asks from everywhere. But no one could see these or know the priorities of them.

Early days - 1

funnelling asks into the business analyst
  • No platform/tool where customers could be on top of their feature requests. All the stakeholders generally spoke to the honorable Business Analyst
  • BA would give a rough estimate of the feature and ETA to the customer prior to consulting with the team. This was gold - the team would have estimated this work very differently
  • BA vaguely knows what is currently being worked on. Gets to attend the sprint reviews.
  • We overestimate the power of human brain here - small company, not too many asks of IT, this will do… until it doesn’t

Early days - 2

funnelling asks into the scrum masters
  • When passing by on the floor or on an instant messaging program, the business analyst would tell the scrum masters that there was this work to be delivered which should nwo be done by this new date!
  • Teams were already working on an existing commitment - previous feature, now interrupted by this surprise priority
  • Scrum masters and teams have to discuss how to cater to the new feature request, sometimes mid sprint, sometimes on the first day of the sprint. This made planning extremely painful
  • The business analyst always looked at velocity of one team regardless of the size of the team and compared it with another with the logic that “if 5 people can deliver 30 points, then 2 must be able to deliver at least 5 points”
  • These artificial deadlines forced teams to cut corners so that they could deliver on time

Execution of the work

  • Every team knew a little bit of everything - no Subject matter expertise also there was a lack of ownership
  • No team owned any particular application
  • There were way too many deadlines to meet, and quite often, work was introduced that should have been done yesterday
  • And scrum? There were team sprints and boards
    • But teams less familiar with an application/domain would do less accurate estimates or just over-estimate the size of the work
    • In order to deliver by deadline - choose the quickest way to build - like tackling a competitive programming problem

Consequences

quality degraded
It is not a surprise that too many shortcuts used in solutions would result in code quality that would be trash.

technical debt - shooting through the roof
Technical debt went high - making it much harder to make further changes. Teams are starting to wondering how this was agile.

frustration, anger, demotivated
Developers were getting increasingly frustrated with the deadlines that were set. They saw a total neglect to quality as a lack of craftsmanship. The lack of ownership fuelled the increase in technical debt as solutions were often implemented as a stop-gap mechanism with the excuse “we’ll fix it for now. let someone else clean this up later”.

What could the solution be?

Do we need to see the work?

The scrum masters lost their patience. As usual, during one of their weekly pub hangouts they decided, they had to do something to prevent losing engineers. The decided it is worth speaking to this business analyst and getting a dump of all the work that we are supposed to do in a backlog somewhere everyone could see.

This was pre-covid times. Meetings in pubs were normal.

scrum masters and business analyst have a chat

As they were scrum masters, they decided to call a meeting with the business analyst who for reasons unknown was also called a product owner - only that was no product to be owned. The scrum masters asked the BA to work together and produce a visual representation of all the work to be done. Started with a bunch of post its on a board. Ended up covering an entire wall.

years of work on the wall

As much as they were relieved of being able to see the work that was promised on the wall, they were equally scared and excited.

They realised we need to change to expect better results.

Starting small

ideas for team topology, start with one team

Finding a team to be the guinea pig was not too difficult. There was already an application that was pretty popular and had a team of engineers who had a pretty decent tenure at the organisation and knew the context and codebase pretty well.

one team chosen!

This was the beginning of the experiment. A new product owner was hired for the team as the team already had a scrum master.

The large backlog of work on the board, was then refined to extract work related to this application and put into the team’s backlog.

But wait, the team needs to relearn scrum. The department decided to create a sprint 0 for the team to train on scrum and come up to speed with iterative delivery and the agile manifesto and things.

the whole team trained in scrum!

Building trust

  • Conversations with the stakeholders started - the product owner developed a relationship with the stakeholder.

    one team, one area or application

  • Started with the tiniest commitment

  • Pulled in more as 1st increment was delivered early

  • Ramped up velocity as sprints went on

    accelerating velocity - more value delivered over time

  • Feedback loop with immediate customer - Product - in this context insurance product

    product and the team in close contact, feedback implemented iteratively

  • Started seeing value being delivered, impact recognized!

  • Experiment successful in one team!

    Event Borat felt it was a great success
    We saw that with the right team topology and the trust, we could get a hold on delivery.

What was learned?

starting from te sticky notes on a wall

The lesson is clear.

If the scrum masters and the business analyst had not put the all the work that had been committed on a wall, nobody would have known how much there was to do. Nobody would have known in what order, work should have been done!

A simple exercise of publishing the work from the Business Analyst’s head to a bunch of sticky-notes helped them all understand:

  • The size of the challenge ahead of them
  • It gave them the vision that they had to hire
  • They had to take immediate action
  • The fact that maybe some of the scope must be renegotiated

This is why visualising work is extremely important. This triggered the change that was to follow.

visualise everything for everyone to see