Ask HN: So many software projects, but no reliable productivity metrics?
I would be interested in hearing your thoughts on ways to understand (intentional not saying measure) productivity of software engineers with metrics. I know it's a polarising topic and difficult to tackle. Usually the path of discussion goes to "commits are not relevant", "my situation is unique", "requirements are changing", and so forth. But in the same time, after so many projects delivered and considering the maturity of tooling we have today (acknowledging that this make it also very complex), there are many, many things which are kind of predictable and similar to other delivered products. Outside a small percentage of developers which work on innovations, r&d and areas which by nature are unpredictable, the vast majority are just doing the same stuff.
So what are your ways of understanding that someone is productive or not which is not based on gut, perception and other subjective factors?
The thing to understand is that software is not useful in and of itself, it is only useful for automating or increasing the efficiency of some other endeavor.
If you imagine, say, a hotel, it has plumbing that makes the toilets flush, and an accounting department that pays the bills and stuff, but none of those are an important driver of how people choose their hotels. Upgrading the plumbing will not lead to more guests, but if the plumbing breaks, that can lead to a negative experience and needs to be fixed quickly by a competent plumber.
It's the same with software - you need to figure out what the actual end goal is (more sales? attract a new client? more reliability?), and then the productivity of the developers measures how their software changes contribute to that outcome.
That's true. But in the same time I think it will be useful to _understand_ if I can deliver that vision with 10 people or with 100 people. You can start with the assumption that everyone has good intentions and will contribute similarly (in correlation with their experience and capabilities), but I think it's missing the bit that will tell you more objectively if that's the case.
Sounds like a read of The Mythical Man-Month is in order? Even with the assumption that intentions are good and contributions are equal, that is not to say that the task of software development scales linearly as more people are added.
You can even deliver with a single person. But that leads to more time spent on building. Adding more people adds more time spent on communication. There is a sweet point somewhere, but there's no formula to find it, as many factors depend on the human side of the software developers.
Almost everything in software development is in a constant state of change. The languages, the libraries, the frameworks, the development tools, the protocols used to transmit data, the platforms you host on. Even though we’re doing the same thing over and over again, it’s not really quite the same. Even something done so often as auth has so many requirement nuances to make each environment unique.
This would be like trying to build a skyscraper but having to keep up on a dozen changes to the raw materials every year. Each time something materially changes you rebuild the entire thing with that change.
All you can reasonably measure is how well the process is going, which is why you have efforts like DORA and SPACE. They effectively try to measure the state of flow that developers need to be most productive.
The problem with metrics on coders who are just doing run-of-the-mill tasks is that even if you do find a good metric... when they want to push beyond run-of-the-mill and start innovating, they now look bad. You ultimately end up incentivizing coders to game the system, not to build quality code.
If you have test automation in place this becomes instantly simple: how much time does it take to deliver a solution with minimal test coverage that also passes prior test coverage?
In my position productivity can be safely measured by total number of hours spent on the tickets every sprint. I'm most focused on stories and tasks but we also log spikes as researches.
If you imagine, say, a hotel, it has plumbing that makes the toilets flush, and an accounting department that pays the bills and stuff, but none of those are an important driver of how people choose their hotels. Upgrading the plumbing will not lead to more guests, but if the plumbing breaks, that can lead to a negative experience and needs to be fixed quickly by a competent plumber.
It's the same with software - you need to figure out what the actual end goal is (more sales? attract a new client? more reliability?), and then the productivity of the developers measures how their software changes contribute to that outcome.
This would be like trying to build a skyscraper but having to keep up on a dozen changes to the raw materials every year. Each time something materially changes you rebuild the entire thing with that change.
All you can reasonably measure is how well the process is going, which is why you have efforts like DORA and SPACE. They effectively try to measure the state of flow that developers need to be most productive.