Measuring and Boosting Engineering Velocity
I have been recommending our portfolio company Code Climate‘s relatively new Velocity product in most of my recent Board meetings.
I hear from management teams again and again that they want to understand their engineering utilization and velocity and benchmark it.
Everyone feels like they are not getting enough production out of engineering but have no way of knowing if that is just how they feel or the reality of the situation.
Velocity is a great tool to figure all of that out.
Becuase you can’t manage what you can’t measure.
If you feel like you need more visibility into your engineer team’s production, check out Velocity.
Comments (Archived):
I am not debating if this is a good tool for engineering teams to use internally.I will debate if it is a good tool for management to use to bludgeon engineers.Saying “everyone feels they are not getting enough production out of engineering” is true, but I would say engineering feels they are not getting enough out of management as well.Disclosure: I am management not engineering.When you do things like “track bugs” People start to argue with testing about is it a bug? They build resentments.When you say how long to code review or check-in? People optimize for that.I remember doing a turn around at a company where they measured lines of code written.Guess what??? 200 lines of shitty code is infinitely worse than 1 line of good code.There was a set of code that parsed data and instead of using a function the programmer literally typed out each case. Turn A into a. Turn l into 1
yes although that is management’s fault. this tool can deliver a lot of value in the right hands.also, i would expect as they get more client data they can apply more machine learning to understand what is good code, what is bad code, what is redundant code (i.e. how many people have written the same function a million different ways, shouldn’t that be standardized into a library, etc). i don’t know much about this company but the problem they are addressing is very real and the solution will enable greater scalability and complexity in software development.
I’m torn on this.I 100% agree with your take (most people will optimize for the thing you measure and deem important)…at the same time, data and measurement is generally a good thing (you often have to base decisions on something more than gut)….so I’m inclined to say there’s nothing wrong with having KPI even in engineering and expose those numbers to management…as long as it’s not the “only” thing management is using to manage. Goals, mission, and people matter as much (if not more).
Maybe I didn’t stress my point. Tools work for those that use them. Use the best tools. Training and onboarding is critical. Hiring the best is the most important.But if the premise from the non-doers (management) is those god-damn doers (programmers) are not getting enough done, I need to track things and crack the whip then you have set yourself up for failure.No different than sales. You have to use a CRM tool. No different than support you have to track cases.The fine line of management of creators is saying lets get the best tools, lets look at metrics but let’s not start off by saying: I don’t think people are getting enough done, let’s use a tool.Sure this “works” at places like Amazon’s warehouses. But you churn through people. Who cares? They are “human resources” I get temp agencies to cycle my people.In creative companies the doers are your Talent and Culture, not your Resources.It’s a fine line. Since we were on sports yesterday, why did Chip Kelley fail in the NFL????? Players hated his measurement systems he would bludgeon them with, they rebelled and he couldn’t get and keep talent.
+100
I worked in a company a few years back that was using agile (2-week sprints) and Jira to micro manage the engineering team for an embedded development. It was a disaster. Software development near the HW/SW interface is notoriously difficult to schedule, especially in ridiculously short time frames like that. I left the company BECAUSE of their micro managing.Here’s the thing. Web development is easier than other types of software development. The architecture of the system is well defined. The tools are very mature and most importantly, the function (i.e. a database driven website) is straightforward. You can throw lightly trained coders at it and have some expectation of success (although the recent pair programming silliness flies in the face of that). I’ve seen a trend to try to push these cookie cutter management techniques to other types of software development that are much less well defined and they don’t work. To your point, you really need to search for and hire the best talent you can and then give them space to create. That doesn’t fit with agile model at all but its a physics problem. There’s no knob that mgmt can turn to make it work better.I will also say that the recent trend toward dumbing down CS programs (e.g. teaching Python/JS vs C/C++) only makes this worse. The CS grads coming out of the average program today are not as well trained as the ones that came out 20 years ago, for a wide variety of reasons. This means they are not as productive because they CAN’T be. They haven’t been given the right tools. But they’re under tremendous pressure to get that CS degree so they can get that $100K job so they can pay off their student loans. Its a vicious cycle.In every situation that I’ve been in over more than 30 years now, when mgmt gets to the point where they feel they have to step in with micro managing, the projects have failed…
I tell my group/team, it’s our job to solve problems.If we’re going to optimize and measure for anything, it’s solving problems.As a manager (and coach)…I also tell them it’s my primary job to put them in a position to succeed.So we’ll use any process or technique that the team feels helps them solve problems…if/when it stops or hinders them from solving more problems, we shake things up and try something different.BTW – “solve problems” also means none of my team is just a “coder” or “tech person” (though they prob. will default to trying to solve *most* problems with software and tech)…they’ll pitch in anything they can if it helps “solve a problem” (including keeping the office clean, moving stuff, etc. etc.)…and it helps ensure there is no such thing as “not my job”.We’re not perfect, but we are a team…and I like to think we have a lot of fun & even some success from time to time too. 🙂
Your last paragraph is the money one. And I guess why I disagreed with this post. If you are discussing what tools you need to “whip” development into shape at the board level.You have failed.
Agreed. If it reaches board level, it is already likely a management or team failure, even if was due to giving in to developer, peer or investor pressure.I mean, this is really basic stuff – not rocket science or even computer science. Software engineering principles [1] and *practical, usable* techniques based on the principles, were invented (mainly) in the US and Europe over a period of decades [2], starting a few decades ago. (due to, and after, more decades of debacles that were labelled “the software crisis” [3], a.k.a. tons of prominent, highly expensive project failures), so those principles and techniques are nothing new. Yet so many companies (these days as well as in the past) insist on reinventing the wheel (I don’t mean the tech wheel but the process wheel, or rather they do this: [4], and screw themselves in the process – pun not intended.)”High-quality software is not expensive. High-quality software is faster and cheaper to build and maintain than low-quality software, from initial development all the way through total cost of ownership.”Above quote from:https://en.wikipedia.org/wi…[1] https://en.wikipedia.org/wi…[2] https://en.wikipedia.org/wi…[3] https://en.wikipedia.org/wi…[4] https://jugad2.blogspot.in/…
data and measurement is generally a good thing (you often have to base decisions on something more than gut).Said in jest by me ‘you just say that because you are a sports guy and rankings and numbers matter’. And those numbers keep getting drilled down with further granularity and provided for entertainment purposes. An award and recognition for everyone (and I mean that not for little league either not even talking about that).I am not a believer in driving a small organization by the numbers. Large company (whatever that is) for sure. Small company ‘you know it when you see it’. Years ago my small (20 people) and very profitable company if people were moving fast and things were in motion we were making money. If people were moving slow and there weren’t things piled that needed to be made up we weren’t. No need to have any reports and what not. If people didn’t have the work (there were no ‘jobs’) didn’t matter how they moved. If people did have the work they adapted and moved and we made money. Sales drove everything production adapted. Simplistically of course. We had the reports of course. But they were geared toward what to produce when. For important customers or to knock out small jobs things that mattered. The hot shit production people were well known and rose to ‘primadonna’ state.
+1.”data helps you make better, informed, decisions” is a core belief behind my current startup too ( Veritonic )…so not just because I’m a sports guy…but sure it doesn’t hurt either. ;-)I would also say you def. “know it when you see it” (and that’s easier in small companies)…but a core use of data is helping you form & justify decisions when you aren’t seeing it…You’ll likely have a number of theories why sales are down, production is down, or whatever and you just aren’t “seeing it”…testing those theories without data & measurements is a sure fire way to be wasting time and money.(btw, I would argue in your small company example, you were watching and using the data [piles of work; sales driven requests; etc], you just did it internally and naturally vs. formally and externally defining it)
you were watching and using the data [piles of work; sales driven requests; etc], you just did it internally and naturally vs. formally and externally defining itWow excellent retort you are correct about that for sure (really).(Picks self up off the ground and dusts off for rebuttal.)However isn’t that just a nuance? My point is those ‘informal’ measurements suffice for the purpose and nothing more sophisticated is needed. Now if I wasn’t physically at the plant then sure I would need something as a proxy for what I could not see with my own eyes.Will also point out that after some time you can even tell if a machine is running correctly by the sound of it and also the way the machine operator is working. Ditto for mechanics and the trades (and I am sure Phil Sugar will confirm) you can tell if someone knows their shit by the way they use their hands and how they work. This is not a 100% measure but it’s highly accurate.
1. Thanks! I officially accepted my “win” and walked away after your first sentence. :-)2. Though I’ve already counted the “win”, yes I agree with you. Often ‘informal’ is enough…but I still “win”. :-)3. I actually know of a company that is working on a system to be installed on factory floors…it literally listens to all the sounds on the factory floor, and can detect when a machine should be looked at for maint. (because the sound of the machine has changed from “norm and working well”). Very cool tech and concept.
#1 – Yeah, well deserved rarely given out.Just took another look at your site (veritonic.com). It looks great graphically. I still think you need another name as ‘tonic’ does not work the way I see it (and I am usually right about this type of thing and human nature).Anyway I think this is a key thing you need to put on your home page which I just plucked (quick look) from the Burlington Factory Page”right sound for their customers”So what you should do is this (rotate on home page instead of ‘audio effectiveness for marketing’ (fails puny brain test).”right sound for your customers””right sound for your employees””right sound for your tv spots””right sound for your web videos”Each of those should also be tagged with the benefit even if obvious ‘customer=higher satisfaction and more sales’etc…. https://uploads.disquscdn.c…
Thanks!We just hired a marketing person (like a week ago)…so the messaging (and web site) is going to get cleaned up and focused a lot in the coming days/weeks. Will forward on this feedback and thoughts for sure – THANKS!
My son bought a crappy 4×4 a few years ago. Every time I was on board I told him something like: “Your front right shock is broken”, or a humming bearing or lose plates that entered in resonance with the engine vibrations or a ticking valve. He didn’t believe me at first, then when he began fixing things he asked why I knew.I had owned a few old cars before so I had a trained ear. Modern and newer cars are more difficult though or I lost the superpower. :)Machines can talk if you pay attention to them.Really awesome concept! What is the name of the company?
I’m not sure what they are calling themselves right now – one of our advisors is involved in it, and it’s coming out of the academic world, so the last I heard (couple of months ago) they were still in the R&D and testing phase (had working tech, but not actually “in market” or “in business” yet)…so it might be awhile before it actually launches to the world (the downside to many businesses that “launch” out of the academic world)…
Modern cars have computers that compensate for certain things. Also truly the tolerances have become much better.But holy crap if something breaks out of warranty?Brake light for GMC Denali: $600. (and it is LED and not supposed to break, but they are notorious)
You can tell by the tools they use. Plain and simple.I knew we were going to pull out of the data center on 401N Broad Street when the network guy pulled out a home made loop back tester to test the network connection to our rack. Are you f’ing kidding me???Hey give your people the best tools. (Fluke in this case)But also as you point out measure too hard and you get the Amazon driver throwing boxes in your door or not delivering.That UPS driver, that “wastes” 15 seconds to say Hi!!! Did you see the game last night??But when you have somebody measuring stuff they don’t do, here is what happens. Man that Joe, he has great metrics, I mean he kicks ass he is a high performer (that is all Joe optimizes for). Jane her metrics suck, we need to transition her out. Problem is that Jane is doing all of the hard work. She leaves. Sally says I see how this work takes another job. Jim goes, I see how to do this I’ll be like Joe.Now you are sitting there wondering why your engineering team sucks. Guess what?? It’s not going to get better, people know you suck. So you crack the whip some more…….you are left with those that can’t find another job.
pull out of the data center on 401N Broad Street when the network guyA place that I was at (carrier hotel) not far from 401 literally lost the spare parts I had left on site. I found that out when I went there to inspect and wasn’t able to find them. So now I keep spare parts also at the office (which obviously creates an additional delivery barrier). These are all things you learn over time by interacting with everyday life that unfortunately that people who have not grown up on the assembly line (meaning either new or in management and never actually did the jobs of those they manage) never come in contact with. As my Dad told me when I was in college ‘they will just tell you to hire someone as if it’s like buying a loaf of bread’ (I added the loaf of bred to make a point). Or ‘hire good people’. Thanks professor.There is really only one reason why all of these startups and the internet have actually worked out at all given how little people who are doing businesses have typically when starting them. It’s because of a few factors almost exclusively:a) Huge and growing market for what they are selling. You have enough new prospects to make up for the ones that you lose or piss off.b) Money invested OPM that allows you to paper over any mistakes and make plenty of them.c) Addictive and/or free product.d) No accountability to customers (no physical place to get abused and not even a phone number)Honestly try doing it with your own money in a business that is not growing. Try doing the same opening even a Pizza restaurant. Entirely different.I love especially people who think all legacy businesses are full of fat and extras that can be cut. Sure you can cut all you want and end up foisting the aggravation on the customer to deal with.
Was that in the same building as News Channel KYW 3 building? Was there too. They had a different problem. They would show their networking room and say “it is Pheasant Under Glass”
Nope different part of center city a carrier ‘hotel’.
Or in other words buy this tool and run your IT department like the post office. Then wonder why your competition is lapping you.
I am not debating the value of the tool. I am debating the premise of I don’t think my engineering team gets enough stuff done so I will use this tool.
Let me ask you this. What size organization is this product even targeted towards? I can’t tell that from their web page at all. In fact the site is very sparse on info to begin. Not to mention that off the top a site that doesn’t mention pricing to me is (often) problematic [1]If you have a small team don’t you pretty much know seat of the pants what is going on and who is productive and good? I would think the product becomes valuable once you get into a large enough number of programmers that you need to ‘measure it’. What number is that. After all programming is not like a production line or measuring the output of printing machine operators. It takes creativity and problem solving many times. [2] And I don’t think any info is good info. Because it can be perverted to easily.[1] It screams bogu and if that is not the case then why not give people more info so they can see the value instead of simply collecting leads that can be chased down (with time wasted) by business development? Not saying it’s never appropriate to not show pricing either.[2] There is a similar situation with measuring medical outcomes. What are considered the ‘better’ hospitals also are the hospitals that take on the more difficult cases which the community hospitals don’t. As such their survival numbers will be impacted. Important concept. Ditto for individual surgeons, attorneys or prosecutors. Your simple numbers and win/loss ratio (or survival) can and is gamed in order to get good numbers. Not a new concept. Simpleton’s of course don’t see it this way.
Took some digging but found the pricing https://codeclimate.com/qua…
Yes and why shouldn’t you have to dig to find the price of something!
Hi Bill — That pricing is actually for our Quality product, not the Velocity product which this post is about. The Velocity product is in a public beta, which is why the pricing is not yet public. We expect to be taking it out of public beta with public pricing in the not-too-distant future.
You need to cough this up, cough up the facts, the real situation.I’ve written a lot of software, going way back. Fred’s post gave me and my well experienced nose a strong sense of a cow barn in need of a lot of shovel work. So, I didn’t visit your Web site. Now that I’ve read more on this thread of today, I don’t want to visit your Web site.Computing has a huge problem: As I’ve stated often enough, IMHO the worst problem in computing, so bad it is a huge bottleneck for progress in our economy and civilization, is that computer technical people are 99 44/100% flatly incompetent at explaining their work in writing in the English language. So, they don’t explain, their work does not get explained, progress is slowed, and that’s where we are.And that’s just for the crucial technical parts. For the public relations, publicity, marketing, virality parts, the situation is much worse.Far too often for some subject at the Web site of some company, I click and click and click and click and start to scream, dream of … in infuriated retaliation, put up with it when necessary, and avoid it like a big case of Ebola otherwise. If you are coy, incomplete, evasive, obscure, seem deceptive or exploitative, use undefined acronyms or terminology, …, then I’ve got MUCH better uses for my time, my mouse finds the big X in the upper right corner of a screen window, and in one click I’m off doing something better with my time.
Thanks for the clarification. I did share share your product with our VP of Engineering
In my experience, it is incredible how much management time is wasted managing each engineering hour.The main problem I see is that managers tend to focus on project costing and release dates completely ignoring the workflow and details of software construction, which is where things happen (or don’t).In managerial lingo, you are looking at the wrong KPIs.
Here is the issue. You either view engineering as a commodity cost, or as incredible creative resource that you take care and nurture. No other way around it. I can tell you what kind of people you get and keep depending on how you view and measure (not them use tools but how you measure)Yup Amazon says how many boxes did you pack: Bottom 5% you are gone.Developing code is not packing boxes.Let’s say there is a really hairy, ill defined, hard piece of code.If you knew you were being measured by metrics, you’d try and stuff that on somebody else, you’d never take on the challenge.
Some parts of a project can be considered a commodity cost, the easy predictable parts. But, as you mention, some projects also havecomplex or risky components, which require some previous R&D or extended time in the chart.Recognizing the hairy parts in advance and selling that to management, which essentially define the amount of extended engineering hours, is the artistic part.In my work I try to use a proof of concept, which is observable by every stake holder so it is easier to sell and follow.
Here is the problem. Remember, I am management. Have been for my career, but programmed in school.As soon as you say some is commodity……all becomes commodity.The line is crossed.So a basic web page might be a commodity…….except…..well I need this one little thing…..not a commodity. And how it looks??? Or page flow??? Not commodity.
I know who you are and respect you, Philip. And I am also happy that managers as you recognize the creative parts of our work.I have been making software for 32 years now and always have to add to the burden (and joy) of staying updated, dealing with clients and their management (less joy).The managers I work with tend to and try to commoditize (is that a word?) everything. I think that it is natural because it makes it easier for them to quantify things and put them in a spreadsheet. So I try to adapt my processes, at least in form, so that they are more easy to deal with.
The issue is they do this to commoditize you. Then it becomes well an hour of programming in X country is 80% cheaper.Now this works for people that know the cost of everything and the value of nothing.Let’s take a low tech example. How much does a kilo of steel cost? Even that low tech. Well heck, I can buy a kilo of the cheapest Chinese steel for 20% of the cost of the best German or Japanese steel. (let’s not go into if that is xenophobic, just example)Well, my business depends on those blades and if they break my machines go down……….which is cheaper????
>Now this works for people that know the cost of everything and the value of nothing.Great comment (the whole one, not just the part I quoted above).I had been remembering the quoted part recently. The form I first read it in, was something like “an economist is a person who knows the cost of everything and the value of nothing”. AKA a bean counter.
I always say it doesn’t mean a hill of beans if you are a bean counter and there are no beans to count.My point is the only person’s job to bludgeon is the direct manager. If you want to bludgeon the manager because you manage them fine.But every tool I have ever seen that allows non direct managers to bludgeon a department is severely abused.Let me give an example. There are guys at my church that work at Amazon. We have more warehouses than anywhere: 5 million square footers that serve from NY to DC in a 15 mile area.There was a guy Vince who works like a mule. I am amazed when I work next to him. He got the flu, his wife got C-Dif, he was not productive that month. Paul his manager (who also goes to the church) had to fire him because his stat’s were down.No choice. That’s Amazon’s metrics.We had to watch Paul choke down an apology at Sunday Pancake Breakfast, tears running down his face, his nose running.Think there is some bitterness???Think that increased productivity???
Crazy. Why is this allowed? Just because it’s not illegal?
They will get unions. Just like they had to start paying taxes. They deserve it.
I had read Jack Welch’s (auto)biography where stuff like this is mentioned (Neutron Jack, lopping off the lowest-performing 10% each year, etc.). Did not really internalize it at the time. (hate such words ending in -ize but can’t think of a better one right now). Of course there can be pros and cons, still, the idea of chopping of the bottom 5% or 10% routinely without individual evaluation / thinking / counseling / chance of remediation is a crappy retarded idea, IMO.In fact, I first encountered it as the concept / technique of “grading on the curve” [1], which was introduced with great fanfare by one of the teachers (chemistry) at the high school where I did my last few years of schooling. Even as a youngster, I intuitively thought it was a dumb idea at the time (although I was a top student in my class). (How about grading the teachers on the same curve, with the same rules, and rewarding / punishing them accordingly – as you say? Like to see their reaction to that. Bet they start “having kittens” (funny term) and go into hysterics.)Reducing people to statistics, and saying everyone’s performance (even in a small class of 20 or 30) has to follow the normal / Gaussian / whatever distribution. Really really dumb only-left-brain-think type of idea.[1] The same concept is found in some companies these days (think M$ is one, but not the only) under the term “stack ranking”. I’ve seen plenty of threads on HN where people hate and bash on that idea. with good reason.
I got rid of stack ranking at our acquiring company. Said it many times. Think of this…..you benefit by hurting other employees, you are hurt by helping other employees.You sabotage somebody and their ranking goes down???? You Win.You help somebody and their ranking goes up???? You Lose.Great fucking system.
>I got rid of stack ranking at our acquiring company.Good achievement.
Think that increased productivity???Remember what I said in some other comments:1) Food on the turnpike could suck in the 60’s and 70’s. There was no competition. You were locked in. If you wanted to eat you ate what they had. If the service was bad and the price was to high you still had only one choice. Anyone remember coffee in college libraries? (Not sure if by the time you were at Penn they still had those god awful machines that turned out bitter sludge or not the vending machines).2) You can mistreat people and/or lose customers by the boatload if there are either no alternatives or a steady stream of new customers or in this case employment applicants. That is why Amazon gets away with it. Plenty of people to take your job. (Reason in part there are unions..)By the way yes Amazon does suck (obligatory). But I also am guessing there is more to the story potentially.My Dad had an employee that worked like a mule. This guy was great. But he was also an alcoholic. Not saying that is the case here btw.There is one other dynamic as well. You are either hot or not. If you are Wawa you are hot. If you are Amazon you are not. People think you suck. Being a little nicer isn’t going to get you anymore. It’s like being the President. He has no reason to not be himself. Not like anyone who hates him will ever cut him slack. It’s all or nothing at a certain point. No sense in trying to please people when there are just so many things that you would have to correct.
No. People like him do that job so they can work during the day and make ends meet at night, loading those trucks.He did some work for me, after that. Was great.He told Paul he knew that he had to do what he had to do. He held no ill will. Maybe it helped things work out for the best, he said.There were many clenched fists and heads down during that exchange. It was painful. I know you find me funny, but I do enjoy the comrade of standing next to another man in our Catholic Fraternity washing dishes, as we raise money for widows and orphans. I have to keep up with him rinsing and the next man drying keeps up with me, and the next putting away. The bus “boys” need to move the dishes. I deeply resent people saying that we need to get rid of such fraternal organizations and try and tear us down.I hold ill will for Amazon about that. Still order, but try every which way not to. Picked up HRX mower blades from Home Depot (Cheaper) Coolers for boat from Monoprice.
A cynic. Oscar Wilde. Its from the play Lady Windermere’s Fan. (Wilde wrote price, not cost).
I would love to be in your mind for just one day. It would just be fascinating. I know that was not my original quote, but I would have absolutely no idea where it came from. I just love your ability to remember quotes. Thank you I will remember where that one came from now!
This is my favorite article about this topic:https://rickmanelius.com/ar…”Features are cheap, details are expensive”
I f’ing love this article. I mean love it.Thank You!!Umm…..can you just change it from GI Joe to Barbie…….I mean just a small change 🙂 I love it……..
When you’ve got Michelangelo painting the ceiling, you don’t send in a lot of house painters to give him advice. And you don’t have a bunch of staffers running around counting his brush strokes or the amount of paint he uses.Look, it’s simple: People who don’t know even dip squat or jack sh*t about software flatly can’t successfully micromanage software development. Blunt suggestion: Give it up. At best, a project will be successful in spite of your efforts.I’ve already posted in this thread some very hard won, solid wisdom on how to manage software projects. And as I hinted there, might also draw some wisdom from management of other large projects going back to, say, even the Pyramids.I’ll give you four examples from my career. In each case, the lesson is the same: Management, for your own good, get your GD dysfunctional, destructive, incompetent hands the F OFF my work in software.(1) I was in a software house near DC working on some projects for the US Navy. The Navy wanted a proposal from us, and I was working on that. Suddenly another project was behind, and starting on Monday I was sent for a week to see if I could do the impossible. Before lunch on Monday, I knew I couldn’t do the impossible. But there was a really nasty loose end in the proposal I was working on. The Navy engineers wanted to get the power spectrum of ocean waves, generate some such sample paths, and use them in an optimization of the SSBN missile launch hover control system. So, I called a Navy engineer and asked about what accuracy they wanted for the power spectra. He mumbled something about common engineering criteria of 10%. Well, the power spectra he wanted was a low frequency. It turns out, sure, similar to the Heisenberg uncertainty principle, that is, some basic Fourier math, that getting good accuracy at really low frequencies requires a really long stretch of data. Details are in Blackman and Tukey, The Measurement of Power Spectra which I’d been reading to be more clear on the issue. Tukey was long a grand, world class figure in applied math at both Bell Labs (sure, interested in power spectra) and Princeton. I had a lot of respect for Tukey because I kept running into his work — stepwise regression, exploratory data analysis, the fast Fourier transform, his statement equivalent to the axiom of choice, his “Convergence and Uniformity in Topology”, and, sure, his work on power spectra, and maybe some more. So, in a rush that week, I junked the assigned work on the impossible project, dug quickly into Blackman and Tukey, had already been quite good at the fast Fourier transform (FFT), including reading the book at dinner over broiled flounder in Silver Spring, and rushed to write some illustrative software to generate the sample paths, report the sample power spectra, and watch the estimates of the power spectra slowly converge to the assumed power spectra for the sample paths I was generating. Nice software. I had it running by Thursday evening. On Friday, I called the Navy engineer, and that evening we met at the computer center, ran my software, and saw the results — he needed a LONG sequence of ocean wave data but with that I had shown him good code for getting the power spectra and sample paths he wanted.I ended the week thinking nothing of the work.The next week the manager of my company asked how I’d done on the impossible project. I confirmed for him it was impossible. I said nothing about my power spectra work.That next week our company had a show and tell before the Navy on our proposal to write the software for their project. It turned out, a big surprise to me, I was the star of the show. So, I gave a quick review of the power spectra work I’d done.So, my power spectra work had given our company essentially sole source on the Navy’s work. Lots of people in the Navy and my company had been talking about my work on power spectra. Finally the manager of my office, really saying nothing specific about what I’d done, said “You are getting to be an important person around here.” Meanwhile I’d gotten a better job and was leaving, and one evening got a phone call from the manager, drunk, upset, that I was leaving. My leaving was about to cost him my victory on the Navy contract. But, he’d never wanted to communicate with me openly. I’d saved his back side, and he essentially just ignored what I’d done.(2) Soon FedEx called, …, and wanted me to solve their “most important problem”, writing software to schedule the planned fleet. So, I joined. There was this and that, but they needed results. I was also teaching courses in computer science at Georgetown so needed to do that, also. And, really, for good computer access, I needed to work from my home in MD, not in Memphis where FedEx was. So, I did: In a big rush, I designed, wrote, and ran the software needed. The severe concerns of the BoD were alleviated, crucial funding was enabled, and I’d literally saved FedEx. But the whole project worked because I wasn’t in Memphis, was able to both DESIGN and write the code, and some incompetent fumble bumblers in Memphis were not able to mess up the work. The design part was just crucial, permitted getting the crucially needed results in six weeks instead of some years. Later when I was in Memphis, the incompetents did try hard to mess up my work. I saved FedEx again, had the incompetents up on their hind legs screaming (mostly with jealousy), and left for my Ph.D.(3) A guy had a bright idea: A banking law had changed and suddenly permitted a lot of “cross selling”. Since the situation was sudden, the selling resources were essentially fixed. So, the question was, how best to allocate the selling resources? So, this was a case of what now might be called targeted marketing. Well, the guy and one of his buddies had formulated the problem as 0-1 integer linear programming. For a solution, they tried the latest, pop-tech, snake oil nonsense, “simulated annealing”. So, they had their software run for days and quit when tired. They got an answer but with no indication how good it was. Their 0-1 integer linear program had 600,000 variables and 40,000 constraints. I got their formulation, looked at it a little, and the next day told them that I thought I’d be able to get a good solution (I’d had some GOOD material in grad school!). The head guy made it clear that I was to report to his CTO, the turkey who had done the simulated annealing nonsense. So, sure, about all I could do with that turkey was just ignore him and did. So I did some derivations in non-linear duality theory, had some okay ideas, wrote the code, and got a feasible solution guaranteed to be within 0.025% of optimality. The work took two weeks. I kept the CEO informed: First I reported to him the apparent success of my applied math. Second I reported to him my plans for the software. Third I reported to him my success with the software and the problem in total. Never heard from him again! He and his CTO were so embarrassed, humiliated, and scared of my success that they would rather fail than be successful with me. NO WAY did the dumb de dumb dumb CEO-CTO want to admit that there was something crucial for their business that I knew but they didn’t.Later had a same song, second verse, similar situation with, it later turned out, some buddies of the banking marketing idiots: I saw that their problem, which looked like something impossible, was really just least cost, integer flows on a capacitated network. So, some work of W. Cunningham, one of my profs, provided a gorgeous solution. I was deep into writing the code when the CEO and his CTO buddy, both incompetents using nonsense heuristics, wanted to proceed their own way.(4) At IBM’s Watson lab, our group was doing some of what was popular at the time in “artificial intelligence” (AI = BS). So, we had a programming language, and I’d been part of the design of its “rule subroutines”, nice if believe in such “rules”. One afternoon one of our programmers was saying how much work he had to do to implement rule subroutines. His approach was a disaster — a lot of work, weeks, and would make a mess out of rule subroutines. Okay. I got some dinner, had some ideas, typed in and ran some illustrative code, wrote e-mail to the group, and, then, at dawn, went home for some sleep. When I got back at noon, the programmer was to be done by the end of the afternoon instead of the weeks and hailed me as a “hero”. Management had NOTHING to do with my work — I simply DID it. I got a lot of jealousy but also an official award.And I can relate more such. The story is the same: The managers don’t know even dip squat or jack sh*t about good technical work. They can’t do it, envision it, or manage it. When they get good work anyway, commonly they get jealous and/or scared. Management still believes that they are at a Ford plant 100+ years ago where they know all the big ideas and their subordinates are there just to add muscle to their ideas. Well, now for the good work, the managers long far from the technical work are NOT good with the work, can’t envision, do, manage, or even like good work.So, now I’m a sole, sole startup founder, with my own business idea, applied math crucial core, and corresponding software. So, that’s one way to get around the nonsense of technically incompetent managers of technical projects.
Could you be more specific? What do you mean you’re not getting enough out of Engineering?
i’m weary of measures like these, they seem pretty superficial and treat engineers like machines. velocity should be a lot more about delivering business value (impact, effort, priority, etc..). Engineers need to share ownership in that, especially as it applies to technical strategy, long term maintenance, complexity, etc… i’d much rather something take more coding days, like if an engineer pushed back at handoff, and steered us away from a maintenance nightmare. or in a more optimistic example, stopped to go through some user research and built something flexible enough to open some new doors for helping other people in the future. this is hard to accomplish when engineers are being judged by things like coding days, unit test coverage, pull requests, etc. many of these things are a lot less under their control then we realize, coming from bad design decisions, time-crunches, distractions and deadlines, etc. every company wants to move fast, and it seems like engineering slow, but to really move fast everyone has to build the right thing together over and over again.
i’m weary of measures like these, they seem pretty superficial and treat engineers like machines.This is exactly the type of ‘objection’ that should be incorporated into the code climate website. Let’s see if they do that (assuming they see this blog post). Plus other observations and free advice.
For sure. I wrote up some of our thinking about using data to optimize engineering and processes and teams in this comment: http://disq.us/p/1sqbkg2 And we’ll be continuing to explain that on our site.
“i’d much rather something take more coding days, like if an engineer pushed back at handoff, and steered us away from a maintenance nightmare. or in a more optimistic example, stopped to go through some user research and built something flexible enough to open some new doors for helping other people in the future.”This crisply defines the difference between having programmers are viewed and fell they are a part of a company and programmers that are just cogs in the machine, or consultants.I have been beating this drum for years:Maintenance Nightmare = Job for life, because you management can’t fixFlexible to help in the Future = No need for job for g-d programmersWe agree completely.
The reactive manager focus is on cost and failure points. Things that already happened.The proactive manager focus is on software architecture, complexity management and quality of process. Things that have not happened yet.
People solve for what you measure them on. Input focused KPIs like these can easily end up driving the wrong behavior including pointing blame when things don’t work for end-users. How do we know what’s good and bad? More lines of code and more pull requests may mean nothing, good, or, bad, depending on what they do to user adoption and success, and the implications on long term maintenance and extensibility.Measuring maker loops end-to-end (Product Mgmt+ Design+ Engg+ QA + actual usage) is better than arbitrary definitions of # of pull requests , test coverage, lines of code, etc. Stuff that happens upstream (customer development, product definition, design) and downstream (customer/developer usage and adoption) of engineering need to be seen together in a larger context.Engineering measurements work best when user adoption is instrumented into the product by design through embedded analytics. (Example: Tools like Gainsight for SaaS products). When “success” is driven by users, it is more likely to drive alignment across the maker chain.
Hi Ron — Thanks for sharing. We’ve built Velocity to focus on removing risks/bottlenecks in processes as well as continuous improvement rather than ranking/judging individual engineer performance. There are two key principles that underly Velocity…The first is that the happiest engineers work on the most productive teams and vice versa. The impact of this principle is that as a manager you could do much worse than to figure out what is driving your engineers crazy (slow CI, changing requirements after coding starts, long review cycles, etc.) and working to reduce those. Velocity (and some other tools) offers a concrete way to understand and take action on those things.The second principle is that quality and speed, which seem like they are opposing forces, tend to go hand in hand. There’s been some really interesting scientific research (https://www.amazon.com/dp/B… in this area over the past few years, but it shows that applying lean manufacturing concepts like lowing batch (deploy or PR) size and increasing throughput (tempo) is correlated with an improvement in metrics like change failure rate and mean time to resolution (MTTR) when a bad change is deployed.Hope that helps clarify the angle we’re coming at it from a bit. As with all data and measurement, it needs to be applied conscientiously, and we actively coach our clients in how to do just that.
agree, removing bottlenecks and risk is great. i guess it is how you apply the tool, but i think the goal should be to create a shared sense of what “productive” is across the org, and work on that process, wherever things need fixing. seems like there is a gap out there, engineers caring about engineering things, and management not understanding why engineering seems slow. my point is the more we can blur the lines and look at a single org trying to move value out the door, the more lean principles will work for the org.
That’s a Drucker line – you can’t manage what you don’t measure.Great CDN CEO – Gwynn Morgan – had a good line on this toos If it matters, measure it.I am with @PhilSugar – hold Engineering leadership accountable for delivery of macro goals. Let them decide what matters and how to measure it.
.It is never the data abstractly, it is the trend.Are you good and getting better?Are you good and getting worse?Make the trend your friend.Measurement is the second step. The first step is setting objectives.The most important step is rewarding performance.Whatever behaviors you reward will be duplicated.On a team, you have to focus on team outcomes, not personal outcomes. The MVP is not playing for the #2 team. Be #1 and guess what? The MVP is on your squad.JLMwww.themusingsofyhebigredca…
in my experience working with engineers this kind of tool would piss them the hell off, the exceptional engineers particularly. not saying its not needed or beneficial just that a softly softly approach works best with them.marketers, sales guys, bus dev – slap them with all the metrics and tracking tools on offer. engineers, stroke them gently.
Disagree with second paragraph completely. This is an us versus them theory. Just like if you “slap them” you don’t keep good engineers, if you “slap them” you don’t keep good sales and marketing. Why would anybody in their right mind who was good and have options go to a place to get “slapped” every day.
completely different mentality.sales guys eat those kind of targets for breakfast. they pride themselves on that kind of prowess.engineers are artists. they benchmark themselves very differently. – just my anecdotal learnings.
Sales people know their targets. How much did they sell. Easy.Can you measure if your top sales gal is working 80hrs a week or only 8??? Well you could, but what would she do if she was selling twice as much as everyone else??? Leave. Now you need to send some of those artists home.Marketing will never provide enough leads, and if you measure them on that they will set up systems where they get unqualified people as “leads”
.Pros measure results.Amateurs measure effort.JLMwww.themusingsofthebigredca…
engineers are artists. they benchmark themselves very differently.This is also the reason they (programmers) give away so much of their time for free. Kind of like artists or musicians. The benefit is seeing the pleasure of others and the reaction of others.I found this out in the 80’s at the company that I owned where when I did some programming was able to experience the delight of people who worked for me as it made their jobs easier. They would ask for some feature and I (their boss) would deliver it. No question the benefit was also how that made me feel. In later years I used that to get the most out of people that do work for me. Make sure they can see my delight and happiness with what they do. This works so well it’s almost scary and manipulative. My wife and I complement each other for things in a similar way and it’s become a joke with us ‘let me stroke you so you do that again’. (She once rearranged my bathroom closet for example..)But to Phil’s point in sales it’s not the same and mainly because of both the type of people who do sales but there is a way of keeping score (money made) that completely overtakes any ‘happiness’ shown in another way. Just dwarfs it.
Is this a good tool for engineering/tech team to communicate performance and needs to non tech management?
Not my Area but grateful for the learning curve comments where else would one find such quality openness………
(1) A lot of applied math and analysis says just what to do with good data when in practice there’s no chance of getting such data.(2) In part, “good data” are “measurements” with at least reliability and validity. Problem is, tough to get such data from Amazon, their AWS, Google, economic statistics from the US Federal Government, or anywhere else.(3) For productivity in software development and related activities, there have been gut wrenching, mind bending projects that went WAY behind schedule and WAY over budget and sometimes still didn’t work. So, long there has been no shortage of efforts to be able to understand, measure, monitor, analyze, manage, predict, get good estimates of time and cost, get good quality, etc. for projects, especially software projects. E.g., long famous and still one of the best isFrederick P. Brooks, The Mythical Man-Month: Essays on Software Engineering,, ISBN 0-201-00650-2, Addison-Wesley, Reading, Massachusetts, 1975.And the candidates are: (1) Hire brilliant programmers. (2) Pick a brilliant, experienced, successful programmer and appoint them as the “Chief Programmer” of a team, (3) Do Agile software development, (4) ship something each day, (5) …. And may I have the envelope please [drum roll]: And the winner is:For a proposed project, go to a team that has recently done at least 10 very similar projects and see what the time, cost, and quality have been and then take seriously their time and cost estimate for doing a similar quality project one more time.Otherwise no very accurate time, cost, or quality estimates are possible!More realistically, about all can do is (A) keep working hard, (B) be a little flexible and try not to get hung up for days on not very important stuff, e.g., the exact color of part of a Web page, (C) get the best documentation available, (D) occasionally hire a consultant for specialized work (e.g., for some high end software installations, might hire an expert who has already done that 50 times, has made a good business out of it, and quips “If you just follow the vendor’s documentation, then I’ll guarantee you absolutely it won’t work.”), (E) when clearly some little task is going slowly, investigate, add some information, expertise, smarts, thinking, etc., (F) at first stay with the KISS principle, “Keep it simple, stupid”, (G) document both problems and solutions, e.g., to be more clear what is going on and to do much better the next time, (H) as long as problems are being solved right along and the main line work is progressing without difficulty and nothing else is obviously wrong, be happy, (I) generally keep the team and the “burn rate” small, (J) start with a relatively clean problem statement and do the same for each project on the divide and conquer subtree, (K) don’t start to give up until have a rock solid reason, (L) have the project people, management, and ownership with “objectives aligned”, etc.People have done good project planning way back — the last effort at the Panama Canal, roads through mountains, big dams, deep tunnels, long bridges, tall buildings, the Large Hadron Collider, the Human Genome Project, the Manhattan Project, the SR-71, the F-117, the first US nuclear submarines, GPS, SOSUS, phased array radar, microelectronics at 14 nm, high bypass turbo fan jet engines, Gulf War I, D-Day, the B-29, the monolith from the upper Nile to a spot in Rome, the Pyramids, etc.I don’t see much hope for doing much better soon.Possible to do a LOT worse? Sure. Do a lot better? No. Sorry ’bout that.
GDPR is great for dumping unwanted email subscriptions.
Hello everyone — Code Climate CEO here. Thanks for the insightful discussion… Will be jumping into the comments below but also just wanted to say if anyone has any questions about Velocity, please let me know!
I would quit any organization that adopts Velocity.This is the kind of thing non-engineers and bad engineering managers adopt. Thinking quality software has anything to do with measures like this.
I am the Lead Developer in my organization. Software engineers are a tough bunch to manage and if you are not equipped they will eat you alive. A group of folk in software development came together and signed something called “The Manifesto for Agile Software Development” (http://agilemanifesto.org/). The manifesto notes:> Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.Agile organizations have adopted practices like continuous delivery and deployment where a tool like this can measure how efficient their developers are generally. It never hurts to measure and have data to identify and help your struggling engineers.However, one of the things to realize is that data is not a substitute for wisdom and knowledge. Without the appropriate context, this data is meaningless.
I have been doing this since 1992. The only thing that I have found works is divide down features and get a time estimate for each. Then decide how much time you want to do the release. Then decide what features you want. Add 10 hrs remove 10 hrs.I have found for the most part developers underestimate the amount of time.Add in time for testing, and preparing the release.The reason why management and boards think engineering is too slow is because they think: “If I just had all this stuff done my life would be perfect.”I want everything!!!It would be, but you know what? The world isn’t perfect. But if you keep working??? Then you win. It seems like “I NEED THIS TODAY” but if you just keep banging stuff out then today eventually comes.
I already in the thread for today wrote the truth about estimating time and cost of software projects: Again, once again, over again, yet again, one more time, this time just for you, there are two cases: (A) If the proposed project is close to what the team has already done successfully recently 10+ times, then can likely get good estimates of time and cost. (B) Otherwise no good estimates are possible. This is old wisdom; I didn’t formulate it; here I’m just repeating it; but I accept it.Believe (B). You can’t fix it. You don’t have good estimates. In case (B), even good programmers don’t have good estimates unless, in some cases, they are very fast learners, have good access to outside experts for details, can bend the project specifications, and can work 80+ hours a week for a few months. So, if the good programmers don’t have good estimates, then absolutely, positively you don’t either.For case (B), there are some things that can be done; I outlined a lot of them; you’ve outlined some of them. But, still, you don’t have good project time and cost estimates, nor will you, or anyone else.
This seems to mask the value of high achievers and give space for the less impactful engineers to hide.
Any success stories? What hidden learnings has it uncovered?