Guest Post from LabVIEW R&D Director David Fuller – Stability vs Annual Releases, chapter 2

David Fuller is one of my counterparts in the LabVIEW R&D team who gladly accepted my invitation to respond to this discussion from the R&D perspective.  David started his career at NI as an intern in our instrument driver development team about 16 years ago - so he started from day one eating our own dog food.  Today, he is one of the two key R&D Directors responsible for LabVIEW development.  As he is a good friend of mine, I will share his post unchanged as he sent it to me, complete with playful jabs (deep down, I know he wants to move to Marketing some day):

Thanks. While I am not the Wolf to Pasquaratte’s Vincent, I can weigh in on the R&D view of quality, the yearly release cycle, and the degree of openness in R&D and marketing for releases, bug fixes and so on. I do happen to work for Bellin, but I won’t suck up to him like JP does to Zogas.

It is no big surprise to anyone that we now release a major version of LabVIEW at NI-Week each year. It just makes business sense for us. From the R&D perspective, we have been pleasantly surprised with the positive side-effects on our product development. First and foremost, the yearly release cycle forces us to be more disciplined while adding new features to LabVIEW. In years past, we had the notion of the “no turning back feature.” A no turning back feature is the kind where development changes are so pervasive that we have to touch most of the code base and its core architectures. Once we started, there was no turning back and the release became gated by the feature. The teams responsible for the feature would then go on a classic death march and work like heroes to get the thing done. The psychology of the heroic effort, the long hours, and the sense of shared purpose all have some positive characteristics, but the negatives of burned out minds, cut corners, and reduced quality don’t make for a sustainable way to develop LabVIEW. Now, because our release date is fixed, all features must either clearly fit the development window, be staged in, or turned off easily and safely. For example, DFIR or data flow intermediate representation, is one of our more difficult efforts aimed at consolidating our compiler architecture. We have actively developed DFIR for almost two years and parts of it were present, but inactive, in LabVIEW 8.5 and 8.6. We believe the architectural changes will make this year’s LabVIEW release, but even so, we are able to disable it safely if we need to. Its a clear example where we are trading off better quality versus our rate of innovation. If I think back to the LabVIEW 8.0 days and how we were operating then, I think we would have jammed it into LabVIEW and released it before it was ready for prime time. I do think we have started striking a better balance between getting in new interesting and useful features and keeping the subjective quality bar high.

Aside from our improved feature development discipline, the yearly release has improved coordination between R&D groups and between R&D and marketing. (Perhaps its freed up time so that the leaders in marketing have time for blogs, but I digress.) The NI-Week release creates a near company wide set of testing milestones. While it may seem obvious, when we release different hardware and software at different times throughout the year, the testing is localized to the group doing the releasing and there is a greater likelihood that some important interaction won’t get the right test coverage. Creating R&D-wide synchronization points, ensures a lot more test coverage for a lot more product combinations. Once we went through the platform-wide release, we realized it would make sense to ship all of the software tested at the milestone and this led to the SingleDVD project (that and a painful 120 slide show from marketing showing the customer installation experience when using cRIO with LV RT, FPGA.) Ironically, when we created the feature, we called it SingleDVD, but LabVIEW, the modules, the toolkits, and the drivers now take up at least three DVDs and counting.

By coordinating nearly of all NI’s R&D around NI week, you get a higher quality product because of our improved feature development discipline and through the concentrated and coordinated testing of the platform. So long as we remain disciplined enough to not force features into the release, I think we are now striking the right balance between innovation and quality.

Finally, Per Milan’s response, I really like the idea of targeted feedback mechanisms and better bug reporting within the product. As we are working on some of our features related to connecting desktop LabVIEW to NI’s business systems, we will look to see what we can do on that front.

Regards

-David Fuller

Director of LabVIEW Platform

LabVIEW Stability vs Annual Releases

In a recent info-lv post, a user was commented on the stability of different versions of LabVIEW and added this:

I hope this reply is read by the person at NI who makes the tradeoff between marketing and the necessity to create a “new” version of LV yearly vs. stability & reliability. I have great respect for the programmers at NI and have full confidence that they’re able to produce a highly reliable product; but not with a looming marketing deadline and huge feature list to implement yearly.

Obviously yes, your hopes that someone at NI reads your post have in fact come true.  “Stability” is a hard thing to measure, but I felt like the info-lv post with users weighing in on their impressions of stability of various versions of LabVIEW were right on the money with how we would respond.  However, I have some comments that I will share about the benefit of a more regular release schedule.  In addition to my comments, I have also invited the LabVIEW R&D guys to weigh in on this too with their opinion – so stay tuned for an upcoming guest blog post.  (I will try to keep these comments brief and leave room for R&D and our users to comment):

1.  An annual release improves stability –  In the past, we’ve gotten locked into huge development efforts with long cycles – like LabVIEW 8.0 – which has resulted in some stability challenges.  The fact is, LabVIEW 8.0 was such a huge release with so many features – many of which involved some very deep and fundamental areas of the product - that we took a hit.  8.0 was a long development cycle that was very painful to get out (I’m sure you’ve all been on “deathmarch” projects as well). With a team as big and complex with as many dependencies as we have with LabVIEW, it’s almost impossible to operate with open-ended schedules.  What tends to happen when you do that is that some particularly important feature becomes the critical path, and while we are finishing that feature, everyone else naturally starts to squeeze in new things.  As new things get added, we have to go back and redo some testing and validation.  A vicious cycle ensues.  The longer testing cycle means new things can get added… etc.  I think this is one of the things that impacted our 8.0 quality.  Not to mention the fact that when there are THAT many new features added to the product, its very difficult for our users (and our marketing department) to fully understand and take advantage of them.
2. Annual releases are critical for alignment across all of our internal (and external) constituents.  With an annual release, we now have moved to a much more predictable schedule.   Admittedly, we’ve added a little overhead in testing and releasing every year.  But we removed a lot of the unknowns that we struggled with in the past.  

   We no longer have these exchanges: 
   Marketing:  “How’s the new version coming?”
   R&D: “Great”
   Marketing: “Is it going to be ready by NIWeek?  We need to prime the pump before then?”
   R&D: “I’ll let you know when its ready?”
   Marketing: “hmmm…  We better come up with a backup plan for the keynote”

   All joking aside, we now have a standard schedule and the train leaves the station every year at about the same time.  The developers know that key features need to get slotted into a release- if they miss the train, they know the next one is coming around a year later.   When we work on larger, multiyear development efforts, we do need to manage the rollout in a more sophisticated way – at times, we test and ship code that is a foundational element for something that users won’t take advantage of until the following release – we just have to roll it out piecemeal that way.   

3. We need a regular release schedule to accomodate the diversity of the LabVIEW platform components.  Different parts of LabVIEW have to move at different speeds.  For LabVIEW users who are using GPIB instruments only, they probably can wait two or three years for new features to the environment.  For less mature parts of LabVIEW, such as LabVIEW Real-Time or FPGA, we have to move pretty quick to continue closing the gaps we have there in functionality, usability, and hardware support.  We are making much bigger strides there on an annual basis.
4. Customers prefer knowing our release schedule - We haven’t explicitly communicated to our customers our strategy shift to annual releases – we wanted to get through a few of them to see how it went before proclaiming success.  However, I have found that many of our customers appreciate knowing the basic plan (new features followed by maintenance releases) on our side to help them manage their own development strategy. A completely different topic is how and when customers upgrade their LabVIEW-built applications.  The fact is that there are many factors that go into this decision.  Sometimes, there are key features they want to take advantage of, but in most cases, the reason customers upgrade is driven by risk, timing, and hardware issues on their side.  This is a known area that we are focusing some resources to help, but I think smaller, more regular releases will ultimately help smooth this process – but that is a different blog post. 

In all of this, there are small set of requirements that we work very hard to maintain and improve upon for every release:

• Every version of LabVIEW needs to be stable
• Upgrading from one version to another should be a smooth process – we should not break your code if at all possible
• Everything should work together when you move to a new version – when you upgrade, you need validated versions of all the toolkits and modules as well (the new DVD in 8.6 goes a long way in simplifying that issue).

Customer Affinity – How are we doing?

A recent marketing study identified customer affinity as the new measure of marketing impact- rather than talk about brand awareness or sales, etc – the more important issues are your relationship with your customers.  Do your customers see that relationship the same way that you (the vendor) sees it?  This is one of the things I hope to get a better feel for through the efforts of this blog.   Because of our direct sales strategy and our long history of customer support (staffed by engineers who eventually move into sales, marketing, or engineering management positions) – I think we have a pretty good feel for what our customers want.  With that said, I am sure it can be better – and with today’s Web 2.0 technologies, we can have a much more accurate look at broad-based opinions and issues from our customer base.  The other topic that is getting a lot of discussion these days is “customer-driven innovation” – how can we get you more involved in the directions we are taking our products and technology?  I hope to get more active in these areas starting in 2009. 

It’s interesting (scary) to see some of the findings from this study, such as:

• 56% of vendors perceive themselves as being extremely customer-centric, but only 12% of customers agree.
• An overwhelming majority of vendors – 85% – are convinced that they are getting better at responding to customer needs, but 45% of customers disagree.
• More than 30% of customer respondents said they would terminate relationships with companies that fail to gain their trust; 62% would scale back existing engagements; and 7% would no longer consider the vendor for future business.
• Co-innovation with customers is vital to building customer affinity: Nearly six out of 10 customers say co-innovation is extremely or very important, with another 30% agreeing that it is at least somewhat important. Customer responses indicated that collaborative, two-way conversations – followed by continuous improvement – build customer affinity.

Ouch!

Resolution #6: Use the Web to Capture User Input for LabVIEW

There are a number of ways that we can use modern web technology to better capture the opinion of the LabVIEW user community collectively – from Wikis to voting systems, we want to put more stock in the voice of the community to help us make the right decisions in development.  This will take some time for us to put in place, but we hope 2009 is the year we start to open up our development priorities for more direct user input.

Resolution #5: Make Upgrading and Deploying Easier

In 2008, we started to focus more energy on what I call “the upgrade problem.”   I have talked to many customers who have run into some nasty challenges with upgrading their complex systems to the latest version of LabVIEW.  The reality is that there are a number of issues that go way beyond the version of LabVIEW you might be using.  There can be issues with drivers, the OS, the APIs you are calling, and the hardware used in your applications, not to mention 3rd party addons, third-party hardware, and your user code.  The fact is that every customer has a different set of circumstances that determine how and when they can move to a new version of LabVIEW.  We can’t control most of those circumstances – but we can try to make the process smoother when they do m0ve their application.  This isn’t an easy technical problem, but I think we can do better than we have up to now.  In 2008, with the launch of LabVIEW 8.6, we put a more focused effort on publishing recommendations, best practices, and general documentation about upgrading to the new version.  We hope to continue this effort in 2009 with better documentation and tools, and a tighter focus in R&D as well.   (One cool idea that we introduced in NI Labs was an Upgrade Analyzer - a utility that would inspect your LabVIEW code and attempt to flag “issues” that you might run into when you upgrade – such as APIs or individual VIs that have changed or structures that have been tweaked.  It hasn’t really caught on yet, but I think it has some potential). 

The other area that I have found myself in a lot of discussions about recently is about deployment – how do I replicate my system from my development machine to the field?  How do I replicate my golden machine to 20 (or 200) machines in the factory?  How do I manage a TestStand and LabVIEW system in the production facility?

We have a R&D cross-functional team looking at all the challenges with deploying applications on the desktop as well as in real-time and embedded platforms and we hope to make some real progress in 2009.