powdermonkey

my WSJ OpEd: Arthur Herman & John Scott: Send in the Tech Reinforcements

first WSJ oped: 

By ARTHUR HERMAN 
AND JOHN SCOTT

Today's Pentagon is losing its most important battle, the one for its own future. The Defense Department's latest evaluation report on the F-35 Joint Strike Fighter reveals how badly that fight is going, and why.

There have been many stories about the F-35 Lightning II's problems and delays, and its looming total price tag of $1 trillion, up from prior estimates of $331 billion and $177 billion before that. Still, the DOD's Jan. 14 report zeroes in on the key issue: software and information technology.

All five software "blocks"—systems to take the plane from testing to full production—that Lockheed Martin is contracted to develop for the Lightning II are behind schedule and over budget. The testing of software for producing F-35s won't get done until 2017. Any new glitches that are found will likely delay deployment even longer.

The fault, however, doesn't lie with Lockheed Martin, or even with the F-35.

The problem is how the Pentagon goes about acquiring the IT and software that modern weapons systems need. If this problem doesn't get fixed, any hope of building a 21st-century American military will be doomed.

The complexity of the software that the F-35 fighter needs is staggering, almost incomprehensible. It runs to almost 10 million lines of software source code. The man in charge of the F-35 program, Maj. Gen. Chris Bogdan, has been quoted as saying about thesoftware, "it scares the heck out of me"—because he knows in the world of high tech, complexity is the enemy of effectiveness.

Reuters/Lockheed Martin

The U.S. Marine Corps version of Lockheed Martin's F-35 Joint Strike Fighter, F-35B test aircraft BF-2 flies with external weapons.

Yet that's the dilemma the Pentagon faces every day when it comes to IT and software.

For example, the Air Force's Oracle -based Expeditionary Combat Support System was supposed to be ready in October 2013. Having spent $1 billion already, it needed another $1 billion just to get to one-quarter functionality by 2020. ECCS was supposed to automate the Air Force's management of parts and equipment. This inherent complexity of the acquisitions process effectively killed ECCS and led to the rampant cost overruns that also killed the Navy's DDG1000 destroyer late last year.

The Pentagon acquires IT and software-based systems the way it buys aircraft carriers—as if they were physical items to be forged or welded or mass-produced. The standard procurement cycle is geared around multiyear milestones and intensive evaluation reviews that can take months or years.

The modern software development cycle, by contrast, moves in weeks, days and even hours—because software is a malleable digital item whose only limits are the human imagination.

In addition, the Pentagon relies on a bureaucratic review process that demands contractors anticipate and resolve problems in advance. Every software engineer, however, knows that's not possible or even desirable.

Modern software constantly evolves. Engineers know that repeated test cycles in small portions at a time work best. Designers can learn from mistakes and glitches embedded in initial and early versions, and they can learn as well from early users of these versions.

More important, designers can take advantage of technologies that are unknown today but emerge later on. Learning by trial and error doesn't work so well for aircraft carriers or Abrams tanks, but it is crucial for the development of effective software.

The DOD's current acquisition strategy hasn't caught up or caught on. By treating software as if it were a product instead of a process, our military services are shutting themselves off from the kind of cost-efficient innovation that rules in the commercial software and IT industries. Amazon, for example, can make over 30 changes a week to its portal, from adding simple code changes to new complex features, without a major glitch. Our service personnel know this only too well, when they see how their children's videogames work better and have more sophisticated apps than the electronic gear they have to use in the field—and at a fraction of the cost.

The Pentagon has tried to go around the problem by buying off-the-shelf software for some systems. But that only postpones the inevitable frustration when it comes time to design software that can integrate those commercial products into warfighting systems. This is what happened when the Air Force tried to create the Expeditionary Combat Support System, which ended up as a $2 billion boondoggle.

Instead, the Pentagon needs a modern software and IT acquisition process that's as flexible, agile and open-ended as software itself—one that's geared for Moore's Law (computing power doubles every 18 months) and Butter's Law (network capacity costs get halved every nine months) instead of Murphy's Law.

Until we do, expect more messes like the F-35 and DDG 1000—which we ultimately can't afford and our fighting men and women ultimately won't survive.

Mr. Herman is the author of "Freedom's Forge: How American Business Produced Victory in World War Two" (Random House, 2012). Mr. Scott is a senior system engineer for Radiant Blue Technologies and co-wrote "Open Technology Development: Lessons Learned and Best Practices for Military Software" (Department of Defense, 2011).

A version of this article appeared Feb. 12, 2013, on page A15 in some U.S. editions of The Wall Street Journal, with the headline: Send in the Tech Reinforcements.

 

Transparency & Failure in .Gov/.Mil IT

Recently at the Gov 2.0 summit Fed CIO Viveck Kundra talked about stopping/killing some very expensive .gov IT programs, video here:

Gov 2.0 Summit 2010: Aneesh Chopra and Vivek Kundra, "Closing the Technology Gap"

But I was left wondering what were the programs and more over why killed? Oftentimes federal (and military) IT programs will be killed or left to die, without a postmortem or documeting lessons learned. Sometimes an intrepid reporter will try to one, but that usually just scratches the surface. GAO or internal Government offices will sometimes do them if asked to by Congress, but GAO reports are usually only for the largest of the large failing programs. .Gov internal reports are usually not published or done, much less published.

Its always struck me as weird that the government doesn't push more of a lessons learned culture like the one ingrained in the military (but sadly not in the military IT field). We do this for plane crashes, but there doesn't seem to be a systematic process for documenting IT failure?

(if there is please post)

I'm hoping that Viveck and Team can come up with some sort of simplified lessons learned document, this would do a few things:

1. Put government / contractors on notice that failure will be documented, like Gunnar says "The gov doesn't look good Naked" but it might be in better shape if it was

2. Learn, Learn, Learn: doomed to repeat mistakes if we don't learn

3. Gather requirements for potential changes in process, law, promotion schedules, etc. IF need be

A lessons learned doc should include: original cost of program (estimate and contractor proposed), original period of performance, winning contractor team, dollars spent, government intellectual property developed, reasons for cancellation, etc. I'd put a floor on the study and get a small team 2-3 weeks to prepare a doc, maybe only document failure under $ xx millions?

Or maybe this would be better as a wiki somewhere?

thoughts? unworkable? makes folks too uncomfortable?

Mil-OSS working group 2 wrap-up

(first published on www.opensource.com) Mil-OSS working group 2 wrap-up

Openness in the military is coming of-age, as was evident to the attendees at the 2nd Military Open Source Software (Mil-OSS) working group held August 3-5 in Arlington, Virginia. This grassroots gathering of practitioners in the art/science of creating military capabilities was unique in its inclusive atmosphere of government civilians, military uniformed (and retired) personnel, government contractors, and university academics. All gathered with one goal: how to more rapidly create and deploy military capabilities utilizing open source software for the U.S. Soldier, Sailor, Airman, and Marine.

What was interesting was a shift from the attitude at last years meeting of: "We can use open source software in the military?" to this years vibe of "How can we use, modify, sustain, and create more open source software around the military?" In the last year, since the update of DoD Open Source policy (see the 2009 DoD Open Source Memo), there has been an important shift in understanding that the benefits of being open vastly outweigh the hassles and foibles of closed and gated source code development--especially when the U.S. taxpayer is picking up the tab. Roadblock issues were brought up, such as:

- What are the best practices for how to run unclassified code communities for a classified client?

- How do we create, fund, and sustain development of open source?

- How do we generate more ideas on how to fix and deal with the government's acquisition system?

The military-focused OSS projects in attendance ran the gamut, from the long-established computer-aided design project BRL-CAD, to cyber (Suricata), modeling and simulation (Delta3D), geospatial (OMAR & OSSIM), and new projects such as EurekaStreams and the to-be-released Analysis of Competing Hypotheses. Presenters and the three tracks were almost too numerous; see MIL-OSS 2 Agenda for details.

The kickoff keynote was Lt. General Elder (retired), who reviewed cyber issues facing the military (video). Subsequent military keynotes discussed internal military community collaboration (Lt. Col. Allen of MilSuite), an update on the need to simplify information technologies acquisitions and deployment processes in the military (Maj. Neushul - Update on Keeping it Stupid Stupid), and a review of the current state of OSS in the DoD (Dan Risacher - DoD OSS Review).

Defensive Homeland security, OSS, and cyber issues were reviewed by Doug Maughan of DHS (Homeland Open Security Technologies) and contrasted by HD Moore of Rapid7/Metasploit who went over Open Source Cyber Weaponry.

Most presentations can be found on Slideshare at Mil-OSS WG2 Meeting. Selected presenters can be found on Vimeo.

A great addition at lunch were the many Ignite sessions, such as Gunnar Hellekson's "Fighting Forks [in DoD]" and Maj. Neushul's talk about the need for a functioning Combat Relevant Position Location Information (CRPLI) capability for the Marines.

An unexpected shout-out for WG2 came from Wired's Dangerroom (Noah Shachtman) who interviewed presenter Mathew Burton on his soon-to-be-released open source project: Analysis of Competing Hypotheses, which is "a system for organizing information that would help analysts think about complex problems more objectively, even when confronted with mountains of evidence." He called it Analysis of Competing Hypotheses, or ACH.

The ball seems to be rolling on releasing or converting military or intelligence source code as pure open source software.

By the third day, it was time for something unplanned: Barcamp, which basically split into two groups: projects and process. Project discussions included CRPLI and the Ozone Widget Framework (being used by the DoD Storefront project). The process track was mainly focused on issues around how to use, develop, and sustain OSS projects tied to the military.

A challenge was issued at the barcamp lunch in response to the need for a canonical set of briefing charts detailing the value of open source software for the military, from security to basic definitions to legal issues. All-in-all, about 100 briefing charts were created and will soon be made available to the community to use/modify/tweak as needed.

Future of Mil-OSS

The future of software in the military was a much discussed and argued topic, but one key issue agreed upon by all was the continued need for community building and collaboration. There was consensus for a Mil-OSS large annual gathering coupled with regional grassroots meetings and/or barcamps. There are rumors of proposed camps in Boston, New Orleans, Charleston, and San Diego. Join the Mil-OSS Google Group to track their progress.

A few reference links for the uninitiated:

• Website: www.mil-oss.org
• Mil-OSS WG2 presentations - http://www.slideshare.net/event/miloss-2/slideshows
• Mil-OSS WG2 videos - http://vimeo.com/channels/miloss
• Mil-OSS WG mailing list
• Mil-OSS LinkedIn group
• twitter hashtag: #miloss

Thanks to our amazing sponsors. We really couldn't have put on a great show without you!

And a big thanks to Georgia Tech Research Institute, especially  Josh Davis and Claire Gould, for doing the truly thankless back (breaking) office work of coordinating everything!

Make Love not CyberWar

Been thinking allot about cyberwar and the orgy of funds raining down upon the tribes of defense contractors and I got to wondering, why fight a war you can't win.

Now try to follow me: the US made it a central part of our national strategy to own the skies, I think the last soldier to die from an air to ground strike was in the Korean conflict. So we spent the funds, developed the tech, trained the pilots and can now pretty much own the sky above any country we want to. We decided that we didn't want any competition in the air and wanted to take away that option from any military.

So cyberwar: we are going to attempt to do the same things, spend the funds, develop the tech, do the training, etc. and we'll own the net. BUT of course the net is this large amorphous blob of foreign governments, public and private entities that don't like to take direction from a central source. In effect we can never own the sky. In this case the lessons learned of 'rule the skys' doesn't work and never will because you can never bound the problem of cyberwar. So what to do?

Don't fight a cyberwar by making sure no one else can either.

Most cyberwar exploits weaknesses in existing technologies (servers not protected, patches not installed, bad security, bad archtecture (internet issues), etc.), so turn that technology that deters cyberwar into a commidity for all to use, make it better and move on.

It's a bit counterintuitive but if you can't bound the problem then you will be constantly in the mode of 'wack a mole' hiting this and that group of nitwits or worse state sponsored folks looking to disrupt things. Or the ole Pearl Harbour attack, etc. This would entail pushing open source software, making it better and publishing it for all (yes including the baddies) to use.

So don't fight this war you can't win or at least make it really hard to see any results fast. Over time the systems should get better so that you can trace back a cyber attack and then drop a bomb on them since we own the sky.

where is Technology Transparency (t2)?

There has been allot of talk about government transparency, which is good, more data to citizens is better than less, etc.

What about technology transparency?

As a taxpayer I would like to be able to use some of the code my government pays for the creation of (unless classified). As a contractor to the military I would like to be able to examine source code for the multiple technologies that are being integrated together to provide the best capability possible for soldiers, sailors, marines and airman in the field and ensure it works, especially if that source code was paid for with taxpayer funding.

The DC CTO and the Sunlight Foundation have recently held contests about creating mashup applications for various data sets, the thing has has made these contests possible is that existing source code was available to change, modify and add to. Otherwise everyone would be creating applications from scratch greatly slowing the speed of innovation. Another great example is Mashable, which has snippets of source code available for use and reuse.

Technology transparency also enables individuals and organizations to learn how the source code functions and maybe ensure that nothing nefarious is going on behind the scenes.

So I hope in the whole transparency discussion going on Transparency of Technology is included in the discussion is included. 

Mobilizing Industry in WW2 & Compare to Today

just finished Mobilizing Industries in WW2, big take away that is pertinent to the new POTUS is clear lines of authority.

Repeatably throughout the book FDR created Offices, Commissions, etc. to run the various aspects of a wartime economy: all clashed with each other, all wasted time (most importantly FDR's), all fought the Army and Navy (while both were trying to take over aspects of economic planning), Congress and special interests were in the mix as well and all made decisions with out working out the impact a decision may have on other aspects of the economy.

Like: defer agricultural workers or people building planes (AG workers had a deferment, so more folks became farmers - manufacturing workers had no deferment, so instead of building a tank, they could be drafted to drive the tank).

There were roughly 125 commissions, etc. created - some subsumed others or were broken off. It didn't get fixed untill May 27, 1943 (Exec Order 27) with the appointment of James F. Byrnes as director of the Office of War Mobilization. FDR called him "the assistant President" and stated that Byrnes decisions were final (freeing up Roosevelt for Grand Stategy efforts). Byrnes portfolio encomapsed everthing expect foreign policy and military grand strategy.

My one concern now with Pres. Obama is that he seems to be building an analagous situation where there are too many chef's or CxO's, TARP managers, Congressmen, Senators, Cabinet Secrataries, etc. with too many overlapping responsibilities reporting or trying to influence him one way or the other. Endless meeting could result...

Applicable lessons from WWII were:
1. Personalities matter and must have the full confidence of the President. Byrnes knew how to get things done in Washington and wasn't too fully versed in econ matters.
2. Domestic and partisan politics will intrude on everything
3. Planning is essential

Joint Light Tactical Vehicle award this week: it won't be open I bet

I saw this story:
BMW seeking partners for open-source car software platform and then insidedefense.com had the story: "The Defense Department this week will announce technology development contract awards in the Joint Light Tactical Vehicle [JLTV] competition, according to the Pentagon acquisition chief.

And then talking with Jim Stogdill and reading his excellent post, I was left feeling its too bad that for all the millions that will go into JLTV, its too bad DoD didn't levy some requirement for the operating system to be more open, as in made open source. (Note I am less worried about vehicles, more so about software intesive systems that go inside them)

DoD is now going pretty explicitly to a winner take all model of acquisitons (2 pilot projects and then they pick a winner that goes into full production). Now this makes sense for big iron, physical goods (need to build factories, etc.), but for software its the the wrong, very wrong model.

When DoD picks a winner for a system that is heavily software reliant the Government is placing a bet that the Contractor Team will stay at the leading edge of technology. This never happens, Contractor Teams that win slowly build a Fortress to keep ideas out, while locking the Government into their platform. Then you end where we are now, with weird data standards, legacy systems, etc. Further, 1 team will win, but the loosing Team will have some better ideas or capabilities that the warfighter would love to have on their platform, but won't get because it would mean the Winner would ultimatley get less of the pie.

So, the industrial base shrinks, a division at a company goes out business, etc. OR as an alternative the loosing team could make open source their platform: kind of a 'loose the battle, but win the war" strategy.

Practical guidence for the Gov would be: 1. open the platform up (put an RFP out to make/deliver/build an open source platform or make OSS an existing one). 2. build a community around a capability. 3. let small contracts out for incremental improvements in platforms. 4. simplify the intellectual property rights lanscape for contracting with the government so that a developer can understand them

GOSCON08: NASA-Cracking the Code on Government Procurement Requirements; Open Source BI Fuels Jet Propulsion

notes from GOSCON 2008 session:
Cracking the Code on Government Procurement Requirements; Open Source BI Fuels Jet Propulsion

using JasperSoft for business intelligence on the ground stations, as well as an enourmous amount of open source throughout the enterprise. OSS is on the mars landers as well

Pentagon: Open source good to go

Not good to go yet, but soon hopefully...
Pentagon: Open source good to go

Military IT folks wondering if their use of Apache, Perl, Linux and other open source software is copacetic with the brass will soon get some answers from the Defense Department's Office of the Chief Information Officer.

The office is preparing a memorandum that further clarifies how open source may be procured and used within the services.

The memo should answer many lingering questions still surrounding the open source, said Daniel Risacher, the data strategy leader for the Office of Secretary of Defense who is drafting the memo. The draft may point out some potential benefits as well.

"Those factors that are in favor of open source have not been appreciated to date," said Risacher, speaking at the Red Hat Government Users and Developers conference, being held today. The DOD CIO office is aiming to release the memo by early November.

From Risacher's description of the draft, the memo may reinforce the acceptability of using open source software within the Defense Department, as well as for other federal agencies. It may even broaden procedures for procuring commercial software.

"Those mandates [in which] we have to consider commercial off-the-shelf software, we have to apply that to open source software as well," Risacher said. "And that is not well appreciated within government."

Risacher said that he first started working on the memo last summer at the behest of the Defense Deputy CIO, David Wennergren. Although widely used in federal government, open source software, due to its unusual form of distribution, has raised questions among regulation-minded program managers.

Forward leaner of the week: NSA & Open Source Software

its interesting that the most secretive government agency is more concerned with software quality than keeping everything complete locked down. via slashdot...

NSA shows the way to develop secure systems

Posted on 06 October 2008.

The development of highly secure, low defect software will be dramatically helped by the release of the Tokeneer research project to the open source community by the US National Security Agency (NSA).

The Tokeneer project was commissioned by the NSA from Praxis High Integrity Systems as a demonstrator of high-assurance software engineering. Developed using Praxis’ Correctness by Construction (CbyC) methodology it uses the SPARK Ada language and AdaCore’s GNAT Pro environment. The project has demonstrated how to meet or exceed Evaluation Assurance Level (EAL) 5 in the Common Criteria thus demonstrating a path towards the highest levels of security assurance.