JC Herz - “For the most part, coverage of technology by journalists doesn’t get beyond ‘Oooh! Shiny!’”
from Ethan Zuckerman's blog
JC Herz - “For the most part, coverage of technology by journalists doesn’t get beyond ‘Oooh! Shiny!’”
from Ethan Zuckerman's blog
Scalability: Set Amazon's Servers on Fire, Not Yours
Don MacAskill, CEO & Chief Geek, SmugMug
his blog: http://blogs.smugmug.com/onethumb
slides posted there, download them
--> great presenation with hard numbers about how to outsource you data storage, processing and hosting <--
SmugMug
192 terrabytes data stored
140 million pictures
doubling every year
$.15 per GB w/ replicas
Why use amazon: not a lot of web-scale expertise on Planet Earth
helps focus on the app, not the muck
Guess: ~ would saved $500k per year
actual
- 64 m to 140 mil photos
- disks $40k to 100k per month
- $922k would have been spent
- $230k spent instead
- $629 in cold, hard savings!
nasty taxes: $295k 'saved' in cash flow! bonus!
- and can resell hard disks, recouping sunk costs
labor mot included, because they had to move 100 TB of data onto
S3 sweet spots
- perfect for startups and small companies
- deal for 'store lots, serve little' business of all sizes
- not so great (yet?) for serving lots if you're a medium or large business. Transfer costs high if you can buy bandwidth
Like SmugFS (file storage) previous:
- everyone building same
-pain to screw in servers
Lessons:
- hot and cold model just right, SmugMug hosts 'hot' data roughly 10% stored locally, s3 holds all of it
- 95% reduction in number of disks bought, no local backup and then only 10% stored local
outages:
3 outages, 15-30 mins, 2 core failures
and one DNS prob. Amazon.com affected
- 2 performance degradations, one our customer noticed, the 2nd they didn't.
- not a big deal, everything fails, expect it.
SLA, Service and Support
Amazon a little weak with customer support
launching ec2 implementation soon
- image processing
- 500k - 1 mil images a day
- 2 terapixles of processing
- ridiculously parallel
from O'Reilly old p2p site:
by John Scott, 05/21/2001
Who Will Make the Rules? Music Industry Faces Off at DC Conference
In what shapes up as an exhibition game for the Big League contests to come, music and movie industry lobbyists, artist representatives, a government official, and a reporter faced off in a debate on the future of digital entertainment. The outlook: threats, litigation, appeals, and (eventually) hearings before the Supreme Court. Most of the panelists who took part in this forum will no doubt be making their cases before Congress, the courts, and the media over the next several months and years.
I participated remotely for the Strong Angel 3 demo (news story here: Disaster tech, this is only a drill).
My big takes aways:
1. Communication is king: being remote really drove that point home, we were to provide reach back for the demo but weren't able to until the last couple of days because out there when the networks came up everyone (~800 people) jumped on the network.
2. Low bandwidth rules: we found that the high bandwidth tools didn't work as well.
3. Keep online tools simple: Groove had a workspace setup prior to the event for coordination, it worked pretty well before SA3 even though their were 80 people on it. Big problem was that it was meant to be used during the event as well where is became unusable - so much so people stopped using it, real bandwidth hog.
- instead IRC and a wiki should have been used for overall coordination
4. Virtualization: ROCKS! I have a MacBook Pro with parallels virtual machine running Windows XP, I needed it to access the Groove space. Having a big red button to kill XP when Groove locked up due to too many people using it was awesome.
5. Data: the virtualization and Groove space taught me that data needs to be accessible regardless of the operating system/platform. Not everyone has the latest laptop (or even a laptop) or cell phone.
6. Virtual data: data needs to be accessible on any platform.
Take aways:
1. data agnostic, no proprietary formats or lock-in to one platform especially for capabilities that are needed across the mission
2. community collaboration tools must be low bandwidth
3. keep it simple, some folks on the ground wanted to do everything in digital when a yell across the room would have sufficed
4. Who is who and who is doing what? social context is really important for when you have distrubuted collaboraiton going on, especially in an environment where you don't have time to meet everyone. Also hugely helpful for the next disaster.
software development:
- (in the government) needs to be more like a marketplace less like an assembly line.
- never has an end state, which is what out system is setup to acquire. It is alwasy changing, due to mision needs, resource, hardware base, etc.
- need to be more bottom up
I'm at OSCON06, where Tim O'Reilly has been talking about the concept of open data, which is much easier to explain versus open standards and interfaces. You should be able to use an application (open or proprietary) and pull you data out and walk away. Now a number of vendors like to lock their customers into their products so that the switching costs are very high.
But I think that if you really tried to implement open data some customers would be more willing to buy a proprietary product if they know they can walk way at any point.
you should attend this conference:
Open Source, New DoD Paradigm, or Business as Usual?
September 14 , 2006
Hyatt Hotel
2799 Jeff Davis Highway
Arlington, VA 22202
DUSD(AS&C), is examining how to deploy new technology development methodologies within DOD to increase the fielding rate of software and related technology based systems. AS&C is specifically focusing on Open Technology Development (OTD), which refers to technology and business process mechanisms to allow DOD-funded software code to be shared and collaboratively developed across DOD activities. This methodology would allow DOD organizations and contractors to rapidly adapt and extend existing software capabilities in response to shifting threats and requirements without, being locked in to a proprietary technologies/standard.
Downloadable here:
Open Source Software Institute
or
Open Technology Development Website
Great article about Vern Rayburn and his company Eclipse. Very, very important to aviation. Also a friend of mine helped foster this and other efforts at NASA and that team just won the Collier Award for their efforts. Eclipse won the trophy. which will be presented May 15 in DC.
Truely wil be disruptive, maybe to the military as well.
The best part is the manufacturing:
About the only thing Eclipse will do is snap the pieces together at its factory in Albuquerque. And even there, Mr. Raburn is counting on the technology to cut labor costs. Eclipse invested more than $30 million in a process called "friction-stir welding," which will enable it to save about 1,800 hours of labor on each airplane by eliminating 60% of the rivets that hold together the aluminum fuselage panels.
Eclipse plans to assemble each aircraft and have it in the customer's hands within 10 days of the beginning of final assembly. Twenty days later, Mr. Raburn will pay his suppliers.
Nice long and detailed article on data mining and looking for signals in the noise: Army project illustrates promise, shortcomings of data mining.
"In the spring of 2000, a year and a half before the 9/11 attacks, Erik Kleinsmith made a decision that history may judge as a colossal mistake.
Then a 35-year-old Army major assigned to a little-known intelligence organization at Fort Belvoir in Virginia, Kleinsmith had compiled an enormous cache of information -- most of it electronically stored -- about the Al Qaeda terrorist network. It described the group's presence in countries around the world, including the United States...."
Recent Comments